Image denoising method, device and equipment and computer readable storage medium

Abstract

The embodiment of the invention provides an image denoising method, device and equipment and a computer readable storage medium. The method comprises the following steps: acquiring a to-be-processed image; preprocessing the to-be-processed image to obtain a preprocessed image; the preprocessing comprises at least one of global brightness enhancement processing, corner brightness enhancement processing and local region compression processing; performing image de-noising processing on the pre-processed image to obtain a de-noised image; and carrying out inverse processing corresponding to the preprocessing on the de-noised image to obtain a target output image. According to the embodiment of the invention, the accuracy of image denoising can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and relates to but is not limited to an image denoising method, apparatus, device and computer-readable storage medium. Background Art

[0002] Image noise refers to unnecessary or redundant interference information in image data. The presence of noise seriously affects the quality of the image. Therefore, it is very necessary to effectively remove image noise.

[0003] At present, deep learning is widely used in various image processing problems, and its excellent data fitting ability makes its effect far superior to various traditional algorithms. Therefore, deep learning is also widely used in image denoising tasks. However, on mobile phones and other terminal devices, their hardware resources and battery capacity are limited, and too large a network scale cannot be effectively applied to these devices. In this way, when the image denoising algorithm is applied to the mobile phone, the accuracy of image denoising will be affected when a lightweight image denoising model is used for image denoising. Summary of the invention

[0004] The embodiments of the present application provide an image denoising method, apparatus, device and computer-readable storage medium, which can improve the accuracy of image denoising.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image denoising method, the method comprising:

[0007] Get the image to be processed;

[0008] Preprocessing the image to be processed to obtain a preprocessed image; the preprocessing includes at least one of global brightness enhancement processing, corner brightness enhancement processing and local area compression processing;

[0009] Performing image denoising on the preprocessed image to obtain a denoised image;

[0010] The denoised image is subjected to an inverse process corresponding to the preprocessing to obtain a target output image.

[0011] In a second aspect, an embodiment of the present application provides an image denoising device, the device comprising an acquisition unit, a processing unit and a denoising unit; wherein:

[0012] The acquisition unit is used to acquire the image to be processed;

[0013] The processing unit is used to preprocess the image to be processed to obtain a preprocessed image; the preprocessing includes at least one of global brightness enhancement processing, corner brightness enhancement processing and local area compression processing;

[0014] The denoising unit is used to perform image denoising on the preprocessed image to obtain a denoised image;

[0015] The processing unit is further used to perform inverse processing corresponding to the preprocessing on the denoised image to obtain a target output image.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0017] A memory for storing executable instructions;

[0018] The processor is used to implement the above-mentioned image denoising method when executing the executable instructions stored in the memory.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores executable instructions. When the executable instructions are executed by one or more processors, the image denoising method as described above is implemented.

[0020] The embodiment of the present application provides an image denoising method, device, equipment and computer-readable storage medium, the method comprising: obtaining an image to be processed; preprocessing the image to be processed to obtain a preprocessed image; performing image denoising on the preprocessed image to obtain a denoised image; performing inverse processing corresponding to the preprocessing on the denoised image to obtain a target output image. By performing at least one of global brightness enhancement processing, corner brightness enhancement processing and local area compression processing on the image to be processed, the denoising effect in dark light scenes and corner positions can be enhanced, and the local area compression processing can effectively improve the performance of the network without much degradation in image quality, thereby improving the accuracy of image denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0023] Figure 1 A schematic diagram of an optional image denoising method provided in an embodiment of the present application Figure 1 ;

[0024] Figure 2 A schematic diagram of an optional image denoising method provided in an embodiment of the present application Figure 2 ;

[0025] Figure 3 A schematic diagram of the structure of an optional image denoising model provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of an optional image denoising method provided in an embodiment of the present application Figure 3 ;

[0027] Figure 5 A schematic diagram of an optional image denoising method provided in an embodiment of the present application Figure 4 ;

[0028] Figure 6 A schematic diagram of an optional image corner brightness provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of an optional image denoising method provided in an embodiment of the present application Figure 5 ;

[0030] Figure 8 A schematic diagram of an optional coordinate mapping provided in an embodiment of the present application;

[0031] Fig. 9 A schematic diagram of an optional compression relationship of a single-dimensional coordinate provided in an embodiment of the present application;

[0032] Fig.10 A schematic diagram of a framework of an optional image denoising method provided in an embodiment of the present application;

[0033] Fig.11 A schematic diagram of the structure of an optional image denoising device provided in an embodiment of the present application;

[0034] Fig.12 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0037] In the following description, reference is made to “some embodiments”, “this embodiment”, “embodiments of the present application” and examples, etc., which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0038] Image noise refers to unexpected, random, unnecessary or redundant interference information in image data, which is usually caused by various factors in the process of image acquisition, transmission, processing, etc. Image noise can cause image details to become blurred, clarity to be reduced, color to be distorted, edges to be unclear, etc., thus affecting the quality and usability of the image. Therefore, it is very necessary to effectively remove image noise.

[0039] Generally speaking, noise is divided into the following three types:

[0040] 1) Shot noise: photons hit the photosensor and generate electrons through the photoelectric effect, but the generation of electrons has errors. For photons of the same intensity, the distribution of the size of the generated electrons satisfies the statistics of the imperceptible distribution. The noise caused by this error is called shot noise.

[0041] Specifically, shot noise is the noise caused by the randomness of the photoelectric effect when photons hit a photosensor to generate electrons. The energy of the photons will generate electrons in the photosensor, but due to quantum properties, the arrival of photons is random, so the number of electrons generated by each photon is also random, conforming to the Poisson distribution. Due to the randomness of photons, even under the same conditions, the number of electrons measured each time will fluctuate to a certain extent. This fluctuation will cause the brightness of the image to fluctuate randomly on a small scale, which is the manifestation of shot noise. In low-light conditions, shot noise may be more obvious due to the small number of photons. Shot noise is an important factor in photography, imaging, and research in quantum optics, especially in low-light conditions, where it will have a significant impact on the quality of the image.

[0042] 2) Read Noise: Electrons are converted into voltage, and the voltage is read out as a specific value after passing through an analog-to-digital converter. The error generated during the reading process can be called read noise. Read noise generally includes two aspects: one is the black level (Dark Current) (also called dark current), and the other is the noise that satisfies the Gaussian distribution.

[0043] Specifically, read noise is the noise generated when converting electrons in a photosensitive sensor into voltage and performing analog-to-digital conversion. Read noise includes two main aspects: black level and Gaussian distribution noise. For black level (DarkCurrent), in the camera sensor, there will be some spontaneous electron flow even without light exposure, which is caused by factors such as temperature. These spontaneous electron flows will cause the sensor to generate a certain voltage, which will appear as a certain brightness in the image, that is, black level. Black level will introduce unnecessary brightness in the dark area, affecting image quality. Read noise also includes Gaussian distribution noise, which is caused by uncertainty in the analog-to-digital conversion process and randomness in electron transmission. This noise will cause subtle random fluctuations in the image, usually presenting a noise pattern that conforms to Gaussian distribution.

[0044] 3) Other noises. Here we simply classify other noises into one category because this type of noise basically does not conform to statistical laws. It includes thermal noise (electrons generated by sensor heat), pixel response non-uniformity (non-uniformity of pixel response to photons), quantization noise (the readout voltage is floating point type and needs to be converted to integer), and fixed noise of the sensor (related to the sensor process).

[0045] According to the imaging process of the image sensor, it can be clearly seen that in some scenes at night and in low light, as the sensitivity of the imaging camera increases, the noise of the image also increases significantly. At this time, the increase in noise affects the acquisition of effective image information. Therefore, it is necessary to effectively apply image denoising solutions to complete image denoising.

[0046] At present, deep learning is widely used in various image processing problems, and its excellent data fitting ability makes its effect far superior to various traditional algorithms. Therefore, deep learning is also widely used in image denoising tasks. However, on mobile phones and other mobile devices, their hardware resources and battery capacity are limited, and too large a network scale cannot be effectively applied to these devices. Therefore, it is very important to effectively improve the effect and performance of deep learning in these resource-constrained devices.

[0047] In order to effectively apply deep learning to denoising tasks on mobile devices such as mobile phones, there are several image denoising solutions in related technologies. One is to design a more lightweight network and operator to improve the network's operating efficiency while improving the network effect. The other is to appropriately compress and prune the large network so that it can be better used in mobile applications.

[0048] However, although the relevant technologies have tried network structures and other compression and pruning methods, they are still limited by the resources of mobile devices such as mobile phones. The final considerable benefits are not particularly significant, and the accuracy of image denoising needs to be improved.

[0049] The image denoising method provided in the embodiment of the present application is executed by an image denoising device and an electronic device, wherein the image denoising device can be stored in the electronic device in the form of a software function model, the image denoising device can also be integrated in the electronic device as a hardware function module, and the image denoising device can also be combined with the electronic device in software and hardware to implement the image denoising method, and the present application does not impose any limitation on this.

[0050] In an embodiment of the present application, the electronic device may be a server, which may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms, but the embodiments of the present application do not limit this.

[0051] In addition, the electronic device can also be a terminal device, which can be a mobile phone, a tablet personal computer (TPC), a media player, a smart TV, a laptop computer (LC), a personal digital assistant (PDA), a personal computer (PC), a camera, a camcorder, a smart watch, a wearable device (WD) or an autonomous driving vehicle, etc., which is not limited to the embodiments of the present application.

[0052] An embodiment of the present application provides an image denoising method, the basic idea of ​​which is: first, an electronic device acquires an image to be processed. Then, the image to be processed is preprocessed to obtain a preprocessed image. Subsequently, the electronic device performs image denoising on the preprocessed image to obtain a denoised image. Finally, the electronic device performs an inverse process corresponding to the preprocessing on the denoised image to obtain a target output image. By preprocessing the image to be processed with at least one of global brightness enhancement processing, corner brightness enhancement processing and local area compression processing, the denoising effect in dark scenes and corner positions can be enhanced, and the deformation operation makes it possible to effectively improve the performance of the network without much degradation in image quality, thereby improving the accuracy of image denoising.

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0054] Figure 1 A schematic diagram of an optional image denoising method provided in an embodiment of the present application Figure 1 ,like Figure 1 As shown, including S101 to S104:

[0055] S101: Obtain an image to be processed.

[0056] In an embodiment of the present application, an electronic device obtains an image to be processed.

[0057] In the embodiment of the present application, the image to be processed is an object that needs to be subjected to image denoising, and the image to be processed contains unexpected, random, unnecessary or redundant interference information (ie, noise).

[0058] In the embodiment of the present application, the image to be processed may be obtained from a camera, scanner or other image capture device, or obtained from other sources, such as a gallery, the Internet, etc. The embodiment of the present application does not impose any limitation on the source of the image to be processed.

[0059] In the embodiment of the present application, the image to be processed is also called the current frame image, the current input frame, the current input frame image, etc., and the embodiment of the present application does not impose any limitation on this.

[0060] S102, preprocessing the image to be processed to obtain a preprocessed image; the preprocessing includes at least one of global brightness enhancement processing, corner brightness enhancement processing and local area compression processing.

[0061] In an embodiment of the present application, before performing denoising on the image to be processed, the electronic device pre-processes the image to be processed by at least one of global brightness enhancement processing, corner brightness enhancement processing, and local area compression processing.

[0062] In the embodiment of the present application, the image to be processed is pre-processed by at least one of global brightness enhancement processing, corner brightness enhancement processing, and local area compression processing according to the shooting scene corresponding to the image to be processed. In other words, different application scenarios may require different pre-processing processes according to the specific shooting scene and image quality requirements.

[0063] In an embodiment of the present application, global brightness enhancement processing is also called adaptive gain mapping (Gain Mapping, GainMap), corner brightness enhancement processing is also called lens shading correction mapping (Lens Shading Correction Map, LSCMap), and local area compression processing is also called geometric transformation processing (Warp).

[0064] In some embodiments of the present application, global brightness enhancement processing is used to adaptively increase the global brightness of an input image; corner brightness enhancement processing is used to enhance the brightness of corner positions in an input image; local area compression processing is used to compress a specific area in an input image; the specific area includes a non-interested area of ​​the image.

[0065] It should be noted that when compressing a specific area in an input image, the specific area may include a non-interest area of ​​the image or an interest area of ​​the image. When compressing a non-interest area, a relatively large compression rate may be selected, and when compressing an interest area, a relatively small compression rate may be selected.

[0066] In the embodiment of the present application, deep learning is essentially data-driven. In mobile devices such as mobile phones (i.e., electronic devices), the network scale is severely limited, and the benefits introduced by network design are very limited. However, through a series of data transformation processing, it is easier to obtain better effect improvement when the network structure remains unchanged. Specifically, for night scenes, due to the lack of incoming light, the contrast of the whole picture is poor. Based on this, the mapping of different brightness is completed by appropriate adaptive gain mapping (GainMap), and the dark area is brightened, thereby improving the responsiveness of the dark area to the network. Taking into account the influence of the lens shading effect (Len Shading) of the actual camera lens, usually, the noise at the corner position of the image is relatively large. In order to effectively activate the network denoising of this area, the image at the corner position of the image can be processed by lens shading correction mapping (LSCMap) to activate its denoising ability for the corner area, thereby improving its denoising effect.

[0067] In the embodiments of the present application, in most cases, the denoising requirements for targets at different image positions are different. For example, in face photography, more attention is usually paid to the effect of the face, while less attention is paid to the position far away from the face. Based on this, the image features can be compressed by performing corresponding local area compression processing (Warp) on the region of interest (ROI) in the input image, reducing the pixels that need to be processed by deep learning from the input image resolution, improving its performance while rarely feeling the decline in image quality.

[0068] In some embodiments of the present application, Figure 2 As shown, the implementation of preprocessing the image to be processed in S102 to obtain the preprocessed image may include S1021 to S1023:

[0069] S1021 . Perform global brightness enhancement processing on the image to be processed to obtain a first output image.

[0070] S1022: Perform corner brightness enhancement processing on the first output image to obtain a second output image.

[0071] S1023: Perform local area compression processing on the second output image to obtain a preprocessed image.

[0072] In the embodiment of the present application, the electronic device sequentially performs at least one global brightness enhancement process, at least one corner brightness enhancement process, and at least one local area compression process on the image to be processed to obtain a final pre-processed image. It should be noted that the embodiment of the present application does not limit the number of executions of the global brightness enhancement process, the corner brightness enhancement process, and the local area compression process, which can be selected according to the actual application scenario.

[0073] In some embodiments of the present application, the implementation of preprocessing the image to be processed in S102 to obtain the preprocessed image may include at least one of the following:

[0074] Perform global brightness enhancement processing on the image to be processed to obtain a preprocessed image;

[0075] Perform edge and corner brightness enhancement processing on the image to be processed to obtain a preprocessed image;

[0076] Perform local area compression processing on the image to be processed to obtain a preprocessed image.

[0077] In some embodiments of the present application, the implementation of preprocessing the image to be processed to obtain the preprocessed image in S102 may also include at least one of the following:

[0078] Performing global brightness enhancement processing and corner brightness enhancement processing on the image to be processed in sequence to obtain a preprocessed image;

[0079] Performing global brightness enhancement processing and local area compression processing on the image to be processed in sequence to obtain a preprocessed image;

[0080] The image to be processed is sequentially subjected to edge brightness enhancement processing and local area compression processing to obtain a preprocessed image.

[0081] It should be noted that the step of performing global brightness enhancement processing on the processed image to obtain the preprocessed image can refer to the implementation of performing global brightness enhancement processing on the processed image to obtain the first output image in S1021. The step of performing corner brightness enhancement processing on the processed image to obtain the preprocessed image can refer to the implementation of performing corner brightness enhancement processing on the first output image to obtain the second output image in S1022. The step of performing local area compression processing on the processed image to obtain the preprocessed image can refer to the implementation of performing local area compression processing on the second output image to obtain the preprocessed image in S1023.

[0082] It is understandable that global brightness enhancement processing can improve the brightness of the image to be processed, especially enhancing details in dark areas. In other words, global brightness enhancement processing aims to improve the detail performance of the image in low-light environments and make the details in dark areas more clearly visible. Corner brightness enhancement processing can further adjust the brightness of the corners of the image and further correct the brightness of the image to make the brightness of the image uniform throughout the field of view to achieve better visual effects. Local area compression processing compresses non-interest areas, thereby saving network power consumption in image transmission while retaining important information in the area of ​​interest.

[0083] S103: performing image denoising processing on the preprocessed image to obtain a denoised image.

[0084] In an embodiment of the present application, after the electronic device pre-processes the acquired image to be processed, image denoising is performed on the pre-processed image to obtain a denoised image.

[0085] In the embodiments of the present application, the image denoising method includes but is not limited to the following: mean filtering, median filtering, Gaussian filtering, wavelet denoising, total variation denoising, non-local means denoising, deep learning method, etc. The embodiments of the present application do not impose any limitation on the image denoising method.

[0086] In some embodiments of the present application, the implementation of performing image denoising on the preprocessed image in S103 to obtain the denoised image may include:

[0087] The preprocessed image is subjected to at least one wavelet forward transform process, at least one convolution process, at least one residual-based convolution process, at least one activation process, at least one pooling process, at least one feature fusion process, at least one upsampling process, and at least one wavelet inverse transform process to obtain a denoised image.

[0088] In the embodiment of the present application, the wavelet forward transform process is used to analyze and process signals with different frequency characteristics, decomposing the signals into sub-signals of different scales and frequencies, so as to better understand the frequency domain and time domain characteristics of the signals. The wavelet inverse transform process is used to restore the wavelet transformed signal to the original signal. The key idea of ​​the wavelet inverse transform is to decompose the signal into components of different frequencies and scales, and then synthesize these components into the original signal through inverse transform.

[0089] In the embodiment of the present application, the activation process can be a threshold learnable CLIP function, which combines the threshold and the learnable parameters to enable the model to automatically learn the appropriate activation range. The CLIP function can control the propagation of the gradient and the convergence of the model by limiting the input to a certain range.

[0090] In an embodiment of the present application, residual-based convolution processing solves the gradient vanishing and gradient exploding problems in the deep network training process by introducing residual blocks. These residual blocks contain jump connections, so that the network can learn residual information, thereby better performing feature extraction and image processing. The key idea of ​​residual-based convolution processing is that the network can learn the residual between the input data and the expected output, rather than directly learning the entire mapping function.

[0091] The following is some explanation of the convolution layer. The convolution layer can include many convolution operators, which are also called kernels. Their role in image processing is equivalent to a filter that extracts specific information from the input image matrix. The convolution operator can essentially be a feature vector. In the process of convolution operation on the image, the feature vector is usually processed one pixel after another along the horizontal direction on the input image to complete the work of extracting specific features from the image. The size of the feature vector should be related to the size of the image. It should be noted that the depth dimension of the feature vector is the same as the depth dimension of the input image. In the process of convolution operation, the feature vector will extend to the entire depth of the input image. Therefore, convolution with a single feature vector will produce a convolution output with a single depth dimension, but in most cases, a single feature vector is not used, but multiple feature vectors of the same size (row × column), that is, multiple isotype matrices, are applied. The values ​​in these feature vectors need to be obtained through a lot of training in practical applications. The feature vectors obtained through training can be used to extract information from the input image, so that the network model can make correct predictions.

[0092] In an embodiment of the present application, the pooling layer is used to perform pooling processing on the input vector; wherein the pooling processing includes average pooling processing and / or maximum pooling processing.

[0093] The following is some explanation of the pooling layer. Since it is often necessary to reduce the number of training parameters, the pooling layer often needs to be periodically introduced after the convolution layer. The pooling layer can also be called a downsampling layer. In the image processing process, the only purpose of the pooling layer is to reduce the spatial size of the image. The pooling layer can include an average pooling operator and / or a maximum pooling operator to sample the input image to obtain an image of smaller size. The average pooling operator can calculate the pixel values ​​in the image within a specific range to produce an average value as the result of average pooling. The maximum pooling operator can take the pixel with the largest value in the range as the result of maximum pooling within a specific range. In addition, just as the size of the weight matrix used in the convolution layer should be related to the image size, the operator in the pooling layer should also be related to the image size. The size of the image output after processing by the pooling layer can be smaller than the size of the image input to the pooling layer. Each pixel in the image output by the pooling layer represents the average value or maximum value of the corresponding sub-region of the image input to the pooling layer.

[0094] In some embodiments of the present application, the preprocessed image is subjected to at least one wavelet forward transform process, at least one convolution process, at least one residual-based convolution process, at least one activation process, at least one pooling process, at least one feature fusion process, at least one upsampling process, and at least one wavelet inverse transform process to obtain a denoised image, including:

[0095] Performing wavelet forward transform processing, activation processing, convolution processing, residual-based convolution processing and pooling processing on the preprocessed image to obtain a first intermediate image;

[0096] Performing convolution processing, residual-based convolution processing, and pooling processing on the first intermediate image to obtain a second intermediate image;

[0097] Performing at least one convolution process, at least one residual-based convolution process, and an upsampling process on the second intermediate image to obtain a third intermediate image;

[0098] Performing feature fusion processing on the second intermediate image and the third intermediate image to obtain a fourth intermediate image;

[0099] performing residual-based convolution processing, convolution processing, and upsampling processing on the fourth intermediate image to obtain a fifth intermediate image;

[0100] performing feature fusion processing on the fifth intermediate image and the first intermediate image to obtain a sixth intermediate image;

[0101] The sixth intermediate image is subjected to at least one residual-based convolution process, at least one convolution process, upsampling process, and inverse wavelet transform process to obtain a denoised image.

[0102] Figure 3 A schematic diagram of the structure of an optional image denoising model provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, first, the electronic device sequentially performs wavelet forward transform processing, activation processing (threshold learnable CLIP function), convolution processing, residual-based convolution processing and pooling processing (maximum pooling processing) on ​​the image to be processed Input (input data frame) to obtain a first intermediate image. Then, the electronic device performs convolution processing, residual-based convolution processing and pooling processing (maximum pooling processing) on ​​the first intermediate image to obtain a second intermediate image. Further, the electronic device sequentially performs convolution processing, residual-based convolution processing, pooling processing, convolution processing, residual-based convolution processing and upsampling processing on the second intermediate image to obtain a third intermediate image. Further, the electronic device performs feature fusion processing on the second intermediate image and the third intermediate image to obtain a fourth intermediate image, and performs residual-based convolution processing, convolution processing and upsampling processing on the fourth intermediate image to obtain a fifth intermediate image. Further, the electronic device performs feature fusion processing on the fifth intermediate image and the first intermediate image to obtain a sixth intermediate image. Finally, the electronic device performs residual-based convolution processing, convolution processing, upsampling processing, convolution processing, residual-based convolution processing, convolution processing, and wavelet inverse transform processing on the sixth intermediate image in sequence to obtain the final denoised image (Ooutput).

[0103] In some implementations of the present application, the denoised image is obtained by performing image denoising on the preprocessed image using an image denoising model, and the image denoising model is obtained by training an initial image denoising model; the method further includes:

[0104] Acquire sample data; the sample data includes original image samples and denoised samples corresponding to the original image samples;

[0105] Performing at least one of global brightness enhancement processing, corner brightness enhancement processing, and local area compression processing on the original image sample to obtain a preprocessed sample image;

[0106] Performing image denoising on the preprocessed sample image through the initial image denoising model to obtain a denoised sample image;

[0107] Performing inverse processing corresponding to preprocessing on the denoised sample image to obtain a sample output image;

[0108] Input the sample output image of the original image sample and the corresponding sample image into the preset loss function to obtain the current loss result;

[0109] The model parameters in the initial image denoising model are modified according to the current loss result to obtain the current intermediate image denoising model;

[0110] Continue to perform image denoising on the original image samples through the current intermediate image denoising model to obtain the next loss result, and modify the model parameters in the current intermediate image denoising model according to the next loss result to obtain the next intermediate image denoising model, until the next loss result corresponding to the next intermediate image denoising model meets the cutoff condition, and the next intermediate image denoising model is used as the image denoising model.

[0111] In an embodiment of the present application, the preset loss function is pre-set. Exemplarily, the preset loss function can be the pixel difference between the candidate result of the original image sample and the corresponding instance sample. The embodiment of the present application does not impose any limitation on the preset loss function.

[0112] In an embodiment of the present application, the cutoff condition may include at least one of the following: the loss result is less than or equal to a threshold, and the number of iterations is met. The embodiment of the present application does not impose any limitation on the cutoff condition.

[0113] In some embodiments of the present application, performing image denoising on the preprocessed sample image by using the initial image denoising model to obtain the denoised sample image may include:

[0114] Through the initial image denoising model, the preprocessed sample image is subjected to at least one wavelet forward transform process, at least one convolution process, at least one residual-based convolution process, at least one activation process, at least one pooling process, at least one feature fusion process, at least one upsampling process, and at least one wavelet inverse transform process to obtain a denoised sample image.

[0115] S104: Perform inverse processing corresponding to the pre-processing on the denoised image to obtain a target output image.

[0116] In the embodiments of the present application, the inverse processing is also referred to as post-processing.

[0117] In the embodiment of the present application, after the image to be processed is subjected to global brightness enhancement processing, corner brightness enhancement processing, and local area compression processing, the brightness and shape of the original input image to be processed are changed. Since the global brightness enhancement processing, corner brightness enhancement processing, and local area compression processing are reversible, the image to be processed can be restored by reverse operations. In this way, the target output image is as consistent as possible with the original input image to be processed in terms of brightness and shape.

[0118] It can be understood that by performing inverse processing corresponding to the preprocessing on the denoised image, the quality and stability of the image can be effectively improved.

[0119] In an embodiment of the present application, an image denoising method is provided. First, an electronic device acquires an image to be processed; then, the electronic device preprocesses the image to be processed to obtain a preprocessed image; the preprocessing includes at least one of global brightness enhancement processing, corner brightness enhancement processing, and local area compression processing; then, the electronic device performs image denoising processing on the preprocessed image to obtain a denoised image; finally, the electronic device performs an inverse processing corresponding to the preprocessing on the denoised image to obtain a target output image. By preprocessing the image to be processed with at least one of global brightness enhancement processing, corner brightness enhancement processing, and local area compression processing, the denoising effect in dark light scenes and corner positions can be enhanced, and the deformation operation effectively improves the performance of the network without much degradation in image quality, thereby improving the denoising performance of the image.

[0120] In some embodiments of the present application, Figure 4 As shown, the implementation of performing global brightness enhancement processing on the image to be processed in S1021 to obtain the first output image may include S201 to S203:

[0121] S201. Determine a current light intensity index according to exposure information corresponding to an image to be processed.

[0122] In the embodiment of the present application, the exposure information corresponding to the image to be processed includes exposure parameters related to the shooting process of the image to be processed, and these parameters can be used to describe the exposure condition of the image to be processed.

[0123] Exemplarily, the exposure information may include: exposure time (Exposure Time), aperture (Aperture), ISO sensitivity (ISO Sensitivity), exposure mode (Exposure Mode), metering mode (Metering Mode), exposure compensation (Exposure Compensation), and white balance (White Balance).

[0124] Among them, exposure time refers to the length of time that the image sensor or film is exposed to light. It is usually expressed in seconds and determines the degree of motion blur and brightness level in the image. Longer exposure times result in brighter images, but motion blur may also occur. Aperture refers to the size of the aperture in the camera lens. ISO sensitivity indicates how sensitive the camera or sensor is to light. Higher ISO values ​​allow shooting in low light conditions, but may result in noise in the image. The exposure mode defines how the camera controls the exposure parameters. Common exposure modes include manual mode, automatic mode, aperture priority mode, and shutter priority mode. The metering mode determines how the camera measures the brightness of the scene to determine the exposure parameters. Common metering modes include evaluative metering, center-weighted metering, and spot metering. Exposure compensation allows users to manually adjust exposure parameters to better suit different shooting conditions. The white balance setting is used to correct color deviations in the image so that white objects look truly white, and is usually adjusted according to the type of light source in the scene.

[0125] It is understandable that exposure information can help understand the shooting environment and conditions of the image, thereby better performing image post-processing or image quality control.

[0126] In an embodiment of the present application, the electronic device obtains exposure information corresponding to the image to be processed according to Automatic Exposure Control (AEC). AEC is a technology for measuring lighting conditions and automatically adjusting camera exposure parameters. Once AEC completes the adjustment of exposure parameters, the exposure information of the image to be processed can be obtained by accessing the camera driver or using the camera API.

[0127] In an embodiment of the present application, the electronic device determines the current light intensity index based on exposure information corresponding to the image to be processed.

[0128] In the embodiment of the present application, the light intensity index can be expressed as Lux Index. The light intensity index is a numerical value that indicates the intensity level of light, that is, the brightness level of the current environment, which can help the camera system or other devices automatically adjust the exposure parameters to obtain appropriate image brightness.

[0129] S202. Determine a target gain parameter corresponding to the current light intensity index according to a preset gain parameter relationship; the preset gain parameter relationship represents a corresponding relationship between the light intensity index and the gain parameter.

[0130] In the embodiment of the present application, the preset gain parameter relationship represents the corresponding relationship between the light intensity index and the gain parameter. The preset gain parameter relationship is pre-set, and different light intensity indices may correspond to different gain parameters. Exemplarily, the preset gain parameter relationship may be expressed by formula (1):

[0131]

[0132] In formula (1), luxidex represents the light intensity index, and Gamma represents the gain parameter. When the current light intensity index is less than or equal to 200 and greater than 100, the target gain parameter is 1; when the current light intensity index is less than or equal to 300 and greater than 200, the target gain parameter is 0.9; when the current light intensity index is less than or equal to 350 and greater than 300, the target gain parameter is 0.8; when the current light intensity index is less than or equal to 400 and greater than 350, the target gain parameter is 0.6; when the current light intensity index is less than or equal to 460 and greater than 400, the target gain parameter is 0.5; when the current light intensity index is less than or equal to 480 and greater than 460, the target gain parameter is 0.4; when the current light intensity index is less than or equal to 500 and greater than 480, the target gain parameter is 0.3; when the current light intensity index is greater than 500, the target gain parameter is 0.2.

[0133] It should be noted that, generally speaking, the larger the light intensity index, the brighter the environment, and a smaller gain parameter may be required; the smaller the light intensity index, the darker the environment, and a larger gain parameter may be required.

[0134] It should be noted that the preset gain parameter relationship shown in formula (1) is only an example, and the embodiment of the present application does not impose any limitation on this, and the specific selection can be made according to actual conditions.

[0135] S203 . Perform brightness enhancement processing on each first pixel in the image to be processed according to the target gain parameter to obtain a first output image.

[0136] In some embodiments of the present application, the implementation of performing brightness enhancement processing on each first pixel in the image to be processed according to the target gain parameter to obtain the first output image in S203 may include S2031 to S2033:

[0137] S2031. Determine a maximum pixel value according to the number of bits of each first pixel point in the image to be processed.

[0138] In the embodiment of the present application, the maximum pixel value can be expressed as Pmax.

[0139] In an embodiment of the present application, the maximum pixel value is related to the number of bits of the first pixel point. For example, when the number of bits of the first pixel point is 10 bits, the maximum pixel value can be 1023; when the number of bits of the first pixel point is 8 bits, the maximum pixel value can be 255.

[0140] S2032. For the brightness value corresponding to each first pixel in the image to be processed, determine the processed pixel value of each first pixel according to the brightness value of each first pixel, the maximum pixel value and the target gain parameter.

[0141] In the embodiment of the present application, the processed pixel value of each first pixel is determined according to the brightness value, the maximum pixel value and the target gain parameter of each first pixel, which can be expressed by formula (2):

[0142]

[0143] In formula (2), GainMap represents the processed pixel value of the first pixel, Pin represents the brightness value of the first pixel, Pmax represents the maximum pixel value, and Gamma represents the target gain parameter corresponding to the current light intensity index.

[0144] It should be noted that the brightness enhancement processing shown in formula (2) is only an example. In practical applications, relevant adjustments can also be made according to actual conditions. The embodiments of the present application do not impose any restrictions on this, and specific adjustments can be made according to actual conditions.

[0145] S2033: Determine a first output image according to the processed pixel values ​​of each first pixel point in the image to be processed.

[0146] In an embodiment of the present application, after brightness enhancement processing is performed on each first pixel point in the image to be processed, the processed pixel value of each first pixel point is obtained, and the first output image is determined based on the processed pixel value of each first pixel point in the image to be processed.

[0147] In some embodiments of the present application, Figure 5As shown, the implementation of performing corner brightness enhancement processing on the first output image in S1022 to obtain the second output image may include S301 to S303:

[0148] S301 . For each second pixel in a first output image, determine an average brightness value of a preset area around each second pixel.

[0149] In the embodiment of the present application, for each second pixel in the first output image, a surrounding neighborhood of a preset area of ​​each second pixel is extracted. In the embodiment of the present application, the preset area is a pre-set parameter, and illustratively, the preset area is 11×11 pixels.

[0150] In the embodiment of the present application, the average brightness value can be expressed as luma_avg.

[0151] S302. Determine the processed pixel value of each second pixel according to the average brightness value of each second pixel and the lens shading calibration parameters corresponding to the second pixel in the shadow effect correction table; the shadow effect correction table represents the correction gain of each second pixel in the second output image under different lens shading effects.

[0152] In an embodiment of the present application, the shadow effect correction table is also referred to as LscTable (Lens Shading Correction Table). The shadow effect correction table is a key tool for correcting shadow effects in digital images. The shadow effect is caused by the propagation of light in the lens system of the lens, causing the edge of the image to become darker relative to the center of the image, which is a natural result of the lens characteristics and optical path design. The shadow effect correction table is usually a two-dimensional array used to correct each pixel in the image. Its function is to provide correction gains for pixels at different positions to compensate for the brightness non-uniformity introduced by the shadow effect. Each element in the shadow effect correction table (i.e., lens shadow calibration parameter) corresponds to a pixel position in the image and specifies the correction gain applied at that position.

[0153] In the embodiment of the present application, the lens shading calibration parameters are a set of parameters used to describe the characteristics of the lens shading effect and the information required for correction. These parameters are usually obtained by analyzing the calibration images collected in advance, or are obtained and used by the camera manufacturer or the developer of the image processing software during the lens calibration and image processing process.

[0154] In the embodiment of the present application, according to the average brightness value of each second pixel point, the lens shadow calibration parameters corresponding to the second pixel point are determined in the shadow effect correction table. Exemplarily, the shadow effect correction table is searched according to the average brightness value of the second pixel point to find the lens shadow calibration parameters corresponding to this brightness value. These parameters include the center brightness attenuation coefficient, the brightness attenuation function, the shadow center position, the shadow area size, etc.

[0155] In some embodiments of the present application, determining the processed pixel value of each second pixel according to the average brightness value of each second pixel and the lens shadow calibration parameter corresponding to the second pixel in the shadow effect correction table in S302 may include S3021 to S3023:

[0156] S3021. Determine a minimum calibration parameter among the various lens shading calibration parameters in the shading effect correction table.

[0157] In the embodiment of the present application, the electronic device determines the minimum calibration parameter among the various lens shadow calibration parameters in the shadow effect correction table corresponding to the image to be processed. The minimum calibration parameter can be expressed as LscTable min .

[0158] S3022: Determine a first adjustment parameter corresponding to each second pixel point according to the position information and the average brightness value of each second pixel point in the first output image.

[0159] In some embodiments of the present application, the implementation of determining the first adjustment parameter corresponding to each second pixel point according to the position information and the average brightness value of each second pixel point in the first output image in S3022 may include:

[0160] In the case where the second pixel is located at a corner position in the first output image, if the average brightness value corresponding to the second pixel is less than a preset threshold, the first reference adjustment parameter is increased to obtain the first adjustment parameter; or,

[0161] If the average brightness value corresponding to the second pixel is greater than or equal to the preset threshold, the first reference adjustment parameter is used as the first adjustment parameter.

[0162] In the embodiment of the present application, the first reference adjustment parameter is a preset parameter.

[0163] S3023. Determine a processed pixel value of each second pixel point according to the lens shading calibration parameter, the first adjustment parameter, and the minimum calibration parameter corresponding to each second pixel point.

[0164] In the embodiment of the present application, the processed pixel value of each second pixel is determined according to the lens shading calibration parameter, the first adjustment parameter and the minimum calibration parameter corresponding to each second pixel, which can be expressed by formula (3):

[0165]

[0166] In formula (3), LSCMap represents the processed pixel value of the second pixel, LscTable represents the lens shading calibration parameter corresponding to the first adjustment parameter of the second pixel, and LscTable min represents the minimum calibration parameter, γ represents the first adjustment parameter of the second pixel point, the first adjustment parameter is an adjustable parameter, and the recommended range is 1.0 to 1.3.

[0167] S303: Determine a second output image according to the processed pixel values ​​of each second pixel point in the first output image.

[0168] Figure 6 A schematic diagram of an optional image corner brightness provided in an embodiment of the present application, such as Figure 6 As shown in the figure, for the image to be processed obtained by taking a photo of a uniformly illuminated scene, the corners of the image without corner brightness enhancement are significantly lower in brightness than the central area. Therefore, it is necessary to perform brightness enhancement processing on the corners of the image. It can be understood that by adjusting the brightness of different gain positions of the image to be processed through the lens shading correction table, the position with large gain has appropriate brightness for denoising, thereby improving the denoising performance of the network for the corner positions.

[0169] It is understandable that each second pixel in the image can be corrected according to the lens shading calibration parameter and the average brightness value to eliminate the brightness non-uniformity caused by the lens shading effect, thereby improving the image quality. This is an important step in the image processing flow, which can adapt to image capture under different lighting conditions.

[0170] In some embodiments of the present application, Figure 7 As shown, the implementation of performing local area compression processing on the second output image in S1023 to obtain a preprocessed image may include S401 to S404:

[0171] S401. Obtain guidance information corresponding to the second output image; the guidance information is used to indicate the position of the region of interest in the second output image.

[0172] In an embodiment of the present application, the electronic device obtains guidance information of the region of interest according to a preset strategy or algorithm. The guidance information may be a set of coordinates, the position of a rectangular box, or other forms, used to indicate the position of the region of interest in the image.

[0173] In the embodiment of the present application, the region of interest can be a quadrilateral, a circle, a polygon, etc. The embodiment of the present application does not limit the shape of the region of interest, and can be selected according to the actual application scenario. For example, the region of interest in the embodiment of the present application is a rectangle.

[0174] In the embodiment of the present application, when the region of interest is a rectangle, the guidance information may be the coordinate values ​​of the upper left corner and the lower right corner of the region of interest.

[0175] S402: Determine an input coordinate range and an output coordinate range of a non-interested region in a second output image based on the guidance information.

[0176] In some embodiments of the present application, the implementation of determining the input coordinate range and the output coordinate range of the non-interested region in the second output image based on the guidance information in S402 may include S4021 to S4023:

[0177] S4021. Determine coordinate information of a region of interest in the second output image according to the guidance information.

[0178] In the embodiment of the present application, the coordinate information of the region of interest can be expressed as (x1, y1) and (x2, y2), where (x1, y1) represents the coordinates of the upper left corner of the region of interest, and (x2, y2) represents the coordinates of the upper right corner of the region of interest.

[0179] S4022: Determine a compression rate of a non-interest region in the second output image based on the coordinate information of the interest region in the second output image.

[0180] In some embodiments of the present application, the compression rate of the non-interest area includes a first compression rate and a second compression rate; wherein the first compression rate represents the average compression rate in the width direction of the non-interest area in the output image obtained after the local area compression processing is performed on the second output image; the second compression rate represents the average compression rate in the height direction of the non-interest area in the output image obtained after the local area compression processing is performed on the second output image.

[0181] In some embodiments of the present application, the implementation of determining the compression rate of the non-interest region in the second output image based on the coordinate information of the interest region in the second output image in S4022 may include:

[0182] Acquire a first resolution of the second output image, and a second resolution of the output image obtained after performing local area compression processing on the second output image;

[0183] Determine, according to the first resolution and the second resolution, a first area range in a width direction and a second area range in a height direction of a non-interested area of ​​an output image obtained after performing local area compression processing on the second output image;

[0184] Determining a first compression ratio according to the first area range, the first resolution, and the second resolution;

[0185] A second compression rate is determined according to the second area range, the first resolution, and the second resolution.

[0186] In the embodiment of the present application, the first resolution can be expressed as W*H, and the second resolution can be expressed as Wo*Ho.

[0187] Figure 8 A schematic diagram of an optional coordinate mapping provided in an embodiment of the present application, such as Figure 8 As shown, the coordinate information of the region of interest can be expressed as (x1, y1) and (x2, y2). After the non-interest region is compressed, the pixel points in the non-interest region ( Figure 8 After coordinate mapping (shown as a black origin in the figure), its relative position remains unchanged.

[0188] In an embodiment of the present application, the first area range in the width direction of the non-interest area of ​​the output image can be expressed as Wo-Wroi, and the second area range in the height direction of the non-interest area of ​​the output image can be expressed as Ho-Hroi.

[0189] In the embodiment of the present application, the first compression ratio can be expressed as The second compression ratio can be expressed as

[0190] S4023. Determine an input coordinate range and an output coordinate range of the non-interest region according to the compression rate of the non-interest region.

[0191] In the embodiment of the present application, the electronic device generates a coordinate compression relationship according to the compression rate of the non-interest area, thereby determining the input coordinate range and the output coordinate range of the non-interest area.

[0192] Fig. 9 A schematic diagram of an optional compression relationship of a single-dimensional coordinate provided in an embodiment of the present application, such as Fig. 9 As shown, Fig. 9 The vertical line on the left is the data before compression, the vertical line on the right is the data after compression, h0 and h1 are the single-dimensional coordinates of the input area, h2 and h3 are the single-dimensional coordinates of the output area, then, the input coordinate range can be expressed as [h0,h1], and the output coordinate range can be expressed as [h2,h3].

[0193] S403 . Compress the coordinate information of each third pixel in the second output image according to the input coordinate range and the output coordinate range to obtain compressed coordinate information of each third pixel.

[0194] In the embodiment of the present application, the compressed coordinate information of each third pixel point can be expressed by formula (4):

[0195]

[0196] In formula (4), ho represents the compressed coordinate information of the third pixel, [h2, h3] represents the output coordinate range of the non-interest area, [[h0, h1] represents the input coordinate range of the non-interest area, gamma represents the second adjustment parameter corresponding to the third pixel, and hi represents the coordinate information of the third pixel before compression (i.e., the input coordinate information).

[0197] In some embodiments of the present application, the implementation of compressing the coordinate information of each third pixel in the second output image according to the input coordinate range and the output coordinate range to obtain the compressed coordinate information of each third pixel in S403 may include S4031 to S4034:

[0198] S4031. For each third pixel point in the non-interest region of the second output image, determine a first position ratio of each third pixel point according to coordinate information of each third pixel point and an input coordinate range.

[0199] In the embodiment of the present application, for each third pixel point in the non-interested area of ​​the second output image, the electronic device determines the first position ratio of each third pixel point according to the coordinate information hi of each third pixel point and the input coordinate range [h0, h1]:

[0200] S4032: Determine a second adjustment parameter according to position information of the non-interest region of the second output image relative to the interest region of the second output image.

[0201] In the embodiment of the present application, the second adjustment parameter can be expressed as gamma.

[0202] In some embodiments of the present application, the implementation of determining the second adjustment parameter of each third pixel point according to the position information of each third pixel point in the non-interested region of the second output image in S4034 may include:

[0203] When the starting position in the non-interest region of the second output image is close to the interest region of the second output image, the second reference adjustment parameter is reduced to obtain the second adjustment parameter; or,

[0204] When the end position in the non-interest region of the second output image is close to the interest region of the second output image, the second reference adjustment parameter is increased to obtain the second adjustment parameter.

[0205] In the embodiment of the present application, the second benchmark adjustment parameter is a pre-set parameter. Exemplarily, the second benchmark adjustment parameter may be 1.

[0206] It can be understood that adaptively adjusting the second reference adjustment parameter according to the position of the non-interest region of the second output image can effectively avoid the problem of an abrupt transition between the region of interest and the non-interest region.

[0207] S4033. Adjust the first position ratio corresponding to each third pixel point according to the second adjustment parameter to obtain the second position ratio of each third pixel point.

[0208] In the embodiment of the present application, the electronic device adjusts the first position ratio corresponding to each third pixel point according to the second adjustment parameter gamma. Adjust to obtain the second position ratio of each third pixel point

[0209] S4034. Determine the compressed coordinate information of each third pixel point according to the second position ratio and the output coordinate range of each third pixel point.

[0210] In the embodiment of the present application, the electronic device calculates the second position ratio of each third pixel point according to the second position ratio of each third pixel point. And the output coordinate range [h2,h3], determine the compressed coordinate information ho of each third pixel point as

[0211] S404: Determine a preprocessed image according to the compressed coordinate information of each third pixel point in the non-interest region of the second output image and the coordinate information of the interest region of the second output image.

[0212] It is understandable that by compressing the non-interested areas, the image resolution is reduced, thereby improving the performance of the network. At the same time, because the area of ​​interest is still set at the original resolution, the image quality of the area of ​​interest will not be reduced. In this way, while ensuring the image quality of the area of ​​interest, the denoising performance of the network can be effectively improved.

[0213] The image denoising method proposed in this application is explained in a specific embodiment below.

[0214] Fig.10 A schematic diagram of a framework of an optional image denoising method provided in an embodiment of the present application, such as Fig.10As shown, first, the electronic device sequentially performs a preprocessing process of global brightness enhancement processing (Gain Map), corner brightness enhancement processing (LSCMap) and local area compression processing (Warp) on the image to be processed (current input frame) to obtain a preprocessed image. Specifically, the electronic device performs a global brightness enhancement processing on the image to be processed according to the illumination intensity index corresponding to the image to be processed to obtain a first output image; performs a corner brightness enhancement processing on the first output image according to the shadow effect correction table corresponding to the image to be processed to obtain a second output image; further, according to the region of interest of the image to be processed, the non-interested region is determined, and the non-interested region of the second output image is subjected to local area compression processing, thereby obtaining a preprocessed image. Then, the electronic device performs an image denoising processing (AINR) on the preprocessed image to obtain a denoised image. Finally, the electronic device sequentially performs an inverse processing process of inverse local area compression processing (Inverse Warp), inverse corner brightness enhancement processing (Inverse LSCMap) and inverse global brightness enhancement processing (Inverse Gain Map) on the denoised image to obtain a target output image (output frame).

[0215] Specifically, for the image to be processed, the global brightness is first adaptively improved through global brightness enhancement processing, so that the details in the dark can be better presented. Then, according to the configuration of lens shading correction (LSC), the image after global brightness enhancement processing is applied with corner brightness enhancement processing to improve the image brightness at the corner position. Finally, according to the region of interest (ROI) of the image after corner brightness enhancement processing, the image of non-interest is compressed to reduce the input image resolution and save network power consumption. The following is an introduction to each preprocessing process.

[0216] (1) Global brightness enhancement processing (Gain Map)

[0217] In the embodiment of the present application, the main purpose of the global brightness enhancement processing is to increase the brightness of the overly dark image so that the details in the dark can be revealed. In order to adapt to different brightness images and adaptively increase the brightness, the current light intensity index (LuxIndex) calculated by the automatic exposure control (AEC) is used to generate a GainMap (equivalent to the first output image). Specifically, first, according to the preset gain parameter relationship shown in formula (1), for different current light intensity indices (LuxIndex), different target gain parameters (Gamma) are applied to achieve the setting of its GainMap. Then, according to formula (2), the brightness enhancement processing of each first pixel point in the image to be processed is implemented, and the global brightness enhancement of the image to be processed is implemented to obtain the image after global brightness enhancement processing (equivalent to the first output image).

[0218] (2) Edge brightness enhancement processing (LSCMap)

[0219] In the embodiment of the present application, the purpose of the corner brightness enhancement processing is to adjust the brightness of different gain positions through the shadow effect correction table (LSCTable), so that there is a suitable brightness for denoising at the position of the lens shadow with large gain, thereby improving the denoising performance of the network for the corner position. The process of performing corner brightness enhancement processing on the first output image to obtain the second output image (LSCMap) includes the following steps:

[0220] Step 1: For each second pixel of the second output image, extract its 11×11 surrounding neighborhood (preset area) and calculate its average brightness value (luma_avg). Then, generate a shadow effect correction table according to its corresponding lens shadow calibration parameter (LSC calibration parameter).

[0221] Step 2: Perform corner brightness enhancement according to the average brightness value of each second pixel and the shadow effect correction table. Wherein, when the second pixel is located at a corner position in the first output image, if the average brightness value corresponding to the second pixel is less than a preset threshold, the first reference adjustment parameter is increased to obtain the first adjustment parameter, and if the average brightness value corresponding to the second pixel is greater than or equal to the preset threshold, the first reference adjustment parameter is used as the first adjustment parameter. Further, according to formula (3), the processed pixel value of each second pixel is determined according to the lens shadow calibration parameter, the first adjustment parameter and the minimum calibration parameter corresponding to each second pixel, so as to enhance the corner brightness of the image to be processed and obtain an image after corner brightness enhancement (equivalent to the second output image).

[0222] (3) Local area compression processing (Warp)

[0223] In the embodiment of the present application, the local area compression processing obtains the guidance information corresponding to the second output image according to a certain strategy or algorithm, and the guidance information is the location information of the region of interest (ROI), and the image resolution is reduced by compressing the non-interested region in the second output image, thereby improving the performance of the network. At the same time, because the region of interest is still set at the original resolution, the image quality of the region of interest will not be reduced.

[0224] In the embodiment of the present application, assuming that the resolution of the second output image (equivalent to the first resolution) is W*H, the resolution after compression (equivalent to the second resolution) is Wo*Ho, and the coordinate information of the region of interest in the second output image is (x1, y1) and (x2, y2), the process of performing local region compression processing on the second output image includes the following steps:

[0225] Step 1: Calculate the size of the region of interest and the size of the non-interest region of the second output image, then Wroi = x2-x1, Hroi = y2-y1. The width direction of the non-interest region of the compressed image (equivalent to the pre-processed image) is Wo-Wroi, and the average compression ratio is: The height dimension after compression is: Ho-Hroi, and the average compression ratio is:

[0226] Step 2: Generate a coordinate compression relationship based on the calculated compression ratio, refer to Fig. 9 Taking the single-dimensional coordinates shown in the figure as an example, h0 and h1 are the single-dimensional coordinates of the input area, and h2 and h3 are the single-dimensional coordinates of the output area. Then, the compression of this area can be achieved by referring to formula (4), where hi is the input coordinate, ho is the output coordinate, and gamma is the corresponding non-uniform compression parameter. When gamma is set to 1.0, it means compression at a uniform scale. If gamma < 1.0, a small scale compression is applied to the input close to h0, and a large scale compression is applied to the input far from h0. On the contrary, if gamma > 1.0, a large scale compression is applied to the input close to h0, and a small scale compression is applied to the input far from h0.

[0227] (4) Reverse processing

[0228] Since the pre-processing (global brightness enhancement processing, corner brightness enhancement processing and local area compression processing) applies some gain and deformation to it, resulting in inconsistency with the brightness and shape of the original input image (the image to be processed), the reversible operation of the pre-processing is completed by post-processing (inverse processing). Since all pre-processing is reversible. Therefore, the post-processing can be directly reversible according to the pre-processing part, and the inverse processing process of global brightness enhancement inverse processing, corner brightness enhancement inverse processing and local area compression inverse processing will not be repeated.

[0229] In an embodiment of the present application, an image denoising method is provided, the key points of which are:

[0230] 1) A global brightness enhancement processing (GainMap) operation based on dark light contrast enhancement is proposed, which effectively improves the denoising effect of the network in extremely dark environments;

[0231] 2) A corner brightness enhancement processing (LSCMap) operation based on lens shading correction (LSC) is proposed, which effectively improves the network denoising effect at the corner position;

[0232] 3) A scheme based on local area compression processing (Warp) of images is proposed, which improves the processing performance of the network without significantly reducing the image quality.

[0233] In the embodiments of the present application, in actual applications, only one or several preprocessing methods may be selected for different scenarios, or they may be integrated during the preprocessing process.

[0234] In the embodiment of the present application, in actual applications, corresponding color gamuts may be selected for processing for different platforms, and the embodiment of the present application does not impose any limitation on this.

[0235] It can be understood that in the embodiment of the present application, on the one hand, the denoising effect in dark scenes and corner positions is enhanced by preprocessing, and on the other hand, the deformation operation effectively improves the performance of the network without much degradation in image quality.

[0236] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.; or, steps in different embodiments may be combined into a new technical solution.

[0237] Based on the foregoing embodiments, the present application provides a schematic diagram of an optional image denoising device, as shown in FIG. Fig.11 As shown, the image denoising device 10 includes an acquisition unit 11, a processing unit 12, a denoising unit 13 and a training unit 14; wherein,

[0238] The acquisition unit 11 is used to acquire the image to be processed;

[0239] The processing unit 12 is used to preprocess the image to be processed to obtain a preprocessed image; the preprocessing includes at least one of global brightness enhancement processing, corner brightness enhancement processing and local area compression processing;

[0240] The denoising unit 13 is used to perform image denoising on the preprocessed image to obtain a denoised image;

[0241] The processing unit 12 is further configured to perform an inverse process corresponding to the preprocessing on the denoised image to obtain a target output image.

[0242] In some embodiments of the present application, the global brightness enhancement processing is used to adaptively increase the global brightness of the input image; the corner brightness enhancement processing is used to enhance the brightness of the corner positions in the input image; the local area compression processing is used to compress the image of a specific area in the input image; the specific area includes a non-interested area of ​​the image; the processing unit 12 is also used to perform global brightness enhancement processing on the image to be processed to obtain a first output image; perform corner brightness enhancement processing on the first output image to obtain a second output image; perform local area compression processing on the second output image to obtain the pre-processed image.

[0243] In some embodiments of the present application, the processing unit 12 is also used for at least one of the following: performing global brightness enhancement processing on the image to be processed to obtain the preprocessed image; performing corner brightness enhancement processing on the image to be processed to obtain the preprocessed image; performing local area compression processing on the image to be processed to obtain the preprocessed image.

[0244] In some embodiments of the present application, the processing unit 12 is also used for at least one of the following: sequentially performing global brightness enhancement processing and corner brightness enhancement processing on the image to be processed to obtain the preprocessed image; sequentially performing global brightness enhancement processing and local area compression processing on the image to be processed to obtain the preprocessed image; sequentially performing corner brightness enhancement processing and local area compression processing on the image to be processed to obtain the preprocessed image.

[0245] In some embodiments of the present application, the processing unit 12 is further used to determine the current light intensity index based on the exposure information corresponding to the image to be processed; determine the target gain parameter corresponding to the current light intensity index based on a preset gain parameter relationship; the preset gain parameter relationship characterizes the correspondence between the light intensity index and the gain parameter; and perform brightness enhancement processing on each first pixel point in the image to be processed according to the target gain parameter to obtain the first output image.

[0246] In some embodiments of the present application, the processing unit 12 is further used to determine the maximum pixel value according to the number of bits of each first pixel in the image to be processed; for the brightness value corresponding to each first pixel in the image to be processed, determine the processed pixel value of each first pixel according to the brightness value of each first pixel, the maximum pixel value and the target gain parameter; determine the first output image according to the processed pixel value of each first pixel in the image to be processed.

[0247] In some embodiments of the present application, the processing unit 12 is further used to determine, for each of the second pixel points in the first output image, an average brightness value of a preset area around each of the second pixel points; determine the processed pixel value of each of the second pixel points based on the average brightness value of each of the second pixel points and the lens shading calibration parameters corresponding to the second pixel points in the shadow effect correction table; the shadow effect correction table represents the correction gain of each of the second pixel points in the second output image under different lens shading effects; and determine the second output image based on the processed pixel values ​​of each of the second pixel points in the first output image.

[0248] In some embodiments of the present application, the processing unit 12 is further used to determine the minimum calibration parameter among the lens shading calibration parameters in the shading effect correction table; determine the first adjustment parameter corresponding to each second pixel point according to the position information of each second pixel point in the first output image and the average brightness value; determine the processed pixel value of each second pixel point according to the lens shading calibration parameter corresponding to each second pixel point, the first adjustment parameter and the minimum calibration parameter.

[0249] In some embodiments of the present application, the processing unit 12 is also used to, when the second pixel point is located at a corner position in the first output image, if the average brightness value corresponding to the second pixel point is less than a preset threshold, increase the first benchmark adjustment parameter to obtain the first adjustment parameter; or, if the average brightness value corresponding to the second pixel point is greater than or equal to the preset threshold, use the first benchmark adjustment parameter as the first adjustment parameter.

[0250] In some embodiments of the present application, the processing unit 12 is also used to obtain guidance information corresponding to the second output image; the guidance information is used to indicate the position of the area of ​​interest in the second output image; based on the guidance information, the input coordinate range and output coordinate range of the non-interest area in the second output image are determined; according to the input coordinate range and the output coordinate range, the coordinate information of each third pixel point in the second output image is compressed to obtain the compressed coordinate information of each third pixel point; according to the compressed coordinate information of each third pixel point in the non-interest area of ​​the second output image, and the coordinate information of the area of ​​interest of the second output image, the preprocessed image is determined.

[0251] In some embodiments of the present application, the processing unit 12 is further used to determine the coordinate information of the area of ​​interest in the second output image according to the guidance information; determine the compression rate of the non-interest area in the second output image based on the coordinate information of the area of ​​interest in the second output image; and determine the input coordinate range and the output coordinate range of the non-interest area according to the compression rate of the non-interest area.

[0252] In some embodiments of the present application, the compression rate of the non-interest area includes a first compression rate and a second compression rate; wherein the first compression rate represents the average compression rate in the width direction of the non-interest area in the output image obtained after the local area compression processing is performed on the second output image; the second compression rate represents the average compression rate in the height direction of the non-interest area in the output image obtained after the local area compression processing is performed on the second output image; the processing unit 12 is also used to obtain a first resolution of the second output image and a second resolution of the output image obtained after the local area compression processing is performed on the second output image; according to the first resolution and the second resolution, determine the first area range in the width direction and the second area range in the height direction of the non-interest area of ​​the output image obtained after the local area compression processing is performed on the second output image; determine the first compression rate according to the first area range, the first resolution and the second resolution; determine the second compression rate according to the second area range, the first resolution and the second resolution.

[0253] In some embodiments of the present application, the processing unit 12 is also used to determine, for each third pixel point in the non-interest area of ​​the second output image, a first position ratio of each third pixel point based on the coordinate information of each third pixel point and the input coordinate range; determine a second adjustment parameter based on the position information of the non-interest area of ​​the second output image relative to the interest area of ​​the second output image; adjust the first position ratio corresponding to each third pixel point based on the second adjustment parameter to obtain a second position ratio of each third pixel point; and determine the compressed coordinate information of each third pixel point based on the second position ratio of each third pixel point and the output coordinate range.

[0254] In some embodiments of the present application, the processing unit 12 is further used to reduce the second baseline adjustment parameter to obtain the second adjustment parameter when the starting position in the non-interest area of ​​the second output image is close to the interest area of ​​the second output image; or to increase the second baseline adjustment parameter to obtain the second adjustment parameter when the ending position in the non-interest area of ​​the second output image is close to the interest area of ​​the second output image.

[0255] In some embodiments of the present application, the denoising unit 13 is also used to perform at least one wavelet forward transform processing, at least one convolution processing, at least one residual-based convolution processing, at least one activation processing, at least one pooling processing, at least one feature fusion processing, at least one upsampling processing, and at least one wavelet inverse transform processing on the preprocessed image to obtain the denoised image.

[0256] In some embodiments of the present application, the denoising unit 13 is further used to perform wavelet forward transform processing, activation processing, convolution processing, residual-based convolution processing and pooling processing on the preprocessed image to obtain a first intermediate image; perform the convolution processing, the residual-based convolution processing and the pooling processing on the first intermediate image to obtain a second intermediate image; perform the convolution processing, the residual-based convolution processing and upsampling processing on the second intermediate image at least once to obtain a third intermediate image; perform feature fusion processing on the second intermediate image and the third intermediate image to obtain a fourth intermediate image; perform the residual-based convolution processing, the convolution processing and the upsampling processing on the fourth intermediate image to obtain a fifth intermediate image; perform feature fusion processing on the fifth intermediate image and the first intermediate image to obtain a sixth intermediate image; perform the residual-based convolution processing, the convolution processing, the upsampling processing and the inverse wavelet transform processing on the sixth intermediate image to obtain the denoised image.

[0257] In some embodiments of the present application, the training unit 14 is used to obtain sample data; the sample data includes an original image sample and a denoised sample corresponding to the original image sample; the original image sample is preprocessed by at least one of the global brightness enhancement processing, the corner brightness enhancement processing and the local area compression processing to obtain a preprocessed sample image; the preprocessed sample image is subjected to image denoising processing by using the initial image denoising model to obtain a denoised sample image; the denoised sample image is subjected to inverse processing corresponding to the preprocessing to obtain a sample output image; the sample output image of the original image sample and the corresponding The sample image is input into a preset loss function to obtain a current loss result; the model parameters in the initial image denoising model are modified according to the current loss result to obtain a current intermediate image denoising model; the original image sample is continued to be denoised by the current intermediate image denoising model to obtain the next loss result, and the model parameters in the current intermediate image denoising model are modified according to the next loss result to obtain the next intermediate image denoising model, until the next loss result corresponding to the next intermediate image denoising model meets the cutoff condition, and the next intermediate image denoising model is used as the image denoising model.

[0258] It should be noted that the division of modules or units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. It may also be implemented in the form of a combination of software and hardware.

[0259] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0260] Fig.12A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application, such as Fig.12 As shown, an embodiment of the present application provides an electronic device 20, including:

[0261] A memory 21, used for storing executable instructions;

[0262] The processor 22 is used to implement the above-mentioned image denoising method when executing the executable instructions stored in the memory 21.

[0263] It can be understood that the memory 21 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 21 of the system and method described in the present application is intended to include, but is not limited to, these and any other suitable types of memory.

[0264] The processor 22 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 22. The above processor 22 may be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 21, and the processor 22 reads the information in the memory 21 and completes the steps of the above method in combination with its hardware.

[0265] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units or combinations thereof for performing functions described in the present application. For software implementation, the technology described in the present application can be implemented by a module (such as a process, function, etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in a processor or outside a processor.

[0266] An embodiment of the present application provides a computer storage medium, which stores executable instructions. When the executable instructions are executed, the processor is used to execute the image denoising method described in one or more of the above embodiments.

[0267] In some embodiments, the computer-readable storage medium can be applied to the electronic device in this embodiment, and when the computer program is executed by at least one processor, it implements the corresponding processes implemented by the electronic device in each method of this embodiment. For the sake of brevity, it will not be repeated here.

[0268] An embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps of the image denoising method provided in the above method embodiment.

[0269] In some embodiments, the computer program product can be applied to the electronic device in this embodiment, and the computer program instructions enable the computer to execute the corresponding processes implemented by the electronic device in each method of this embodiment, which will not be repeated here for the sake of brevity.

[0270] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0271] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0272] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0273] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0274] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0275] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0276] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0277] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0278] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0279] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0280] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0281] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0282] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image denoising method, characterized in that: The method comprises: Get the image to be processed; Preprocessing the image to be processed to obtain a preprocessed image; the preprocessing includes at least one of global brightness enhancement processing, corner brightness enhancement processing and local area compression processing; Performing image denoising on the preprocessed image to obtain a denoised image; The denoised image is subjected to an inverse process corresponding to the preprocessing to obtain a target output image.

2. The method according to claim 1, characterized in that The global brightness enhancement process is used to adaptively increase the global brightness of the input image; the corner brightness enhancement process is used to enhance the brightness of the corner positions in the input image; the local area compression process is used to compress the specific area in the input image; the specific area includes the non-interested area of ​​the image; The preprocessing of the image to be processed to obtain the preprocessed image includes: Performing global brightness enhancement processing on the image to be processed to obtain a first output image; Performing edge and corner brightness enhancement processing on the first output image to obtain a second output image; Performing local area compression processing on the second output image to obtain the preprocessed image.

3. The method according to claim 1 or 2, characterized in that: The preprocessing of the image to be processed to obtain a preprocessed image includes at least one of the following: Performing global brightness enhancement processing on the image to be processed to obtain the preprocessed image; Performing edge and corner brightness enhancement processing on the image to be processed to obtain the pre-processed image; Performing local area compression processing on the image to be processed to obtain the preprocessed image.

4. The method according to claim 1 or 2, characterized in that: The preprocessing of the image to be processed to obtain a preprocessed image includes at least one of the following: Performing global brightness enhancement processing and corner brightness enhancement processing on the image to be processed to obtain the pre-processed image; Performing global brightness enhancement processing and local area compression processing on the image to be processed to obtain the preprocessed image; The image to be processed is subjected to edge and corner brightness enhancement processing and local area compression processing to obtain the pre-processed image.

5. The method according to claim 2, characterized in that: The step of performing global brightness enhancement processing on the image to be processed to obtain a first output image includes: Determining a current light intensity index according to exposure information corresponding to the image to be processed; Determine the target gain parameter corresponding to the current light intensity index according to a preset gain parameter relationship; the preset gain parameter relationship represents the corresponding relationship between the light intensity index and the gain parameter; According to the target gain parameter, brightness enhancement processing is performed on each first pixel in the image to be processed to obtain the first output image.

6. The method according to claim 5, characterized in that The step of performing brightness enhancement processing on each first pixel in the image to be processed according to the target gain parameter to obtain the first output image includes: Determining a maximum pixel value according to the number of bits of each first pixel point in the image to be processed; For the brightness value corresponding to each of the first pixels in the image to be processed, determine the processed pixel value of each of the first pixels according to the brightness value of each of the first pixels, the maximum pixel value and the target gain parameter; The first output image is determined according to the processed pixel values ​​of each of the first pixels in the image to be processed.

7. The method according to claim 2, characterized in that The performing corner brightness enhancement processing on the first output image to obtain a second output image includes: For each second pixel in the first output image, determining an average brightness value of a preset area around each second pixel; Determine the processed pixel value of each second pixel according to the average brightness value of each second pixel and the lens shading calibration parameter corresponding to the second pixel in the shading effect correction table; the shading effect correction table represents the correction gain of each second pixel in the second output image under different lens shading effects; The second output image is determined according to the processed pixel values ​​of each of the second pixel points in the first output image.

8. The method according to claim 7, characterized in that The step of determining the processed pixel value of each second pixel point according to the average brightness value of each second pixel point and the lens shadow calibration parameter corresponding to the second pixel point in the shadow effect correction table includes: Determine a minimum calibration parameter among the lens shading calibration parameters in the shading effect correction table; determining a first adjustment parameter corresponding to each of the second pixels according to the position information of each of the second pixels in the first output image and the average brightness value; The processed pixel value of each second pixel point is determined according to the lens shading calibration parameter, the first adjustment parameter and the minimum calibration parameter corresponding to each second pixel point.

9. The method according to claim 8, characterized in that The determining, according to the position information of each second pixel point in the first output image and the average brightness value, a first adjustment parameter corresponding to each second pixel point includes: In the case where the second pixel is located at a corner position in the first output image, if the average brightness value corresponding to the second pixel is less than a preset threshold, the first reference adjustment parameter is increased to obtain the first adjustment parameter; or, If the average brightness value corresponding to the second pixel point is greater than or equal to a preset threshold, the first reference adjustment parameter is used as the first adjustment parameter.

10. The method according to any one of claims 2 to 9, characterized in that: The performing local area compression processing on the second output image to obtain the preprocessed image includes: Acquire guidance information corresponding to the second output image; the guidance information is used to indicate the position of the region of interest in the second output image; Based on the guidance information, determining an input coordinate range and an output coordinate range of a non-interested region in the second output image; According to the input coordinate range and the output coordinate range, compressing the coordinate information of each third pixel in the second output image to obtain compressed coordinate information of each third pixel; The preprocessed image is determined according to the compressed coordinate information of each of the third pixels in the non-interest region of the second output image and the coordinate information of the interest region of the second output image.

11. The method according to claim 10, characterized in that The step of determining an input coordinate range and an output coordinate range of a non-interested region in the second output image based on the guidance information includes: Determining coordinate information of a region of interest in the second output image according to the guidance information; determining a compression rate of a non-interested region in the second output image based on coordinate information of the interested region in the second output image; The input coordinate range and the output coordinate range of the non-interest region are determined according to the compression rate of the non-interest region.

12. The method according to claim 11, characterized in that The compression rate of the non-interest region includes a first compression rate and a second compression rate; wherein the first compression rate represents an average compression rate in a width direction of the non-interest region in the output image obtained after the local area compression processing is performed on the second output image; and the second compression rate represents an average compression rate in a height direction of the non-interest region in the output image obtained after the local area compression processing is performed on the second output image; The step of determining the compression rate of the non-interested region in the second output image based on the coordinate information of the interested region in the second output image comprises: Acquire a first resolution of the second output image, and a second resolution of the output image obtained after performing local area compression processing on the second output image; Determine, according to the first resolution and the second resolution, a first area range in a width direction and a second area range in a height direction of a non-interested area of ​​an output image obtained after performing local area compression processing on the second output image; Determining the first compression rate according to the first area range, the first resolution, and the second resolution; The second compression rate is determined according to the second area range, the first resolution and the second resolution.

13. The method according to claim 10, characterized in that The compressing the coordinate information of each third pixel in the second output image according to the input coordinate range and the output coordinate range to obtain the compressed coordinate information of each third pixel includes: For each third pixel point in the non-interested region of the second output image, determining a first position ratio of each third pixel point according to coordinate information of each third pixel point and the input coordinate range; determining a second adjustment parameter according to position information of a non-interest region of the second output image relative to a region of interest of the second output image; According to the second adjustment parameter, the first position ratio corresponding to each of the third pixel points is adjusted to obtain a second position ratio of each of the third pixel points; The compressed coordinate information of each of the third pixel points is determined according to the second position ratio and the output coordinate range of each of the third pixel points.

14. The method according to claim 13, characterized in that The determining the second adjustment parameter according to the position information of the non-interest region of the second output image relative to the interest region of the second output image includes: When the starting position in the non-interest region of the second output image is close to the interest region of the second output image, the second reference adjustment parameter is reduced to obtain the second adjustment parameter; or, When the end position in the non-interest region of the second output image is close to the interest region of the second output image, the second reference adjustment parameter is increased to obtain the second adjustment parameter.

15. The method according to any one of claims 1 to 14, characterized in that The performing image denoising on the preprocessed image to obtain a denoised image includes: The preprocessed image is subjected to at least one wavelet forward transform process, at least one convolution process, at least one residual-based convolution process, at least one activation process, at least one pooling process, at least one feature fusion process, at least one upsampling process, and at least one wavelet inverse transform process to obtain the denoised image.

16. The method according to claim 15, characterized in that The preprocessed image is subjected to at least one wavelet forward transform process, at least one convolution process, at least one residual-based convolution process, at least one activation process, at least one pooling process, at least one feature fusion process, at least one upsampling process, and at least one wavelet inverse transform process to obtain the denoised image, including: Performing wavelet forward transform processing, activation processing, convolution processing, residual-based convolution processing and pooling processing on the preprocessed image to obtain a first intermediate image; Performing the convolution process, the residual-based convolution process, and the pooling process on the first intermediate image to obtain a second intermediate image; Performing the convolution process, the residual-based convolution process, and the upsampling process on the second intermediate image at least once, to obtain a third intermediate image; performing feature fusion processing on the second intermediate image and the third intermediate image to obtain a fourth intermediate image; performing the residual-based convolution process, the convolution process, and the upsampling process on the fourth intermediate image to obtain a fifth intermediate image; performing feature fusion processing on the fifth intermediate image and the first intermediate image to obtain a sixth intermediate image; The sixth intermediate image is subjected to at least one of the residual-based convolution processing, at least one of the convolution processing, the upsampling processing, and the inverse wavelet transform processing to obtain the denoised image.

17. The method according to any one of claims 1 to 14, characterized in that: The denoised image is obtained by performing image denoising on the preprocessed image using an image denoising model, wherein the image denoising model is obtained by training an initial image denoising model; the method further comprises: Acquire sample data; the sample data includes an original image sample and a denoised sample corresponding to the original image sample; Performing at least one of the global brightness enhancement processing, the edge brightness enhancement processing, and the local area compression processing on the original image sample to obtain a preprocessed sample image; Performing image denoising processing on the preprocessed sample image by using the initial image denoising model to obtain a denoised sample image; Performing an inverse process corresponding to the preprocessing on the denoised sample image to obtain a sample output image; Inputting the sample output image of the original image sample and the corresponding denoised sample into a preset loss function to obtain a current loss result; Modifying the model parameters in the initial image denoising model according to the current loss result to obtain a current intermediate image denoising model; Continue to perform image denoising on the original image sample through the current intermediate image denoising model to obtain the next loss result, and modify the model parameters in the current intermediate image denoising model according to the next loss result to obtain the next intermediate image denoising model, until the next loss result corresponding to the next intermediate image denoising model meets the cutoff condition, and use the next intermediate image denoising model as the image denoising model.

18. An image denoising device, characterized in that: The image denoising device comprises an acquisition unit, a processing unit and a denoising unit; wherein, The acquisition unit is used to acquire the image to be processed; The processing unit is used to preprocess the image to be processed to obtain a preprocessed image; the preprocessing includes at least one of global brightness enhancement processing, corner brightness enhancement processing and local area compression processing; The denoising unit is used to perform image denoising on the preprocessed image to obtain a denoised image; The processing unit is further used to perform inverse processing corresponding to the preprocessing on the denoised image to obtain a target output image.

19. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the image denoising method described in any one of claims 1 to 17 when executing the executable instructions stored in the memory.

20. A computer-readable storage medium, characterized in that: The storage medium stores executable instructions, and when the executable instructions are executed by one or more processors, the image denoising method according to any one of claims 1 to 17 is implemented.

Images