Image noise reduction method, electronic equipment, readable storage medium and program product

Abstract

The invention relates to the technical field of images, in particular to an image noise reduction method, electronic equipment, a readable storage medium and a program product. According to the method, filtering parameters of a filtering formula are obtained through optical parameters of an image to be denoised and a filtering coefficient model. And obtaining a noise reduction error of the to-be-denoised image after being processed by the noise reduction algorithm through the filtering formula. And correcting the image processed by the noise reduction algorithm according to the noise reduction error to obtain a noise-free image of the image to be subjected to noise reduction. Therefore, some details in the image can be reserved or more image noisy points can be removed based on the image noise reduction method, and the noise reduction effect of the image to be subjected to noise reduction can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image technology, and in particular to an image noise reduction method, electronic equipment, a readable storage medium and a program product. Background Art

[0002] Using electronic devices to take photos has become a way of recording in people's daily lives. When taking photos, various noises will be introduced because the electronic device image is affected by the sensor material properties, components, circuit structure, and working environment during the acquisition process. Alternatively, during the image signal transmission process, the image will be contaminated by various noises, such as Gaussian noise or Poisson noise, due to the influence of the transmission medium and recording equipment.

[0003] At present, the method for processing image noise is generally to perform noise reduction on the image, such as Gaussian filtering algorithm, median filtering algorithm and non-local mean filtering algorithm. However, due to the problem of processing power, the current noise reduction algorithm has noise reduction errors, that is, the image after noise reduction has image details and texture features lost (hereinafter referred to as smearing) or the noise is not removed cleanly. Summary of the invention

[0004] The embodiments of the present application provide an image denoising method, electronic device, readable storage medium and program product, which can correct the denoising error in the initial denoising image to the image processed by the denoising algorithm to obtain a target denoised image that retains complete details and features.

[0005] In a first aspect, the embodiments of the present application provide an image denoising method, an electronic device and a readable storage medium. The method comprises: obtaining a first image to be denoised; performing a first denoising process on the first image to obtain a second image; calculating a denoising error of the first denoising process based on image optical parameters of the first image; and correcting the second image based on the denoising error to obtain a first denoised image of the first image.

[0006] Exemplarily, the first image to be denoised may be acquired by the first electronic device, for example, by photographing the first electronic device, or by receiving an image or video sent by other electronic devices by the first electronic device, which is not limited here.

[0007] It can be understood that when the first image is subjected to the first processing, there is a noise reduction error, and there is a mapping relationship between the noise reduction error of the first noise reduction processing and the image optical parameters of the first image. Therefore, the noise reduction error of the first noise reduction processing can be calculated based on the image optical parameters of the first image, such as the brightness, sensitivity, and pixels of the first image. And based on the noise reduction error, the second image is corrected to obtain a first noise reduction image of the first image.

[0008] In this way, based on the above image denoising method, some details in the image to be denoised can be retained or more image noise points can be removed, which is beneficial to improving the denoising effect of the image to be denoised.

[0009] In a possible implementation of the first aspect above, the first noise reduction process includes at least one of the following: a Gaussian filtering algorithm, a median filtering algorithm, and a non-local mean filtering algorithm.

[0010] In a possible implementation of the first aspect above, a denoising error of a first denoising process is calculated based on image optical parameters of the first image, including: determining a weight coefficient and a filtering frequency of a first filtering formula based on the image optical parameters of the first image, wherein the first filtering formula is used to calculate the denoising error of the first denoising process; and calculating the denoising error of the first denoising process based on the weight coefficient and the filtering frequency of the first filtering formula in combination with the first filtering formula.

[0011] In a possible implementation of the first aspect above, based on the weight coefficient and the filtering frequency of the first filtering formula, in combination with the first filtering formula, a denoising error of the first denoising process is calculated, including: performing a first filtering process on the denoised image using a filter of a first frequency to obtain a first filtering result; and performing a second filtering process on the denoised image using a filter of a second frequency to obtain a second filtering result, wherein the first frequency is different from the second frequency; based on a first weight coefficient corresponding to the filter of the first frequency and a second weight coefficient corresponding to the filter of the second frequency, linearly superimposing the first filtering result and the second filtering result to obtain a denoising error of the second image.

[0012] It can be understood that the above noise map includes one or more noise signals of different frequency bands, and each frequency band corresponds to a different noise reduction error. Therefore, the noise signals of one or more different frequency bands can be filtered out by setting one or more filters of different frequencies to obtain the noise reduction errors of the corresponding frequency bands. Finally, the noise reduction errors of the above different frequency bands are linearly superimposed to obtain the noise reduction errors of the image to be processed. In this way, based on the above-mentioned form of segmented filtering, the noise reduction errors of different frequency bands can be obtained from the noise map, and the noise reduction errors of the above frequency bands can be linearly superimposed through different weight coefficients.

[0013] In a possible implementation of the first aspect, the first filtering formula is:

[0014] addnoise=noisegain*(alpha_1*filter_1(noisemap)+alpha_2

[0015] *filter_2(noisemap)+…+alpha_n*filter_n(noisemap))

[0016] Among them, addnoise represents the noise reduction error of the second image; noisemap represents the noise reduction map; noisegain represents the gain value of the noise reduction error; alpha_1 to alpha_n represent the weight coefficients corresponding to filters of different frequencies; filter_1 to filter_n represent different filtering frequencies of multiple filters.

[0017] In a possible implementation of the first aspect above, a method for determining the weight coefficient and the filter frequency includes: taking the image optical parameters of the first image and the second image as input data, and inputting the filter coefficient model; based on the output result of the filter coefficient model, determining the weight coefficient and the filter frequency; wherein the filter coefficient model is obtained based on training of a convolutional neural network model.

[0018] It can be understood that the weight coefficient and the frequency coefficient are related to the image optical parameters of the image, such as pixels and sensitivity. However, the relationship between the weight coefficient and the frequency coefficient, and the pixels and sensitivity of the image is relatively complex, and it is difficult to obtain the correct relationship through simple formula modeling. Therefore, the relationship between the weight coefficient, the frequency coefficient, and the pixels and sensitivity of the image can be fitted by fitting the filter coefficient model.

[0019] It can be understood that the filter coefficient model obtained by training the convolutional neural network model has higher accuracy in outputting weight coefficients and filter frequencies and faster output efficiency.

[0020] In a possible implementation of the first aspect above, the method for acquiring the filter coefficient model includes: the first electronic device trains the filter coefficient model based on training data obtained by processing the acquired training sample set; or the first electronic device deploys the filter coefficient model provided by the second electronic device, wherein the filter coefficient model is trained by the second electronic device based on training data obtained by processing the acquired training sample set.

[0021] It can be understood that the above-mentioned filter coefficient model can be obtained by training the first electronic device based on the training data obtained by processing the acquired training sample set, or it can be a filter coefficient model provided by the first electronic device and the second electronic device, which is not limited here.

[0022] In a possible implementation of the first aspect above, the training process of the filter coefficient model includes: obtaining a training sample set, the training sample set including a first sample image set, a sensitivity corresponding to the first sample image set, and a second sample image set; training a first model based on the second sample image in the second sample image set and the sensitivity corresponding to the first sample image set to obtain a filter coefficient model, wherein the first model includes a convolutional neural network model; obtaining a third weight coefficient and a third frequency and a third weight coefficient and a third frequency of the corresponding filter output by the first model; performing a third filtering process on a third sample image in the third sample image set using a first filter of a third frequency to obtain a third filtering result; and performing a fourth filtering process on the third sample image in the third sample image set using a second filter of a fourth frequency to obtain a third filtering result, wherein the third frequency is different from the fourth frequency; linearly superimposing the third filtering result and the fourth filtering result based on the third weight coefficient corresponding to the filter of the third frequency and the fourth weight coefficient corresponding to the filter of the fourth frequency to obtain a denoising error of the second sample image set; wherein the noise data of the third sample image set is calculated based on the difference between the first sample image set data and the second sample image set data.

[0023] It can be understood that the second sample images in the second sample image set and the sensitivities corresponding to the first sample image set are used as training data to train the first model and obtain the filter coefficient model.

[0024] In a possible implementation of the first aspect above, the training process of the filter coefficient model also includes: based on the denoising error of the second sample image set, correcting the second sample image in the second sample image set to obtain a fourth sample image set; based on the first loss function, determining a first similarity between the fourth sample image set and the first sample image set; wherein the first loss function is used to calculate the difference between the fourth sample image set data and the first sample image set data; and based on the first similarity, adjusting the model parameters of the first model to obtain the filter coefficient model.

[0025] It can be understood that the second sample image after denoising is corrected according to the denoising error to obtain the fourth sample image set of the first sample image set. Then the difference between the fourth sample image set data and the first sample image set data is calculated by the first loss function, and the training parameters of the first model are adjusted in time to obtain the filter coefficient model

[0026] In a possible implementation of the first aspect above, the training sample set includes a first sample image set, a sensitivity corresponding to the first sample image set, and a second sample image set, and obtaining the training sample set includes: obtaining the first sample image set and the first sensitivity corresponding to the first sample image set; preprocessing the first sample image set to obtain the second sample image set.

[0027] It can be understood that the first sample image set is a noise-free image set, so the first sample image set needs to be preprocessed to obtain the second sample image set. The second sample image set is the first sample image set that has been preprocessed and includes noise data. The noise data includes a noise signal to reduce noise errors.

[0028] In a possible implementation of the first aspect above, the first sample image set is preprocessed to obtain the second sample image set, including: performing degradation processing and noise adding processing on the first sample image set based on the first degradation parameter and the first noise parameter to obtain the first sample image set containing noise; and then performing noise reduction processing on the first sample image set containing noise based on the second noise reduction processing to obtain the second sample image set.

[0029] It can be understood that the above preprocessing process is to simulate the generation of degraded data and noise data in the real transmission process of the image. Specifically, the first sample image set can be degraded based on the preset degradation parameters, such as through a convolution operation or a blur filter of an application in an electronic device. Then, based on the preset noise parameters, noise data, such as Gaussian noise, Poisson noise, etc., are added to the first sample image set after the degradation process, and the first sample image set with noise is obtained after the above degradation process and noise addition process. Then, the first sample image set with noise is subjected to noise reduction processing (i.e., the second noise reduction processing) to obtain the second sample image set and the third sample image set. Among them, the noise data of the third sample image set is calculated based on the difference between the first sample image set data and the second sample image set image data.

[0030] The second noise reduction process includes at least one of a Gaussian filter algorithm, a median filter algorithm and a non-local mean filter algorithm.

[0031] In a possible implementation of the first aspect above, the image optical parameters include sensitivity, brightness, and pixel value of the first image.

[0032] In a second aspect, an embodiment of the present application provides a model training method, which is applied to a first electronic device or a second electronic device, the method comprising: obtaining a training sample set, the training sample set comprising a first sample image set, a first sensitivity corresponding to the first sample image set, and a second sample image set, wherein the first sample image in the first sample image set corresponds one-to-one to the second sample image in the second sample image set, and the second sample image is obtained based on degradation processing, noise addition processing and noise reduction processing of the first sample image; using the second sample image and the first sensitivity in the second sample image set, training the network model to obtain a filter coefficient model.

[0033] It can be understood that the filter coefficient model obtained by training the convolutional neural network model has higher accuracy in outputting weight coefficients and filter frequencies and faster output efficiency.

[0034] In a possible implementation of the second aspect above, based on the third sample image set and the filter coefficient, combined with the first filtering formula, a denoising error of the denoising processing is obtained; wherein the noise data of the third sample image set is calculated based on the difference between the first sample image set data and the second sample image set data; based on the denoising error, the second sample image set is corrected to obtain a fourth sample image set; based on the first sample image set data, the fourth sample image data and the first loss function, the training parameters of the network model are adjusted to obtain a filter coefficient model.

[0035] It can be understood that the difference between the fourth sample image set data and the first sample image set data is calculated through the first loss function, and the training parameters of the network model are adjusted in time, so that the weight coefficient and filter coefficient output by the filter coefficient model are more accurate, and therefore the noise reduction error obtained based on the weight coefficient, filter coefficient and the first filtering formula is more accurate.

[0036] In a possible implementation of the second aspect, the first sample image set is a noise-free image set.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; one or more memories; one or more memories storing one or more programs, wherein when one or more programs are executed by one or more processors, the electronic device executes the image denoising method provided in the first aspect and various possible implementations of the first aspect, or the model training method provided in the second aspect and various possible implementations of the second aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the computer executes the image denoising method provided in the first aspect and various possible implementations of the first aspect, or the model training method provided in the second aspect and various possible implementations of the second aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a program product, and the computer program product includes: a computer program code, when the computer program code runs on a computer, enables the computer to implement the image denoising method as described in the first aspect and any one of the various implementations of the first aspect, or implement the model training method as described in the second aspect and any one of the various implementations of the second aspect.

[0040] The beneficial effects of the third to fifth aspects mentioned above can be referred to the relevant descriptions in various possible implementations of the first and second aspects mentioned above, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of the process of ISP 000 processing an image is shown;

[0042] Figure 2a According to the present application, a schematic diagram of an image without noise reduction processing is shown;

[0043] Figure 2b According to the present application, a schematic diagram of an image after noise reduction processing is shown;

[0044] Figure 2c According to the present application, another schematic diagram of an image after noise reduction processing is shown;

[0045] Figure 3 According to the present application, a schematic diagram of an implementation process of an image noise reduction method is shown;

[0046] Figure 4 According to an embodiment of the present application, a specific implementation flow chart of image noise reduction is shown;

[0047] Figure 5 According to an embodiment of the present application, a schematic diagram of a scenario for obtaining an image to be processed is shown;

[0048] Figure 6 According to an embodiment of the present application, a training process diagram of a filter coefficient model training method is shown;

[0049] Figure 7 A hardware structure diagram of an electronic device 100 is shown according to an embodiment of the present application;

[0050] Figure 8 According to an embodiment of the present application, a software system architecture diagram of an electronic device 100 is shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0052] It should also be stated that the steps in the methods and processes in the embodiments of the present application are numbered for ease of reference, rather than to limit the order of precedence. If there is an order between the steps, the text description shall prevail.

[0053] Combine the following Figure 1The specific implementation process of image signal processor (ISP) processing images is described.

[0054] Figure 1 Figure 1 is a schematic diagram of the image processing process in ISP 000. Figure 1 As shown, the original image data entering ISP 000 for processing passes through the following modules in ISP 000 in sequence: image preprocessing module 01, image enhancement processing module 02, color correction module 03, and image compression processing module 04.

[0055] Among them, the image preprocessing module 01 will preprocess the collected raw image data to improve the image quality. It mainly includes operations such as denoising, color correction, white balance, color enhancement, and geometric transformations such as cropping, rotating, and scaling the original image. The image noise reduction process mentioned in the embodiment of the present application is carried out in the ISP000 image preprocessing module 01. After the original image is preprocessed, the image enhancement processing module 02 can also enhance the image to improve the details, contrast, and saturation of the original image. It mainly includes operations such as sharpening, edge enhancement, and local contrast enhancement of the original image. Further, the color correction module 03 can convert the original image from one color space to another color space, such as from RGB color space to YUV color space, or from YUV color space to RGB color space. The image compression processing module 04 can compress the original image to reduce the size of the original image data for easy storage and transmission. Common compression algorithms include JPEG, H.264, etc.

[0056] Figure 2a According to the present application, a schematic diagram of an image without noise reduction processing is shown. Figure 2a As shown, image 101 shows leaf texture features of green plants (e.g. Figure 2a A” in the figure) and multiple noise points A. After being processed by a noise reduction algorithm, such as a Gaussian filter algorithm, the noise points may not be completely removed due to the insufficient noise reduction capability of the noise reduction algorithm. Figure 2b There are still many noise points A' in the image 102 shown in FIG. Figure 2c The image 103 in the figure shows excessive noise reduction, resulting in Figure 2b The texture characteristics of green leaves are lost (i.e. Figure 2c A”' in ), can also be called image smearing.

[0057] As mentioned above, when the current denoising algorithm performs denoising on the denoised image, problems such as incomplete noise removal or excessive denoising resulting in smearing of the image may occur.

[0058] In order to solve the above problems, it is necessary to first analyze what parts are removed by the denoising algorithm when the denoising image is subjected to denoising. Generally, the image to be denoised acquired by the electronic device includes Poisson noise and Gaussian noise. When the denoising image is subjected to denoising, there are generally two types of denoising errors. One is that the denoising algorithm is not capable enough, resulting in insufficient denoising, that is, the noise is not removed cleanly. The other is excessive denoising, resulting in the loss of details and texture features in the image to be denoised, that is, the image is smeared. Therefore, when the denoising algorithm is subjected to denoising on the denoised image, the initial noise map obtained may include incomplete noise signals, or may include complete noise signals-image signals. Therefore, the signal content in the above initial noise map can be summarized as noise signals and denoising error signals.

[0059] The initial denoised image is obtained by spectral subtraction, that is, the image to be denoised is subtracted from the image processed by the denoising algorithm to obtain the initial denoised image.

[0060] It should be understood that Poisson noise and Gaussian noise are affected by brightness (LUMA) and sensitivity (ISO). The relationship between the noise signal and brightness and sensitivity can be expressed by formula (1):

[0061] Noise signal = f(ISO, LUMA, ERR) (1)

[0062] Among them, ISO represents the sensitivity of the image; LUMA represents the brightness of the image; ERR represents the noise reduction error; f represents the mapping relationship between sensitivity, brightness, noise reduction error and noise signal.

[0063] It can be understood that the above initial noise image includes one or more noise signals of different frequency bands, and each frequency band corresponds to a different noise reduction error. Therefore, the noise signals of one or more different frequency bands can be filtered out by setting one or more filters of different frequencies to obtain the noise reduction error of the corresponding frequency band. Finally, the noise reduction errors of the above different frequency bands are linearly superimposed to obtain the noise reduction error (ERR) of the image to be processed.

[0064] Therefore, the noise reduction error can be obtained by the first filtering formula (2):

[0065] addnoise=noisegain*(alpha_1*filter_1(noisemap)+alpha_2*

[0066] filter_2(noisemap)+alpha_3*filter_3(noisemap)+…+alpha_n*

[0067] filter_n(noisemap))(2)

[0068] Among them, addnoise represents the denoising error of the second image; noisemap represents the initial denoising map; noisegain represents the gain value of the denoising error; alpha_1 to alpha_n represent the weight coefficients corresponding to filters of different frequencies; filter1 to filter_n represent different filtering frequencies of multiple filters.

[0069] It can be understood that the gain value of the above-mentioned noise reduction error can dynamically adjust the noise reduction error according to the user's requirements for image processing.

[0070] Based on the above analysis, the present application provides an image denoising method. In this method, a neural network model is trained to obtain the above weight coefficients and filtering frequencies based on the image, the brightness and sensitivity of the image, so as to obtain the corresponding weight coefficients and filtering frequencies, such as the weight coefficients and filtering frequencies in the above first filtering formula (2). Then, the denoising error of the image to be processed is obtained based on the above first filtering formula, and finally the denoising error is corrected to the image to obtain the target denoised image.

[0071] It should be understood that the image denoising method provided in the present application is to correct the denoising error in the initial denoising image to the image processed by the denoising algorithm, so as to obtain a target denoised image that retains complete details and features.

[0072] In some optional embodiments, the filter coefficient model may be a neural network model obtained through training with a large amount of data. The specific training process may be: collecting a sample image set (i.e., a first sample image set, denoted as GTs), and performing degradation processing and noise adding processing on the sample image set, i.e., first performing degradation processing on the sample image set, such as through convolution operation or application of blur filter, etc. Then adding noise, such as Gaussian noise, Poisson noise, etc., to the sample image set after degradation processing. After the above degradation processing and noise adding processing, a data set (i.e., a first sample image set with noise, denoted as noise_Imgs) is obtained. Then, the above simulated degradation and noise adding data set is subjected to noise reduction processing to obtain a training data set (train_datas) (i.e., a second sample image set, denoted as clear_Imgs) and a noise set (i.e., a third sample image set, denoted as noise_maps).

[0073] It can be understood that the above data set can be transformed by YUV processing to obtain the Y channel signal before the noise reduction process is performed to determine the brightness of the data set. For example, the image in the data set is transformed into a color space by the aforementioned color correction module 03. However, the brightness of each pixel position in the image is different, so the brightness is replaced by image pixels in the embodiment of the present application. In some embodiments, the RGB signal of the image can also be used directly to obtain the pixels of the image, which is not limited here. Among them, RGB (red, green, blue) color space or RGB domain refers to a color model related to the structure of the human visual system. According to the structure of the human eye, all colors are regarded as different combinations of red, green and blue.

[0074] It can be understood that the training set [(train_datas, noise_maps), GTs] required for training is constructed through the above process. Figure 3 As shown, the training data set and the corresponding sensitivity are input into the training network (i.e. Figure 3 In the network), the filter coefficient of the filter formula is obtained through the network output. For example, taking Gaussian filtering as an example, the network outputs the gamma coefficient (gamma) and weight coefficient of the Gaussian filter. Among them, the above gamma coefficient is used to represent the filter frequency of the Gaussian filter. Then based on the preset filter formula (i.e. Figure 3 ), perform segmented filtering on the noise set (i.e., the third sample image set, denoted as noise_maps), and determine the denoising error of the denoising process.

[0075] Furthermore, the images in the training data are corrected based on the denoising error to obtain a noise-superimposed image set (i.e., the fourth sample image set, denoted as en_Imgs). The similarity between the image set and the images in the first sample image set is calculated through the loss function (LOSS), the training model direction is continuously optimized, and the filter coefficient model is corrected.

[0076] It can be understood that the embodiments of the present application are applicable to electronic devices, which may be mobile phones, smart screens, tablet computers, wearable electronic devices, vehicle-mounted electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), projectors, and the like.

[0077] Understandably, Figure 4 According to an embodiment of the present application, a specific implementation flow chart of image noise reduction is shown.

[0078] Understandably, Figure 4 The execution entities of each step of the process shown can all be electronic devices such as mobile phones, tablet computers or laptop computers. The execution entities of each step will not be repeatedly introduced when introducing the specific execution content of each step below.

[0079] like Figure 4 As shown, the process includes the following steps:

[0080] 401: Obtain a first image to be denoised and a corresponding sensitivity.

[0081] For example, taking mobile phone 200 as an example, refer to Figure 5 As shown in (A) of FIG. 1 , the user clicks the camera application icon 101 of the mobile phone 200 by performing operation (1). Correspondingly, the mobile phone 200 obtains the camera shooting interface in response to the user operation. Figure 5 As shown in (B), the user selects a shooting mode based on shooting needs, performs operation 2, and clicks the shooting control 102. The mobile phone 200 acquires and stores the image in response to the click operation of operation 2. Figure 5 The shooting mode in (B) can be a photo mode, a video mode, etc. It should be understood that the above Figure 5 The image acquisition scenario shown is for illustration only and does not impose any restrictions on the application scenarios of the present application. The image denoising method provided in the embodiment of the present application can be applied to but not limited to the following scenarios: video calls, video conferencing applications, long and short video applications, video live broadcast applications, video online course applications, smart mirror application scenarios, system camera recording function video recording, video surveillance, and smart cat-eye and other shooting scenarios, etc., without limitation here. After performing operation 2, the mobile phone obtains the picture to be denoised, and at the same time, the corresponding sensitivity can be obtained through the shooting parameters during the shooting process.

[0082] 402: Perform noise reduction processing on the first image to be denoised to obtain a second image.

[0083] Exemplarily, the above-mentioned denoising process for the denoised image can be implemented by any one of a Gaussian filter algorithm, a median filter algorithm and a non-local means (NLM) filter algorithm. Among them, the Gaussian filter algorithm is a linear smoothing filter algorithm, and the Gaussian filter can perform weighted averaging on the entire image. The value of each pixel is obtained by weighted averaging the value of itself and other pixels in the neighborhood.

[0084] The median filter algorithm is a nonlinear smoothing filter algorithm. The median filter can replace the value of a pixel with the median value of each pixel in a neighborhood of the pixel.

[0085] The NLM algorithm sets two fixed-size windows, a large search window (D×D) and a small neighborhood window (d×d), slides the neighborhood window in the search window, and determines the influence of the corresponding central pixel on the current pixel based on the similarity between the neighborhoods. In some embodiments, the above-mentioned denoising process for the denoised image may also be other filtering algorithms or a combination of multiple filtering algorithms, which is not limited here.

[0086] It can be understood that the image to be denoised is processed by any one or more of the above algorithms to obtain the second image.

[0087] 403: Calculate a noise reduction error of a first noise reduction process based on image optical parameters of the first image to be noise reduced.

[0088] Exemplarily, the image optical parameters of the image to be denoised may be, for example, brightness, sensitivity and pixels of the image to be denoised.

[0089] It can be understood that the initial denoised image of the first image needs to be obtained when executing step 403. The initial denoised image is obtained by spectral subtraction, that is, by subtracting the second image from the first image.

[0090] In some optional embodiments, in order to facilitate calculation, the weight coefficients in the above formula (2) can be set to a map (i.e., a weight map, denoted as alpha_map), and the corresponding size can be the same as the noise image. The specific process can be, for example: create an empty map object and name the empty map alpha_map. Then obtain the size of the initial noise map. For example, the size function of the initial noise map can be used to obtain the number of rows and columns. Then use the adjustment function of alpha_map to set the size of the weight map to be the same as the initial noise image. The adjustment function accepts the two data of the number of rows and columns, uses a two-layer loop to traverse each element of the initial noise map, and assigns its value to the element at the corresponding position of the weight map. Among them, the adjustment function can change the size of the weight map. In other embodiments, some other reasonable methods can also be used to make the corresponding size of the weight map the same as the initial noise image, which is not limited here. In this way, the above formula (2) can also be expressed as:

[0091] addnoise=noisegain*(alpha_map1*filter_1(noisemap)+alpha_map2

[0092] *filter_2(noisemap)+alpha_map3*filter_3(noisemap)+…

[0093] +alpha_n*filter_n(noisemap))

[0094] The meanings of the terms in the formula have been explained in detail above and will not be repeated here.

[0095] It can be understood that the corresponding relationship between the weight map and the brightness and sensitivity of the image can be expressed by formula (3):

[0096] alpha_map = f(LUMA, ISO) (3)

[0097] Among them, LUMA represents the brightness of the image, ISO represents the sensitivity of the image, and f represents the mapping relationship between sensitivity, brightness and weight map.

[0098] As mentioned above, the brightness of each pixel position in the image is different, and the image pixel can be used to replace the brightness. Therefore, the corresponding relationship between the weight map and the brightness and sensitivity of the image can also be expressed by formula (3):

[0099] alpha-map = f(PX, ISO) (4)

[0100] Among them, PX represents the constituent pixels of the image, ISO is the sensitivity of the image, and f represents the mapping relationship between sensitivity, pixels and weight map.

[0101] It can be understood that the selection of filtering frequency of different filtering methods is affected by the image information (texture, color, etc.) and noise. Taking Gaussian filtering as an example, Gaussian filtering can select a Gaussian kernel of a fixed size, such as kernel_size = 5, and the gamma coefficient corresponding to the Gaussian filter is related to the pixels and sensitivity of the image. Therefore, the corresponding relationship between the gamma coefficient and the pixels and sensitivity of the image can be expressed by formula (5):

[0102] gamma = g(PX, ISO) (5)

[0103] Among them, g represents the mapping relationship between sensitivity, pixel and gamma coefficient, gamma is the gamma coefficient, which can represent the filtering frequency and filtering effect of Gaussian filtering, and ISO is the sensitivity of the image.

[0104] As mentioned above, it can be known from formula (4) and formula (5) that the weight coefficient and the gamma coefficient are both related to the pixels and sensitivity of the image. However, the relationship between the weight coefficient and the gamma coefficient, and the pixels and sensitivity of the image is relatively complex, and it is difficult to obtain the correct relationship through simple formula modeling. Therefore, in the embodiment of the present application, the relationship between the weight coefficient, the gamma coefficient and the pixels and sensitivity of the image can be fitted by fitting the filter coefficient model. Finally, the noise reduction error is calculated by the above formula (2). Among them, the above filter coefficient model can be obtained by, for example, training a convolutional neural network model. The specific training process is described below. Figure 6 It should be understood that the filter coefficient model obtained by training the convolutional neural network model has higher accuracy in outputting weight coefficients and filter frequencies and faster output efficiency.

[0105] 404: Correct the second image based on the denoising error to obtain a first denoised image of the first image.

[0106] It can be understood that the second image is corrected based on the noise reduction error to obtain the first noise reduction image of the first image. For example, the second image can be linearly processed based on the noise reduction error. In this way, the image obtained based on the above image noise reduction method retains complete texture features and has a strong noise reduction capability.

[0107] It can be understood that the above-mentioned filter coefficient model can be synchronously added to the algorithm library or model library that can be called by the ISP together with other image processing algorithms.

[0108] In some optional embodiments, Figure 6 According to an embodiment of the present application, a training process diagram of a filter coefficient model training method is shown.

[0109] Understandably, Figure 6 The execution entities of each step of the process shown can all be electronic devices such as mobile phones, tablet computers or laptop computers. The execution entities of each step will not be repeatedly introduced when introducing the specific execution content of each step below.

[0110] like Figure 6 As shown, the training process may include the following steps:

[0111] 601: Obtain a first sample image set and the sensitivity corresponding to the first sample image set.

[0112] It can be understood that the first sample image set may be a noise-free image set, and the sensitivity corresponding to the first sample image set may be a shooting parameter of the electronic device when shooting the first sample image set.

[0113] 602: Preprocess the first image sample set to obtain a second sample image set and a third sample image set.

[0114] In some embodiments, the preprocessing of the first image sample set includes degradation processing, noise adding processing and noise reduction processing. The specific preprocessing process may be based on, for example: based on preset degradation parameters, the first sample image set is subjected to degradation processing, such as through convolution operation or blur filter of application program in electronic device, etc. Then, based on preset noise parameters, noise data such as Gaussian noise, Poisson noise, etc. are added to the first sample image set after degradation processing, and the first sample image set with noise is obtained after the above-mentioned degradation processing and noise adding processing.

[0115] Furthermore, the first sample image set with noise is subjected to noise reduction processing to obtain a second sample image set and a third sample image set, wherein the noise data of the third sample image set is calculated based on the difference between the first sample image set data and the second sample image set image data.

[0116] The above-mentioned denoising method for denoising the first sample image set with noise can be, for example, any one or more implementations of the Gaussian filtering algorithm, median filtering algorithm and non-local means (NLM) filtering algorithm mentioned in the aforementioned step 402, which will not be described in detail here.

[0117] 603: Based on the second sample image set and the sensitivity of the corresponding first sample image set, the network model is trained to obtain a filter coefficient model.

[0118] It can be understood that, by executing step 601 to step 602, the training set required for training the network model is determined. The second sample image set and the sensitivity of the corresponding first sample image set. Figure 3 , the second sample image and the sensitivity of the corresponding first sample image set are input into the training network (i.e. Figure 3 The network in the network) is trained to obtain a filter coefficient model. The filter coefficient of the filter formula is obtained through the output of the filter coefficient model. For example, taking Gaussian filtering as an example, the network outputs the gamma coefficient and weight coefficient of the Gaussian filter. Among them, the above gamma coefficient is used to represent the filter frequency of the Gaussian filter.

[0119] 604: Based on the third sample image set and the filter coefficient, in combination with the first filter formula, obtain a noise reduction error of the noise reduction process.

[0120] The first filtering formula may be, for example, the aforementioned formula (2). The specific calculation process may refer to the relevant description of the aforementioned formula (2), which will not be elaborated here.

[0121] It can be understood that the first filtering formula (ie Figure 3The first filtering formula shown in ), the third sample image set is subjected to segmented filtering processing, and the weight coefficients of each filter are used to linearly superimpose the multiple filtering results to obtain the denoising error of the denoising processing.

[0122] 605: Based on the noise reduction error, correct the second sample image set to obtain a fourth sample image set.

[0123] It can be understood that the second sample image set is corrected based on the noise reduction error to obtain the fourth sample image set. For example, the second sample image set can be linearly processed based on the noise reduction error.

[0124] 606: Based on the first sample image set and the fourth sample image set and in combination with the first loss function, correct the filter coefficient model.

[0125] It can be understood that the second sample image set is corrected based on the noise reduction error to obtain the fourth sample image set with noise superimposed on it. The difference between the fourth sample image set data and the first sample image set data is calculated by the loss function (LOSS), and the similarity between the fourth sample image set and the first sample image set is obtained according to the difference, the training parameters of the model are adjusted, and the model training direction is continuously optimized, and finally the filter coefficient model is obtained.

[0126] As mentioned above, the weight coefficient and filter frequency in formula (2) are related to the pixels and sensitivity of the image. However, the relationship between the weight coefficient, filter coefficient and the pixels and sensitivity of the image is relatively complex, and it is difficult to obtain the correct relationship through simple formula modeling. However, the weight coefficient and filter frequency can be obtained according to the pixels and sensitivity of the image through the above filter coefficient model, and the noise reduction error is finally obtained through formula (2). In addition, the above filter coefficient model calculates the similarity between the fourth sample image set data and the first sample image set data through the loss function (LOSS) during the training process, adjusts the training parameters of the model, and continuously optimizes the training model direction. In this way, the weight coefficient and filter frequency can be quickly and accurately obtained according to the pixels and sensitivity of the image through the above filter coefficient model. And according to the image corrected by the noise reduction error, some details in the image can be retained or more image noise points can be removed, which is conducive to improving the noise reduction effect of the image.

[0127] Figure 7 According to an embodiment of the present application, a hardware structure diagram of an electronic device 100 is shown. In the embodiment of the present application, the electronic device 100 can be the mobile phone 200 mentioned above, or can be other electronic devices, such as a tablet computer.

[0128] like Figure 7As shown, the electronic device 100 may include a processor 110 (or host), a power module 140, a memory 180, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, a camera 170, an interface module 160, a button 101, a display screen 102, and an image signal processor 000, etc.

[0129] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute the only possible implementation method for limiting the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0130] The processor 110 may include one or more processing units, for example, a processing module or processing circuit including a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor, a microprocessor (MCU), an artificial intelligence (AI) processor, or a programmable logic device (FPGA). Among them, different processing units may be independent devices or integrated into one or more processors. A storage unit may also be provided in the processor 110 for storing instructions and data. In some embodiments, the storage unit in the processor 110 is a cache memory 180, which may store instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it may be directly called from the storage unit, thereby avoiding repeated access, reducing the waiting time of the processor 110, and thus improving the efficiency of the system.

[0131] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0132] The power module 140 may include a power source, a power management component, etc. The power source may be a battery. The power management component is used to manage the charging of the power source and the power supply of the power source to other modules. In some embodiments, the power management component includes a charging management module and a power management module. The charging management module is used to receive charging input from the charger; the power management module is used to connect the power source, and the charging management module is connected to the processor 110. The power management module receives input from the power source and / or the charging management module, and supplies power to the processor 110, the display screen 102, the camera 170, and the wireless communication module 120.

[0133] The mobile communication module 130 may include, but is not limited to, an antenna, a power amplifier, a filter, a low noise amplifier (LAN), etc. The mobile communication module 130 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to terminal devices. The mobile communication module 130 may receive electromagnetic waves by an antenna, and perform filtering, amplification, and other processing on the received electromagnetic waves, and transmit them to a modulation and demodulation processor for demodulation. The mobile communication module 130 may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, at least some of the functional modules of the mobile communication module 130 may be arranged in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 130 may be arranged in the same device as at least some of the modules of the processor 110.

[0134] The wireless communication module 120 may include an antenna, and transmit and receive electromagnetic waves via the antenna. The wireless communication module 120 may provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., which are applied to the terminal device. The terminal device can communicate with the network and other devices through wireless communication technology.

[0135] In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the terminal device may also be located in the same module.

[0136] The display screen 102 is used to display a human-computer interaction interface, images, videos, etc. The display screen 102 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), a quantum dot light emitting diode (QLED), etc.

[0137] The sensor module 190 may include a proximity light sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, an electromagnetic wave absorption rate sensor, a sound sensor, an image sensor, etc. Among them, the image audio module 150 is used to convert a digital audio signal into an analog audio signal output, or to convert an analog audio input into a digital audio signal. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 can be set in the processor 110, or some functional modules of the audio module 150 can be set in the processor 110. In some embodiments, the audio module 150 may include a speaker, an earpiece, a microphone, and a headphone interface.

[0138] The camera 170 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then passed to the image signal processing (ISP) to be converted into a digital image signal. The terminal device can implement the shooting function through the ISP, the camera 170, the video codec, the graphic processing unit (GPU), the display screen 102, and the application processor.

[0139] The image signal processor 000 may include Figure 1 The image preprocessing module, image enhancement processing module, color correction module, and image compression processing module shown in the figure. The image signal processor 000 is used to process the digital image signal collected by the image sensor. In some embodiments, the image signal processor 000 can be used to execute the image noise reduction method provided in the embodiment of the present application.

[0140] The interface module 160 includes an external memory interface, a universal serial bus (USB) interface, etc. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 110 through the external memory interface to implement the data storage function. The universal serial bus interface is used for the terminal device to communicate with other electronic devices.

[0141] In some embodiments, the terminal device further includes a button 101, a motor, and an indicator. The button 101 may include a volume button, an on / off button, etc. The motor is used to make the terminal device produce a vibration effect, and different application scenarios (e.g., time reminder, receiving information, alarm clock, game, etc.) may correspond to different vibration feedback effects. The indicator may include a laser indicator, a radio frequency indicator, a light emitting diode (LED) indicator, etc.

[0142] The above describes the hardware structure of the electronic device in detail. The following is a system structure block diagram of the electronic device.

[0143] It is understood that the operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes the Android system of the layered architecture as an example to exemplify the system software structure of the electronic device 100.

[0144] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, Android TM The system is divided into four layers, from top to bottom: application layer, application framework layer, Android TM Runtime (Android TM runtime) and system libraries, as well as the kernel layer.

[0145] like Figure 8 As shown, the application layer may include a series of application packages, which may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0146] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0147] The application framework layer may include a window manager, content provider, view system, telephony manager, resource manager, notification manager, and the like.

[0148] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0149] Content providers are used to store and retrieve data and make it accessible to applications. This data can include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0150] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0151] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0152] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0153] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0154] Android TM Runtime includes core libraries and virtual machines. Android TM runtime is responsible for Android TM Scheduling and management of the system.

[0155] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0156] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0157] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as openGLES), 2D graphics engine (such as SGL), etc.

[0158] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0159] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0160] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0161] A 2D graphics engine is a drawing engine for 2D drawings.

[0162] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0163] The following is an illustrative description of the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0164] When the touch sensor in the sensor module 190 receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is a control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a static image or video through the camera 170.

[0165] The embodiment of the present application also provides a computer program product for implementing the image noise reduction method provided in the above embodiments.

[0166] The present application provides a readable storage medium, wherein the readable storage medium stores instructions, and when the instructions are executed on an electronic device, the electronic device executes the image noise reduction method mentioned in the present application.

[0167] The various embodiments disclosed in the present application may be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application may be implemented as a computer program or program code executed on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0168] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor, a microcontroller, an application specific integrated circuit, or a microprocessor.

[0169] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0170] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program module or module code executed on a programmable system, and the programmable system includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0171] A computer program module or module code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0172] Module code can be implemented with high-level modular language or object-oriented programming language to communicate with the processing system. When necessary, module code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0173] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0174] References to "one embodiment" or "an embodiment" in the specification mean that the specific features, structures, or characteristics described in conjunction with the embodiment are included in at least one exemplary implementation or technology disclosed according to the embodiment of the present application. The appearance of the phrase "in one embodiment" in various places in the specification does not necessarily all refer to the same embodiment.

[0175] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, a floppy disk, an optical disk, an optical disk, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), a magnetic card or an optical card, a flash memory, an electrically-erasable programmable read-only memory (EEPROM), or a tangible machine-readable memory for transmitting information (e.g., carrier wave, infrared signal digital signal, etc.) using the Internet in an electrical, optical, acoustic or other form of propagation signal. Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.

[0176] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0177] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0178] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the above elements.

[0179] References to "one embodiment" or "an embodiment" in the specification mean that the specific features, structures, or characteristics described in conjunction with the embodiment are included in at least one exemplary implementation or technology disclosed according to the embodiment of the present application. The appearance of the phrase "in one embodiment" in various places in the specification does not necessarily all refer to the same embodiment.

[0180] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. An image noise reduction method, applied to a first electronic device, characterized in that: The method comprises: Acquire a first image to be denoised; Performing a first noise reduction process on the first image to obtain a second image; Calculating a noise reduction error of the first noise reduction processing based on image optical parameters of the first image; Based on the denoising error, the second image is corrected to obtain a first denoised image of the first image.

2. The method according to claim 1, characterized in that The first noise reduction process includes at least one of the following: Gaussian filtering algorithm, median filtering algorithm and non-local mean filtering algorithm.

3. The method according to claim 1, characterized in that The calculating the noise reduction error of the first noise reduction processing based on the image optical parameters of the first image includes: Determining a weight coefficient and a filtering frequency of a first filtering formula based on the image optical parameters of the first image, wherein the first filtering formula is used to calculate a noise reduction error of a first noise reduction process; Based on the weight coefficient and the filtering frequency of the first filtering formula and in combination with the first filtering formula, a noise reduction error of the first noise reduction processing is calculated.

4. The method according to claim 3, characterized in that The weight coefficient and the filtering frequency based on the first filtering formula are combined with the first filtering formula to calculate the noise reduction error of the first noise reduction processing, including: Performing a first filtering process on the noise reduction image using a filter of a first frequency to obtain a first filtering result; and, Performing a second filtering process on the noise reduction image using a filter of a second frequency to obtain a second filtering result, wherein the first frequency is different from the second frequency; Based on a first weight coefficient corresponding to the filter of the first frequency and a second weight coefficient corresponding to the filter of the second frequency, the first filtering result and the second filtering result are linearly superimposed to obtain a denoising error of the second image.

5. The method according to claim 3 or 4, characterized in that: The first filtering formula is: addnoise=noisegain*(alpha_1*filter_1(noisemap)+alpha_2 *filter_2(noisemap)+…+alpha_n*filter_n(noisemap)) Among them, the addnoise represents the noise reduction error of the second image; the noisemap represents the noise reduction map; the noisegain represents the gain value of the noise reduction error; the alpha_1 to alpha_n represent the weight coefficients corresponding to filters of different frequencies; and the filter_1 to filter_n represent different filtering frequencies of multiple filters.

6. The method according to claim 3 or 4, characterized in that: The weight coefficient and the filtering frequency are determined in the following manner: Taking the image optical parameters of the first image and the second image as input data, and inputting them into a filter coefficient model; Determining the weight coefficient and the filtering frequency based on the output result of the filtering coefficient model; Wherein, the filter coefficient model is obtained based on the training of the convolutional neural network model.

7. The method according to claim 6, characterized in that The method of obtaining the filter coefficient model includes: The first electronic device obtains the filter coefficient model by training based on the training data obtained by processing the acquired training sample set; or The first electronic device deploys the filter coefficient model provided by the second electronic device, wherein the filter coefficient model is trained by the second electronic device based on training data obtained by processing an acquired training sample set.

8. The method according to claim 7, characterized in that The training process of the filter coefficient model includes: Acquire a training sample set, the training sample set comprising a first sample image set, sensitivities corresponding to the first sample image set, and a second sample image set; Based on the second sample images in the second sample image set and the sensitivity corresponding to the first sample image set, training a first model to obtain the filter coefficient model, wherein the first model includes a convolutional neural network model; Obtaining a third weight coefficient and a third frequency of a corresponding filter output by the first model and a third weight coefficient and a third frequency; Using a first filter with a third frequency to perform a third filtering process on a third sample image in a third sample image set to obtain a third filtering result; and, performing a fourth filtering process on a third sample image in a third sample image set using a second filter with a fourth frequency to obtain a third filtering result, wherein the third frequency is different from the fourth frequency; Based on a third weight coefficient corresponding to the filter of the third frequency and a fourth weight coefficient corresponding to the filter of the fourth frequency, linearly superimposing the third filtering result and the fourth filtering result to obtain a denoising error of the second sample image set; The noise data of the third sample image set is calculated based on the difference between the image data of the first sample image set and the image data of the second sample image set.

9. The method according to claim 8, characterized in that The training process of the filter coefficient model also includes: Based on the denoising error of the second sample image set, correcting the second sample images in the second sample image set to obtain a fourth sample image set; Based on a first loss function, determining a first similarity between the fourth sample image set and the first sample image set; wherein the first loss function is used to calculate the difference between the fourth sample image set data and the first sample image set data; And based on the first similarity, the model parameters of the first model are adjusted to obtain the filter coefficient model.

10. The method according to claim 8, characterized in that The training sample set includes a first sample image set, sensitivities corresponding to the first sample image set, and a second sample image set, and, The obtaining of the training sample set comprises: Acquire the first sample image set and a first sensitivity corresponding to the first sample image set; The first sample image set is preprocessed to obtain the second sample image set.

11. The method according to claim 10, characterized in that Preprocessing the first sample image set to obtain the second sample image set includes: Performing degradation processing and noise adding processing on the first sample image set based on a first degradation parameter and a first noise parameter to obtain a first sample image set containing noise; Based on the second denoising process, the first sample image set containing noise is subjected to denoising process to obtain the second sample image set.

12. The method according to claim 1, characterized in that The image optical parameters include the sensitivity, brightness and pixel value of the first image.

13. A model training method, applied to a first electronic device or a second electronic device, characterized in that: The method comprises: Acquire a training sample set, the training sample set comprising a first sample image set, a first sensitivity corresponding to the first sample image set, and a second sample image set, wherein the first sample images in the first sample image set correspond one-to-one to the second sample images in the second sample image set, and the second sample images are obtained by performing degradation processing, noise addition processing, and noise reduction processing on the first sample images; The network model is trained using the second sample images in the second sample image set and the first sensitivity to obtain a filter coefficient model.

14. The method according to claim 13, characterized in that The method further comprises: Based on the third sample image set and the filter coefficient, in combination with the first filter formula, a noise reduction error of the noise reduction process is obtained; wherein the noise data of the third sample image set is calculated based on the difference between the first sample image set data and the second sample image set data; Based on the denoising error, correcting the second sample image set to obtain a fourth sample image set; Based on the first sample image set data, the fourth sample image data and the first loss function, the training parameters of the network model are adjusted to obtain the filter coefficient model.

15. The method according to claim 13, characterized in that The first sample image set is a noise-free image set.

16. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the image denoising method described in any one of claims 1-12, or executes the model training method described in any one of claims 13-15.

17. A computer readable medium, characterized in that The readable medium stores instructions, which, when executed on a computer, cause the computer to execute the image denoising method described in any one of claims 1-12, or the model training method described in any one of claims 13-15.

18. A computer program product, characterized in that It includes a computer program / instruction, which, when executed by a processor, implements the image denoising method described in any one of claims 1 to 12, or implements the model training method described in any one of claims 13 to 15.

Images