Image noise reduction method, electronic device, readable storage medium and program product

Abstract

The present invention relates to the field of image technology, and more particularly to an image denoising method, electronic device, readable storage medium, and program product. This method obtains filter parameters of a filtering formula using optical parameters of the image to be denoised and a filter coefficient model. The filtering formula then determines the denoising error of the image to be denoised after it has been processed by a denoising algorithm. The image processed by the denoising algorithm is then corrected based on the denoising error to obtain a noise-free image of the image to be denoised. This image denoising method can preserve some image details or remove more image noise, thereby improving the denoising effect of the image to be denoised.

Description

Technical Field

[0001] The present invention relates to the field of image technology, and in particular to an image noise reduction method, electronic equipment, a readable storage medium, and a program product. Background Art

[0002] Using electronic devices to capture images has become a common way for people to record their daily lives. However, the image acquisition process can introduce various noises due to factors such as the sensor material properties, components, circuit structure, and operating environment. Furthermore, during image signal transmission, the image can be contaminated by various noises, such as Gaussian and Poisson noise, due to the influence of the transmission medium and recording device.

[0003] Currently, the most common method for processing image noise is to perform noise reduction on the image, such as Gaussian filtering, median filtering, and non-local means filtering. However, due to processing power limitations, current noise reduction algorithms suffer from noise reduction errors, i.e., the noise-reduced image may suffer from a loss of image details and texture features (hereinafter referred to as smearing) or incomplete noise removal. Summary of the Invention

[0004] The embodiments of the present application provide an image denoising method, electronic device, readable storage medium, and program product, which can correct the denoising errors in an initial denoised image to an image processed by a denoising algorithm, thereby obtaining a target denoised image that retains complete details and features.

[0005] In a first aspect, embodiments of the present application provide an image denoising method, electronic device, and readable storage medium. The method includes: acquiring a first image to be denoised; performing a first denoising process on the first image to obtain a second image; calculating a denoising error from the first denoising process based on image optical parameters of the first image; and correcting the second image based on the denoising error to obtain a first denoised image of the first image.

[0006] Exemplarily, the first image to be denoised can be obtained by the first electronic device, for example, by photographing it, or by receiving an image or video sent by other electronic devices by the first electronic device, which is not limited here.

[0007] It can be understood that when the first processing is performed on the first image, a noise reduction error occurs, and a mapping relationship exists between the noise reduction error of the first noise reduction processing and the image optical parameters of the first image. Therefore, the noise reduction error of the first noise reduction processing can be calculated based on the image optical parameters of the first image, such as the brightness, sensitivity, and pixel count of the first image. Based on this noise reduction error, the second image is corrected to obtain a first noise-reduced image of the first image.

[0008] In this way, the above-mentioned image denoising method can retain some details in the image to be denoised or remove more image noise points, which is beneficial to improving the denoising effect of the image to be denoised.

[0009] In a possible implementation of the first aspect above, the first noise reduction process includes at least one of the following: a Gaussian filtering algorithm, a median filtering algorithm, and a non-local mean filtering algorithm.

[0010] In a possible implementation of the first aspect above, the noise reduction error of the first noise reduction processing is calculated based on the image optical parameters of the first image, including: determining the weight coefficient and filtering frequency of the first filtering formula based on the image optical parameters of the first image, wherein the first filtering formula is used to calculate the noise reduction error of the first noise reduction processing; based on the weight coefficient and filtering frequency of the first filtering formula, in combination with the first filtering formula, the noise reduction error of the first noise reduction processing is calculated.

[0011] In a possible implementation of the first aspect above, based on the weight coefficient and filtering frequency of the first filtering formula, in combination with the first filtering formula, the noise reduction error of the first noise reduction processing is calculated, including: using a filter of a first frequency to perform a first filtering process on the noise reduction image to obtain a first filtering result; and using a filter of a second frequency to perform a second filtering process on the noise reduction image to obtain a second filtering result, wherein the first frequency is different from the second frequency; based on the first weight coefficient corresponding to the filter of the first frequency and the second weight coefficient corresponding to the filter of the second frequency, the first filtering result and the second filtering result are linearly superimposed to obtain the noise reduction error of the second image.

[0012] It will be appreciated that the noise map described above includes noise signals in one or more different frequency bands, each corresponding to a different noise reduction error. Therefore, noise signals in one or more different frequency bands can be filtered out by setting one or more filters with different frequencies to obtain the noise reduction errors for the corresponding frequency bands. Finally, the noise reduction errors for these different frequency bands are linearly superimposed to obtain the noise reduction error for the image to be processed. In this way, based on the aforementioned segmented filtering approach, noise reduction errors for different frequency bands can be obtained from the noise map, and the noise reduction errors for each frequency band can be linearly superimposed using different weighting coefficients.

[0013] In a possible implementation of the first aspect, the first filtering formula is:

[0014] addnoise=noisegain*(alpha_1*filter_1(noisemap)+alpha_2

[0015] *filter_2(noisemap)+…+alpha_n*filter_n(noisemap))

[0016] Among them, addnoise represents the noise reduction error of the second image; noisemap represents the noise reduction map; noisegain represents the gain value of the noise reduction error; alpha_1 to alpha_n represent the weight coefficients corresponding to filters of different frequencies; filter_1 to filter_n represent different filtering frequencies of multiple filters.

[0017] In a possible implementation of the first aspect above, the method for determining the weight coefficient and the filtering frequency includes: taking the image optical parameters of the first image and the second image as input data, and inputting them into a filtering coefficient model; determining the weight coefficient and the filtering frequency based on the output result of the filtering coefficient model; wherein the filtering coefficient model is obtained based on training of a convolutional neural network model.

[0018] It's understandable that both the weight coefficient and the frequency coefficient are related to the image's optical parameters, such as pixels and sensitivity. However, the relationship between the weight coefficient, the frequency coefficient, and the image's pixels and sensitivity is complex, making it difficult to accurately model this relationship using simple formulas. Therefore, a filter coefficient model can be used to fit the relationship between the weight coefficient, the frequency coefficient, and the image's pixels and sensitivity.

[0019] It can be understood that the filter coefficient model obtained by training the convolutional neural network model has higher accuracy in outputting weight coefficients and filter frequencies and faster output efficiency.

[0020] In a possible implementation of the first aspect above, the method for obtaining the filter coefficient model includes: the first electronic device trains the filter coefficient model based on training data obtained by processing the acquired training sample set; or the first electronic device deploys the filter coefficient model provided by the second electronic device, wherein the filter coefficient model is trained by the second electronic device based on training data obtained by processing the acquired training sample set.

[0021] It can be understood that the above-mentioned filter coefficient model can be obtained by training the first electronic device based on the training data obtained by processing the acquired training sample set, or it can be a filter coefficient model provided by the first electronic device and deployed by the second electronic device, which is not limited here.

[0022] In a possible implementation of the first aspect above, a training process of a filter coefficient model includes: obtaining a training sample set, the training sample set including a first sample image set, a sensitivity corresponding to the first sample image set, and a second sample image set; training a first model based on the second sample image in the second sample image set and the sensitivity corresponding to the first sample image set to obtain a filter coefficient model, wherein the first model includes a convolutional neural network model; obtaining a third weight coefficient and a third frequency and a third weight coefficient and a third frequency of a filter corresponding to an output of the first model; performing a third filtering process on the third sample image in the third sample image set using the first filter of the third frequency to obtain a third filtering result; and performing a fourth filtering process on the third sample image in the third sample image set using the second filter of the fourth frequency to obtain a third filtering result, wherein the third frequency is different from the fourth frequency; linearly superimposing the third filtering result and the fourth filtering result based on the third weight coefficient corresponding to the filter of the third frequency and the fourth weight coefficient corresponding to the filter of the fourth frequency to obtain a noise reduction error of the second sample image set; wherein the noise data of the third sample image set is calculated based on the difference between the data of the first sample image set and the data of the second sample image set.

[0023] It can be understood that the second sample images in the second sample image set and the sensitivities corresponding to the first sample image set are used as training data to train the first model and obtain the filter coefficient model.

[0024] In a possible implementation of the first aspect above, the training process of the filter coefficient model also includes: based on the denoising error of the second sample image set, correcting the second sample image in the second sample image set to obtain a fourth sample image set; based on the first loss function, determining a first similarity between the fourth sample image set and the first sample image set; wherein the first loss function is used to calculate the difference between the fourth sample image set data and the first sample image set data; and based on the first similarity, adjusting the model parameters of the first model to obtain the filter coefficient model.

[0025] It can be understood that the second sample image after noise reduction processing is corrected according to the noise reduction error to obtain the fourth sample image set of the first sample image set. Then, the difference between the fourth sample image set data and the first sample image set data is calculated by the first loss function, and the training parameters of the first model are adjusted in time to obtain the filter coefficient model

[0026] In a possible implementation of the first aspect above, the training sample set includes a first sample image set, a sensitivity corresponding to the first sample image set, and a second sample image set, and obtaining the training sample set includes: obtaining the first sample image set and the first sensitivity corresponding to the first sample image set; and preprocessing the first sample image set to obtain the second sample image set.

[0027] It is understood that the first sample image set is a noise-free image set, and therefore it is necessary to preprocess the first sample image set to obtain a second sample image set. The second sample image set is the first sample image set that has been preprocessed to include noise data. The noise data includes a noise signal to reduce noise errors.

[0028] In a possible implementation of the first aspect, preprocessing the first sample image set to obtain the second sample image set includes: performing degradation processing and noise addition processing on the first sample image set based on a first degradation parameter and a first noise parameter to obtain a first sample image set containing noise; and then performing noise reduction processing on the first sample image set containing noise based on a second noise reduction processing to obtain the second sample image set.

[0029] It can be understood that the above-mentioned preprocessing process is intended to simulate the generation of degraded data and noise data during the actual transmission process of the image. Specifically, the first sample image set can be degraded based on preset degradation parameters, such as through a convolution operation or a blur filter of an application in an electronic device. Then, based on preset noise parameters, noise data, such as Gaussian noise, Poisson noise, etc., is added to the first sample image set after degradation. After the above-mentioned degradation and noise addition processes, a first sample image set with noise is obtained. The first sample image set with noise is then subjected to noise reduction processing (i.e., a second noise reduction process) to obtain a second sample image set and a third sample image set. The noise data of the third sample image set is calculated based on the difference between the first sample image set data and the second sample image set image data.

[0030] The second noise reduction process includes at least one of a Gaussian filtering algorithm, a median filtering algorithm and a non-local mean filtering algorithm.

[0031] In a possible implementation of the first aspect, the image optical parameters include sensitivity, brightness, and pixel values ​​of the first image.

[0032] In a second aspect, an embodiment of the present application provides a model training method, which is applied to a first electronic device or a second electronic device, the method comprising: obtaining a training sample set, the training sample set comprising a first sample image set, a first sensitivity corresponding to the first sample image set, and a second sample image set, wherein the first sample image in the first sample image set corresponds one-to-one to the second sample image in the second sample image set, and the second sample image is obtained based on degradation processing, noise addition processing, and noise reduction processing of the first sample image; using the second sample image and the first sensitivity in the second sample image set, the network model is trained to obtain a filter coefficient model.

[0033] It can be understood that the filter coefficient model obtained by training the convolutional neural network model has higher accuracy in outputting weight coefficients and filter frequencies and faster output efficiency.

[0034] In a possible implementation of the second aspect above, based on the third sample image set and the filter coefficient, combined with the first filtering formula, a denoising error of the denoising processing is obtained; wherein, the noise data of the third sample image set is calculated based on the difference between the first sample image set data and the second sample image set data; based on the denoising error, the second sample image set is corrected to obtain a fourth sample image set; based on the first sample image set data, the fourth sample image data and the first loss function, the training parameters of the network model are adjusted to obtain a filter coefficient model.

[0035] It can be understood that the above-mentioned calculation of the difference between the fourth sample image set data and the first sample image set data through the first loss function and timely adjustment of the training parameters of the network model make the weight coefficient and filter coefficient output by the filter coefficient model more accurate, and therefore the noise reduction error obtained based on the weight coefficient, filter coefficient and the first filtering formula is more accurate.

[0036] In a possible implementation of the second aspect, the first sample image set is a noise-free image set.

[0037] In a third aspect, an embodiment of the present application provides an electronic device comprising: one or more processors; one or more memories; one or more memories storing one or more programs, wherein when one or more programs are executed by one or more processors, the electronic device executes the image denoising method provided in the first aspect and various possible implementations of the first aspect, or the model training method provided in the second aspect and various possible implementations of the second aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the computer executes the image denoising method provided in the first aspect and various possible implementations of the first aspect, or the model training method provided in the second aspect and various possible implementations of the second aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a program product, and the computer program product includes: computer program code, which, when the computer program code runs on a computer, enables the computer to implement the image denoising method as described in the first aspect and any one of the various implementations of the first aspect, or to implement the model training method as described in the second aspect and any one of the various implementations of the second aspect.

[0040] The beneficial effects of the third to fifth aspects mentioned above can be found in the relevant descriptions of various possible implementations of the first and second aspects mentioned above, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 FIG. 1 shows a schematic diagram of an image processing process of ISP 000;

[0042] Figure 2a According to the present application, a schematic diagram of an image without noise reduction processing is shown;

[0043] Figure 2b According to the present application, a schematic diagram of an image after noise reduction processing is shown;

[0044] Figure 2c According to the present application, another schematic diagram of an image after noise reduction processing is shown;

[0045] Figure 3 According to the present application, a schematic diagram of an implementation process of an image noise reduction method is shown;

[0046] Figure 4 According to an embodiment of the present application, a specific implementation flow chart of image noise reduction is shown;

[0047] Figure 5 According to an embodiment of the present application, a schematic diagram of a scenario for obtaining an image to be processed is shown;

[0048] Figure 6 According to an embodiment of the present application, a training process diagram of a filter coefficient model training method is shown;

[0049] Figure 7 A hardware structure diagram of an electronic device 100 is shown according to an embodiment of the present application;

[0050] Figure 8 According to an embodiment of the present application, a software system architecture diagram of an electronic device 100 is shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0052] It should also be stated that the steps in the methods and processes in the embodiments of the present application are numbered for ease of reference, rather than to limit the order of precedence. If there is a sequence between the steps, the written description shall prevail.

[0053] The following combination Figure 1The specific implementation process of image processing by the image signal processor (ISP) is described.

[0054] Figure 1 Figure 1 is a schematic diagram of the image processing process in ISP 000. Figure 1 As shown, the original image data entering ISP 000 for processing passes through the following modules in ISP 000 in sequence: image pre-processing module 01, image enhancement processing module 02, color correction module 03, and image compression processing module 04.

[0055] Among them, the image preprocessing module 01 preprocesses the collected raw image data to improve image quality. This mainly includes operations such as denoising, color correction, white balancing, and color enhancement on the raw image, as well as geometric transformations such as cropping, rotation, and scaling. The image noise reduction process mentioned in the embodiments of this application is performed in the ISP000 image preprocessing module 01. After the raw image is preprocessed, the image enhancement processing module 02 can also enhance the image to improve the details, contrast, and saturation of the raw image. This mainly includes operations such as sharpening, edge enhancement, and local contrast enhancement on the raw image. Furthermore, the color correction module 03 can convert the raw image from one color space to another, for example, from RGB color space to YUV color space, or from YUV color space to RGB color space. The image compression processing module 04 can compress the raw image to reduce the size of the raw image data for easier storage and transmission. Common compression algorithms include JPEG and H.264.

[0056] Figure 2a According to the present application, a schematic diagram of an image without noise reduction processing is shown. Figure 2a As shown, image 101 shows the texture features of leaves of green plants (e.g. Figure 2a A” in the image and multiple noise points A. After noise reduction processing by a noise reduction algorithm, such as a Gaussian filter algorithm, the noise may not be completely removed due to the insufficient noise reduction capability of the noise reduction algorithm. Figure 2b There are still many noise points A' in the image 102 shown in FIG. Figure 2c Image 103 shows excessive noise reduction, resulting in Figure 2b The texture characteristics of green leaves are lost (i.e. Figure 2c A'' in , can also be called image smearing.

[0057] As mentioned above, when the current denoising algorithm performs denoising on the denoised image, problems such as incomplete noise removal or excessive denoising may occur, resulting in smearing of the image.

[0058] To address the aforementioned issues, it is necessary to first analyze what parts of the content are removed by the denoising algorithm when performing denoising on the image to be denoised. Generally, the image to be denoised, acquired by an electronic device, contains Poisson noise and Gaussian noise. When denoising an image to be denoised, two types of noise reduction errors generally occur. One is insufficient noise reduction due to insufficient denoising capabilities, meaning that the noise is not removed cleanly. The other is excessive noise reduction, which results in a loss of detail and texture features in the image to be denoised, meaning that image smearing occurs. Therefore, when the denoising algorithm performs denoising on the image to be denoised, the initial noise map obtained may include either an incomplete noise signal or a complete noise signal-image signal. Therefore, the signal content in the initial noise map can be summarized as a noise signal and a noise reduction error signal.

[0059] The initial denoised image is obtained by spectral subtraction, that is, the image to be denoised is subtracted from the image processed by the denoising algorithm to obtain the initial denoised image.

[0060] It should be understood that Poisson noise and Gaussian noise are affected by brightness (LUMA) and sensitivity (ISO). The relationship between the noise signal and brightness and sensitivity can be expressed by formula (1):

[0061] Noise signal = f(ISO, LUMA, ERR) (1)

[0062] Among them, ISO represents the sensitivity of the image; LUMA represents the brightness of the image; ERR represents the noise reduction error; and f represents the mapping relationship between sensitivity, brightness, noise reduction error and noise signal.

[0063] It will be appreciated that the initial noise image includes noise signals in one or more different frequency bands, each of which corresponds to a different noise reduction error. Therefore, one or more filters with different frequencies can be set to filter out the noise signals in one or more different frequency bands, thereby obtaining the noise reduction errors for the corresponding frequency bands. Finally, the noise reduction errors for these different frequency bands are linearly superimposed to obtain the noise reduction error (ERR) of the processed image.

[0064] Therefore, the noise reduction error can be obtained by the first filtering formula (2):

[0065] addnoise=noisegain*(alpha_1*filter_1(noisemap)+alpha_2*

[0066] filter_2(noisemap)+alpha_3*filter_3(noisemap)+…+alpha_n*

[0067] filter_n(noisemap))(2)

[0068] Among them, addnoise represents the denoising error of the second image; noisemap represents the initial denoising map; noisegain represents the gain value of the denoising error; alpha_1 to alpha_n represent the weight coefficients corresponding to filters of different frequencies; filter1 to filter_n represent the different filtering frequencies of multiple filters.

[0069] It can be understood that the gain value of the noise reduction error can be dynamically adjusted according to the user's requirements for image processing.

[0070] Based on the above analysis, the present application provides an image denoising method. In this method, a neural network model is trained to obtain the weight coefficients and filter frequencies based on the image, as well as the image's brightness and sensitivity, to obtain corresponding weight coefficients and filter frequencies, such as the weight coefficients and filter frequencies in the first filtering formula (2). Then, based on the first filtering formula, a denoising error of the image to be processed is obtained, and finally, the denoising error is corrected to the image to obtain a target denoised image.

[0071] It should be understood that the image denoising method provided in this application is to correct the denoising error in the initial denoising image to the image processed by the denoising algorithm, so as to obtain a target denoised image that retains complete details and features.

[0072] In some optional embodiments, the filter coefficient model may be a neural network model obtained through training with a large amount of data. The specific training process may be as follows: a sample image set (i.e., a first sample image set, denoted as GTs) is collected and subjected to degradation and noise processing. Specifically, the sample image set is first subjected to degradation processing, such as through a convolution operation or the application of a blur filter. Noise, such as Gaussian noise or Poisson noise, is then added to the degraded sample image set. After the degradation and noise processing, a data set is obtained (i.e., the first sample image set with noise, denoted as noise_Imgs). The simulated degradation and noise-added data set is then subjected to noise reduction processing to obtain a training data set (train_datas) (i.e., a second sample image set, denoted as clear_Imgs) and a noise set (i.e., a third sample image set, denoted as noise_maps).

[0073] It is understandable that the above-mentioned data set can be transformed by YUV processing to obtain the Y channel signal before the noise reduction process is performed to determine the brightness of the data set. For example, the image in the data set is converted into a color space by the aforementioned color correction module 03. However, the brightness of each pixel position in the image is different, so the brightness is replaced by the image pixel in the embodiment of the present application. In some embodiments, the RGB signal of the image can also be used directly to obtain the pixels of the image, which is not limited here. Among them, RGB (red, green, blue) color space or RGB domain refers to a color model related to the structure of the human visual system. According to the structure of the human eye, all colors are regarded as different combinations of red, green and blue.

[0074] It can be understood that the training set [(train_datas, noise_maps), GTs] required for training is constructed through the above process. Figure 3 As shown, the training data set and the corresponding sensitivity are input into the training network (i.e. Figure 3 In the network), the filter coefficient of the filter formula is obtained through the network output. For example, taking Gaussian filtering as an example, the network outputs the gamma coefficient (gamma) and weight coefficient of the Gaussian filter. Among them, the above gamma coefficient is used to represent the filter frequency of the Gaussian filter. Then based on the preset filter formula (i.e. Figure 3 ), perform segmented filtering on the noise set (ie, the third sample image set, denoted as noise_maps), and determine the noise reduction error of the noise reduction process.

[0075] Furthermore, the images in the training data are corrected based on the noise reduction error to obtain a set of noise-superimposed images (i.e., the fourth sample image set, denoted as en_Imgs). The loss function (LOSS) is then used to calculate the similarity between this set of images and the images in the first sample image set, continuously optimizing the training model direction and correcting the filter coefficient model.

[0076] It can be understood that the embodiments of the present application are applicable to electronic devices, which may be mobile phones, smart screens, tablet computers, wearable electronic devices, in-vehicle electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), projectors, and the like.

[0077] I understand. Figure 4 According to an embodiment of the present application, a specific implementation flow chart of image noise reduction is shown.

[0078] I understand. Figure 4 The execution entities of each step of the process shown can all be electronic devices such as mobile phones, tablet computers or laptop computers. The execution entities of each step will not be repeatedly introduced when introducing the specific execution content of each step below.

[0079] like Figure 4 As shown, the process includes the following steps:

[0080] 401: Obtain a first image to be denoised and the corresponding sensitivity.

[0081] For example, taking mobile phone 200 as an example, refer to Figure 5 As shown in (A) in FIG, the user clicks the camera application icon 101 of the mobile phone 200 by performing operation (1). Correspondingly, the mobile phone 200 obtains the camera shooting interface in response to the user operation. Figure 5 As shown in (B), the user selects a shooting mode based on shooting needs, performs operation 2, and clicks the shooting control 102. The mobile phone 200 obtains and stores the image in response to the click operation of operation 2. Figure 5 The shooting mode in (B) can be a photo mode, a video mode, etc. It should be understood that the above Figure 5 The image acquisition scenario shown is for illustration only and does not impose any restrictions on the application scenarios of this application. The image noise reduction method provided in the embodiment of this application can be applied to, but not limited to, the following scenarios: video calls, video conferencing applications, long and short video applications, video live broadcast applications, video online course applications, smart mirror application scenarios, system camera recording function video, video surveillance, and smart cat-eye shooting scenarios, etc., without limitation here. After performing operation 2, the mobile phone obtains the image to be denoised, and at the same time, the corresponding sensitivity can be obtained through the shooting parameters during the shooting process.

[0082] 402: Perform noise reduction processing on the first image to be denoised to obtain a second image.

[0083] For example, the noise reduction processing of the image to be denoised can be implemented using any one of a Gaussian filter algorithm, a median filter algorithm, and a non-local means (NLM) filter algorithm. The Gaussian filter algorithm is a linear smoothing filter algorithm that performs a weighted average on the entire image. The value of each pixel is a weighted average of its own value and the values ​​of other pixels in its neighborhood.

[0084] The median filter algorithm is a nonlinear smoothing filter algorithm. The median filter can replace the value of a pixel with the median value of each pixel in a neighborhood of the pixel.

[0085] The NLM algorithm sets two fixed-size windows: a large search window (D×D) and a small neighborhood window (d×d). The neighborhood window slides within the search window and determines the influence of the corresponding central pixel on the current pixel based on the similarity between the neighborhoods. In some embodiments, the denoising process for the image to be denoised can also be performed using other filtering algorithms or a combination of multiple filtering algorithms, which are not limited here.

[0086] It can be understood that the image to be denoised is processed by any one or more of the above algorithms to obtain the second image.

[0087] 403: Calculate a noise reduction error of the first noise reduction process based on the image optical parameters of the first image to be denoised.

[0088] Exemplarily, the image optical parameters of the image to be denoised may be, for example, brightness, sensitivity, and pixels of the image to be denoised.

[0089] It is understood that the initial denoised image of the first image needs to be obtained before executing step 403. The initial denoised image is obtained by spectral subtraction, that is, by subtracting the second image from the first image.

[0090] In some optional embodiments, in order to facilitate calculation, the weight coefficients in the above formula (2) can be set to a map (i.e., a weight map, denoted as alpha_map), and the corresponding size can be the same as the noise image. The specific process can be, for example: create an empty map object and name the empty map alpha_map. Then obtain the size of the initial noise map, for example, the size function of the initial noise map can be used to obtain the number of rows and columns. Then use the adjustment function of alpha_map to set the size of the weight map to be the same as the initial noise image. The adjustment function accepts the two data of the number of rows and columns, uses a two-layer loop to traverse each element of the initial noise map, and assigns its value to the element at the corresponding position of the weight map. Among them, the adjustment function can change the size of the weight map. In other embodiments, some other reasonable methods can also be used to make the corresponding size of the weight map the same as the initial noise image, which is not limited here. In this way, the above formula (2) can also be expressed as:

[0091] addnoise=noisegain*(alpha_map1*filter_1(noisemap)+alpha_map2

[0092] *filter_2(noisemap)+alpha_map3*filter_3(noisemap)+…

[0093] +alpha_n*filter_n(noisemap))

[0094] The meanings of the terms in the formula have been explained in detail above and will not be repeated here.

[0095] It can be understood that the corresponding relationship between the weight map and the brightness and sensitivity of the image can be expressed by formula (3):

[0096] alpha_map = f(LUMA, ISO) (3)

[0097] Among them, LUMA represents the brightness of the image, ISO represents the sensitivity of the image, and f represents the mapping relationship between sensitivity, brightness and weight map.

[0098] As mentioned above, the brightness of each pixel position in the image is different, and the image pixels can be used to replace the brightness. Therefore, the corresponding relationship between the weight map and the brightness and sensitivity of the image can also be expressed by formula (3):

[0099] alpha-map = f(PX, ISO) (4)

[0100] Among them, PX represents the constituent pixels of the image, ISO is the sensitivity of the image, and f represents the mapping relationship between sensitivity, pixels and weight map.

[0101] It is understandable that the selection of filtering frequency for different filtering methods is affected by the image information (texture, color, etc.) and noise. Taking Gaussian filtering as an example, Gaussian filtering can select a Gaussian kernel of a fixed size, such as kernel_size = 5. The gamma coefficient corresponding to the Gaussian filter is related to the pixels and sensitivity of the image. Therefore, the corresponding relationship between the gamma coefficient and the pixels and sensitivity of the image can be expressed by formula (5):

[0102] gamma = g(PX, ISO) (5)

[0103] Among them, g represents the mapping relationship between sensitivity, pixel and gamma coefficient, gamma is the gamma coefficient, which can represent the filtering frequency and filtering effect of Gaussian filtering, and ISO is the sensitivity of the image.

[0104] As mentioned above, it can be known from formula (4) and formula (5) that the weight coefficient and the gamma coefficient are both related to the pixels and sensitivity of the image. However, the relationship between the weight coefficient and the gamma coefficient, and the pixels and sensitivity of the image is relatively complex, and it is difficult to obtain the correct relationship through simple formula modeling. Therefore, in the embodiment of the present application, the relationship between the weight coefficient, the gamma coefficient and the pixels and sensitivity of the image can be fitted by fitting the filter coefficient model. Finally, the noise reduction error is calculated by the above formula (2). Among them, the above filter coefficient model can be obtained by, for example, training a convolutional neural network model. The specific training process is as follows. Figure 6 The relevant description will not be repeated here. It should be understood that the filter coefficient model obtained by training the convolutional neural network model has higher accuracy in outputting weight coefficients and filter frequencies and faster output efficiency.

[0105] 404: Correct the second image based on the noise reduction error to obtain a first noise-reduced image of the first image.

[0106] It will be appreciated that the above-described correction processing of the second image based on the noise reduction error yields a first noise-reduced image of the first image. For example, the second image can be subjected to linear processing based on the noise reduction error. In this manner, the image obtained using the above-described image noise reduction method retains complete texture features and exhibits strong noise reduction capabilities.

[0107] It can be understood that the above-mentioned filter coefficient model can be synchronously added to the algorithm library or model library that can be called by the ISP together with other image processing algorithms.

[0108] In some optional embodiments, Figure 6 According to an embodiment of the present application, a training process diagram of a filter coefficient model training method is shown.

[0109] I understand. Figure 6 The execution entities of each step of the process shown can all be electronic devices such as mobile phones, tablet computers or laptop computers. The execution entities of each step will not be repeatedly introduced when introducing the specific execution content of each step below.

[0110] like Figure 6 As shown, the training process may include the following steps:

[0111] 601: Obtain a first sample image set and the sensitivity corresponding to the first sample image set.

[0112] It can be understood that the first sample image set may be a noise-free image set, and the sensitivity corresponding to the first sample image set may be a shooting parameter of the electronic device when shooting the first sample image set.

[0113] 602: Preprocess the first image sample set to obtain a second sample image set and a third sample image set.

[0114] In some embodiments, the preprocessing of the first image sample set includes degradation processing, noise addition processing, and noise reduction processing. The specific preprocessing process may be based on, for example, performing degradation processing on the first sample image set based on preset degradation parameters, such as through a convolution operation or a blur filter applied by an electronic device application. Then, based on preset noise parameters, noise data, such as Gaussian noise or Poisson noise, is added to the degraded first sample image set. The degradation and noise addition processes result in a noisy first sample image set.

[0115] Furthermore, the first sample image set with noise is subjected to noise reduction processing to obtain a second sample image set and a third sample image set, wherein the noise data of the third sample image set is calculated based on the difference between the image data of the first sample image set and the image data of the second sample image set.

[0116] The above-mentioned denoising method for denoising the first sample image set with noise can be, for example, any one or more implementations of the Gaussian filtering algorithm, median filtering algorithm and non-local means (NLM) filtering algorithm mentioned in the aforementioned step 402, which will not be described in detail here.

[0117] 603: Based on the second sample image set and the sensitivity of the corresponding first sample image set, the network model is trained to obtain a filter coefficient model.

[0118] It can be understood that the steps 601 to 602 are executed to determine the training set required for training the network model. The second sample image set and the sensitivity of the corresponding first sample image set. Figure 3 , input the second sample image and the sensitivity of the corresponding first sample image set into the training network (i.e. Figure 3 The network model is trained to obtain a filter coefficient model. The filter coefficient model outputs the filter coefficients of the filter formula. For example, in the case of Gaussian filtering, the network outputs the gamma coefficient and weight coefficient of the Gaussian filter. The gamma coefficient is used to represent the filter frequency of the Gaussian filter.

[0119] 604: Obtain a noise reduction error of the noise reduction process based on the third sample image set and the filter coefficients in combination with the first filter formula.

[0120] The first filtering formula may be, for example, the aforementioned formula (2). The specific calculation process may refer to the relevant description of the aforementioned formula (2), which will not be elaborated here.

[0121] It can be understood that the first filtering formula (ie Figure 3), perform segmented filtering on the third sample image set, and use the weight coefficient of each filter to linearly superimpose the multiple filtering results to obtain the noise reduction error of the noise reduction process.

[0122] 605: Based on the noise reduction error, correct the second sample image set to obtain a fourth sample image set.

[0123] It can be understood that the fourth sample image set is obtained by performing correction processing on the second sample image set based on the noise reduction error. For example, the second sample image set can be linearly processed based on the noise reduction error.

[0124] 606: Based on the first sample image set and the fourth sample image set, and in combination with the first loss function, modify the filter coefficient model.

[0125] It can be understood that the above-mentioned correction processing of the second sample image set based on the noise reduction error produces a fourth sample image set with noise superimposed thereon. The difference between the fourth sample image set data and the first sample image set data is then calculated using the loss function (LOSS). Based on this difference, the similarity between the fourth sample image set and the first sample image set is obtained. The model training parameters are then adjusted, and the model training direction is continuously optimized, ultimately resulting in the above-mentioned filter coefficient model.

[0126] As mentioned above, the weight coefficient and filter frequency in formula (2) are related to the pixels and sensitivity of the image. However, the relationship between the weight coefficient, filter coefficient and the pixels and sensitivity of the image is relatively complex, and it is difficult to obtain the correct relationship through simple formula modeling. However, the above filter coefficient model can be used to obtain the weight coefficient and filter frequency based on the pixels and sensitivity of the image, and finally the noise reduction error obtained by formula (2). In addition, the above filter coefficient model calculates the similarity between the fourth sample image set data and the first sample image set data through the loss function (LOSS) during the training process, adjusts the training parameters of the model, and continuously optimizes the training model direction. In this way, the above filter coefficient model can quickly and accurately obtain the weight coefficient and filter frequency based on the pixels and sensitivity of the image. And according to the image corrected by the noise reduction error, some details in the image can be retained or more image noise points can be removed, which is conducive to improving the noise reduction effect of the image.

[0127] Figure 7 According to an embodiment of the present application, a hardware structure diagram of an electronic device 100 is shown. In the embodiment of the present application, the electronic device 100 can be the mobile phone 200 mentioned above, or can be other electronic devices, such as a tablet computer.

[0128] like Figure 7As shown, the electronic device 100 may include a processor 110 (or host), a power module 140, a memory 180, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, a camera 170, an interface module 160, a button 101, a display screen 102 and an image signal processor 000, etc.

[0129] It is understood that the structures illustrated in the embodiments of the present invention do not constitute the only possible implementation of the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown, or combine or separate certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0130] The processor 110 may include one or more processing units, for example, a processing module or processing circuit including a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor, a microprocessor (MCU), an artificial intelligence (AI) processor, or a programmable logic device (FPGA). Different processing units may be independent devices or integrated into one or more processors. A storage unit may also be provided in the processor 110 for storing instructions and data. In some embodiments, the storage unit in the processor 110 is a cache memory 180, which may store instructions or data that the processor 110 has just used or is recycling. If the processor 110 needs to use the instruction or data again, it may be directly called from the storage unit, thereby avoiding repeated accesses, reducing the waiting time of the processor 110, and thus improving the efficiency of the system.

[0131] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0132] The power module 140 may include a power supply, a power management component, and the like. The power supply may be a battery. The power management component manages the charging of the power supply and the supply of power to other modules. In some embodiments, the power management component includes a charging management module and a power management module. The charging management module receives charging input from a charger; the power management module connects to a power supply and is connected to the processor 110. The power management module receives input from the power supply and / or the charging management module and provides power to the processor 110, the display 102, the camera 170, and the wireless communication module 120.

[0133] The mobile communication module 130 may include but is not limited to an antenna, a power amplifier, a filter, a low noise amplifier (LAN), etc. The mobile communication module 130 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to terminal devices. The mobile communication module 130 can receive electromagnetic waves through the antenna, filter, amplify and other processing on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 130 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, at least some of the functional modules of the mobile communication module 130 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 130 can be set in the same device as at least some of the modules of the processor 110.

[0134] The wireless communication module 120 may include an antenna and realize the transmission and reception of electromagnetic waves via the antenna. The wireless communication module 120 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc. applied to the terminal device. The terminal device can communicate with the network and other devices through wireless communication technology.

[0135] In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the terminal device may also be located in the same module.

[0136] Display screen 102 is used to display human-computer interaction interfaces, images, videos, etc. Display screen 102 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flexible light emitting diode (FLED), a quantum dot light emitting diode (QLED), etc.

[0137] The sensor module 190 may include a proximity light sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, an electromagnetic wave absorption rate sensor, a sound sensor, an image sensor, and the like. The image and audio module 150 is configured to convert a digital audio signal into an analog audio signal output, or to convert an analog audio input into a digital audio signal. The audio module 150 may also be configured to encode and decode audio signals. In some embodiments, the audio module 150 may be provided in the processor 110, or some functional modules of the audio module 150 may be provided in the processor 110. In some embodiments, the audio module 150 may include a speaker, an earpiece, a microphone, and a headphone jack.

[0138] Camera 170 is used to capture still images or videos. The lens generates an optical image of an object and projects it onto a photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then passed to an image signal processor (ISP) for conversion into a digital image signal. The terminal device can implement its camera function through the ISP, camera 170, video codec, graphics processing unit (GPU), display 102, and application processor.

[0139] The image signal processor 000 may include: Figure 1 The image preprocessing module, image enhancement processing module, color correction module, and image compression processing module shown in FIG. The image signal processor 000 is used to process the digital image signal collected by the image sensor. In some embodiments, the image signal processor 000 can be used to perform the image noise reduction method provided in the embodiments of the present application.

[0140] Interface module 160 includes an external memory interface, a universal serial bus (USB) interface, and other components. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with processor 110 via the external memory interface to implement data storage. The USB interface is used for communication between the terminal device and other electronic devices.

[0141] In some embodiments, the terminal device further includes a button 101, a motor, and an indicator. The button 101 may include a volume button, an on / off button, and the like. The motor is used to vibrate the terminal device, and different application scenarios (e.g., time reminders, receiving messages, alarm clocks, games, etc.) may correspond to different vibration feedback effects. The indicator may include a laser indicator, a radio frequency indicator, a light-emitting diode (LED) indicator, and the like.

[0142] The above describes the hardware structure of the electronic device in detail. The following is a system structure block diagram of the electronic device.

[0143] It is understood that the operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the system software structure of the electronic device 100.

[0144] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, Android TM The system is divided into four layers, from top to bottom: application layer, application framework layer, Android TM Runtime (Android TM runtime) and system libraries, as well as the kernel layer.

[0145] like Figure 8 As shown, the application layer may include a series of application packages, including camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0146] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0147] The application framework layer may include a window manager, content provider, view system, telephony manager, resource manager, notification manager, etc.

[0148] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0149] Content providers are used to store and retrieve data and make it accessible to applications. This data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0150] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0151] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0152] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0153] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0154] Android TM Runtime includes core libraries and virtual machines. Android TM runtime is responsible for Android TM System scheduling and management.

[0155] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0156] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0157] The system library can include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as openGLES), 2D graphics engine (such as SGL), etc.

[0158] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0159] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0160] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0161] A 2D graphics engine is a drawing engine for 2D drawings.

[0162] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0163] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0164] When the touch sensor in the sensor module 190 receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a still image or video through the camera 170.

[0165] The embodiments of the present application also provide a computer program product for implementing the image noise reduction methods provided in the above embodiments.

[0166] The present application provides a readable storage medium, wherein the readable storage medium stores instructions, which, when executed on an electronic device, enable the electronic device to perform the image noise reduction method mentioned in the present application.

[0167] The various embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0168] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor, a microcontroller, an application specific integrated circuit, or a microprocessor.

[0169] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0170] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as computer program modules or module codes executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0171] A computer program module or module code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0172] Module code can be implemented with high-level modular language or object-oriented programming language to communicate with the processing system. When necessary, module code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0173] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to floppy disks, optical disks, optical discs, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Accordingly, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0174] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technique disclosed according to the embodiment of the present application. The appearances of the phrase "in one embodiment" in various places in the specification do not necessarily all refer to the same embodiment.

[0175] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to a floppy disk, an optical disk, an optical disc, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), a magnetic or optical card, a flash memory, an electrically erasable programmable read-only memory (EEPROM), or a tangible machine-readable memory for transmitting information (e.g., a carrier wave, an infrared signal, a digital signal, etc.) using the Internet in an electrical, optical, acoustic, or other form of propagation signal. Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0176] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0177] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0178] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the presence of other identical elements in the process, method, article or device that includes the above elements.

[0179] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technique disclosed according to the embodiment of the present application. The appearances of the phrase "in one embodiment" in various places in the specification do not necessarily all refer to the same embodiment.

[0180] While the present application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. An image noise reduction method, applied to a first electronic device, characterized in that: The method comprises: Acquire a first image to be denoised; performing a first noise reduction process on the first image to obtain a second image; Calculating a noise reduction error of the first noise reduction processing based on image optical parameters of the first image; Correcting the second image based on the noise reduction error to obtain a first noise-reduced image of the first image; The calculating the noise reduction error of the first noise reduction processing based on the image optical parameters of the first image includes: determining a weight coefficient and a filtering frequency of a first filtering formula based on image optical parameters of the first image, wherein the first filtering formula is used to calculate a noise reduction error of a first noise reduction process; Calculating a noise reduction error of the first noise reduction process based on a weight coefficient and a filtering frequency of the first filtering formula in combination with the first filtering formula; The weight coefficient and the filtering frequency are determined in the following manner: Taking the image optical parameters of the first image and the second image as input data, and inputting them into a filter coefficient model; Determining the weight coefficient and the filtering frequency based on an output result of the filtering coefficient model; The filter coefficient model is obtained based on the training of the convolutional neural network model.

2. The method according to claim 1, characterized in that The first noise reduction process includes at least one of the following: Gaussian filtering algorithm, median filtering algorithm and non-local mean filtering algorithm.

3. The method according to claim 1, characterized in that The calculating of the noise reduction error of the first noise reduction process based on the weight coefficient and the filtering frequency of the first filtering formula in combination with the first filtering formula includes: performing a first filtering process on the denoised image using a filter of a first frequency to obtain a first filtering result, wherein the denoised image is obtained by subtracting the second image from the first image; and Performing a second filtering process on the noise reduction image using a filter of a second frequency to obtain a second filtering result, wherein the first frequency is different from the second frequency; Based on a first weight coefficient corresponding to the filter of the first frequency and a second weight coefficient corresponding to the filter of the second frequency, the first filtering result and the second filtering result are linearly superimposed to obtain a noise reduction error of the second image.

4. The method according to claim 1 or 3, characterized in that The first filtering formula is: addnoise = noisegain * (alpha_1 * filter_1 (noisemap) + alpha_2 * filter_2 (noisemap) + … + alpha_n * filter_n (noisemap)) Among them, the addnoise represents the noise reduction error of the second image; the noisemap represents the noise reduction map, which is obtained by subtracting the second image from the first image; the noisegain represents the gain value of the noise reduction error; the alpha_1 to alpha_n represent the weight coefficients corresponding to filters of different frequencies; and the filter_1 to filter_n represent different filtering frequencies of multiple filters.

5. The method according to claim 1, wherein The method of obtaining the filter coefficient model includes: The first electronic device obtains the filter coefficient model by training the training data obtained by processing the acquired training sample set; or The first electronic device deploys the filter coefficient model provided by the second electronic device, wherein the filter coefficient model is trained by the second electronic device based on training data obtained by processing the acquired training sample set.

6. The method according to claim 5, characterized in that The training process of the filter coefficient model includes: Acquire a training sample set, the training sample set including a first sample image set, sensitivities corresponding to the first sample image set, and a second sample image set; Training a first model based on the second sample images in the second sample image set and the sensitivities corresponding to the first sample image set to obtain the filter coefficient model, wherein the first model includes a convolutional neural network model; Obtaining a third weight coefficient and a third frequency and a fourth weight coefficient and a fourth frequency of a filter corresponding to the output of the first model; performing a third filtering process on the third sample image in the third sample image set using the first filter of the third frequency to obtain a third filtering result; and performing a fourth filtering process on a fourth sample image in a fourth sample image set using a second filter with a fourth frequency to obtain a fourth filtering result, wherein the third frequency is different from the fourth frequency; performing linear superposition on the third filtering result and the fourth filtering result based on a third weight coefficient corresponding to the filter of the third frequency and a fourth weight coefficient corresponding to the filter of the fourth frequency to obtain a denoising error of the second sample image set; The noise data of the third sample image set is calculated based on the difference between the image data of the first sample image set and the image data of the second sample image set.

7. The method according to claim 6, characterized in that The training process of the filter coefficient model also includes: Correcting the second sample images in the second sample image set based on the noise reduction error of the second sample image set to obtain a fourth sample image set; Determining a first similarity between the fourth sample image set and the first sample image set based on a first loss function; wherein the first loss function is used to calculate a difference between the fourth sample image set data and the first sample image set data; Based on the first similarity, the model parameters of the first model are adjusted to obtain the filter coefficient model.

8. The method according to claim 6, characterized in that The training sample set includes a first sample image set, sensitivities corresponding to the first sample image set, and a second sample image set, and, The obtaining of the training sample set comprises: Acquire the first sample image set and a first sensitivity corresponding to the first sample image set; The first sample image set is preprocessed to obtain the second sample image set.

9. The method according to claim 8, characterized in that Preprocessing the first sample image set to obtain the second sample image set includes: performing degradation processing and noise adding processing on the first sample image set based on a first degradation parameter and a first noise parameter to obtain a first sample image set containing noise; Based on the second noise reduction process, the noise-containing first sample image set is subjected to noise reduction process to obtain the second sample image set.

10. The method according to claim 1, characterized in that The image optical parameters include sensitivity, brightness, and pixel values ​​of the first image.

11. A model training method, applied to a first electronic device or a second electronic device, characterized in that: For implementing the filter coefficient model according to any one of claims 1 to 10, the method comprises: Obtaining a training sample set, the training sample set including a first sample image set, a first sensitivity corresponding to the first sample image set, and a second sample image set, wherein the first sample images in the first sample image set correspond one-to-one to the second sample images in the second sample image set, and the second sample images are obtained by performing degradation processing, noise addition processing, and noise reduction processing on the first sample images; The network model is trained using the second sample images in the second sample image set and the first sensitivity to obtain a filter coefficient model.

12. The method according to claim 11, characterized in that The method further comprises: Based on the third sample image set and the filter coefficient, combined with the first filter formula, a noise reduction error of the noise reduction process is obtained; wherein the noise data of the third sample image set is calculated based on the difference between the first sample image set data and the second sample image set data; Based on the noise reduction error, the second sample image set is corrected to obtain a fourth sample image set; Based on the first sample image set data, the fourth sample image set and the first loss function, the training parameters of the network model are adjusted to obtain the filter coefficient model.

13. The method according to claim 11, characterized in that The first sample image set is a noise-free image set.

14. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the image denoising method described in any one of claims 1-10, or executes the model training method described in any one of claims 11-13.

15. A computer-readable medium, characterized in that The readable medium stores instructions, which, when executed on a computer, cause the computer to execute the image denoising method described in any one of claims 1-10, or the model training method described in any one of claims 11-13.

16. A computer program product, characterized in that It includes a computer program / instruction, which, when executed by a processor, implements the image denoising method described in any one of claims 1 to 10, or implements the model training method described in any one of claims 11 to 13.

Images