Image noise reduction method and device, and storage medium
By collecting multiple frames of images for pixel-by-pixel averaging and true value pair training, and combining it with a preset noise reduction intensity function, the problems of computing resource consumption and adjustability of the neural network model in the image denoising process are solved, and the synergy and noise reduction effect of the image processing process are improved.
Patent Information
- Application Number
- CN202510875554.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-10-10
AI Technical Summary
Existing neural network models consume excessive computing resources during image denoising, have insufficient generalization capabilities, have poor adjustability of denoised images, and have poor coordination with image processing processes.
By collecting multiple frames of static images of the same scene and averaging them pixel by pixel, a noise neural network model is established. The model is trained using true value pairs, and the predicted noise image is adjusted in combination with a preset noise reduction intensity function to generate the final denoised image.
The adjustability and coordination of image noise reduction are improved, ensuring the coordination of noise reduction effects in the image processing flow and subsequent processing modules, and realizing flexible image noise reduction control.
Smart Images
Figure CN120765495A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image noise reduction method, device, and storage medium. Background Art
[0002] The development of artificial intelligence (AI) technology has provided new methods and solutions to image denoising in the field of image processing. The emergence of neural network models, in particular, has significantly improved the efficiency and quality of image denoising. Neural network models are a relatively early step in the image processing process, and their output quality is crucial to the subsequent processing of image data. However, due to their inherent characteristics, applying neural network models to image denoising can lead to problems such as excessive computational resource consumption, insufficient generalization capabilities, and poor adjustability of the denoised image. Therefore, when applying neural network models to image denoising, the flexibility of denoising deserves attention. Summary of the Invention
[0003] In view of this, the embodiments of the present application provide an image denoising method, device, and storage medium in order to improve the flexibility of image denoising processing. In a first aspect, an image denoising method is provided, comprising: inputting an image to be processed into a preset noise neural network model to determine a first predicted noise image; wherein, establishing the noise neural network model comprises: collecting multiple original images, where the multiple original images are multiple static frames of the same picture; performing pixel-by-pixel averaging on the multiple original images to determine a noise-free image; selecting any frame of the original image from the multiple original images as a noisy image; determining a true value pair based on the noisy image and the noise-free image; generating a noise neural network model based on the training of the true value pair; determining a second predicted noise image based on the first predicted noise image and a preset noise reduction intensity function; and generating a first denoised image based on the image to be processed and the second predicted noise image.
[0004] The above image denoising method can obtain a noisy image (a first predicted noise image) of the image to be processed through a pre-set noise neural network model, and then adjust the first predicted noise image to a second predicted noise image in combination with a preset noise reduction intensity function to improve the adjustability of image noise reduction. On the basis of improving the adjustability of image noise reduction, because the image noise reduction process is more controllable, although image noise reduction is at the front end of the image processing flow, the degree of noise reduction can continue to be dynamically adjusted based on the image quality obtained in the subsequent stages, thereby improving the synergy of image processing.
[0005] Optionally, a noisy neural network model is generated based on true value pair training, including: using the noisy image as a training sample, forward propagating the noisy neural network model to determine the predicted image; using the loss function to calculate the loss function value based on the difference between the predicted image and the noise-free image corresponding to the predicted image; based on the loss function value, calculating the gradient of the noisy neural network model through a backpropagation algorithm; based on the gradient, using an optimizer to update the parameters of the noisy neural network model.
[0006] Optionally, loss functions include: mean absolute error and mean squared error.
[0007] Optionally, determining the second predicted noise image based on the first predicted noise image and a preset noise reduction strength function includes: performing weighted processing on the first predicted noise image based on a noise reduction strength parameter to determine the second predicted noise image.
[0008] Optionally, determining the second predicted noise image based on the first predicted noise image and a preset noise reduction intensity function also includes: determining the second denoised image based on the first predicted noise image and the image to be processed; performing pixel-by-pixel processing on the second denoised image based on the noise reduction intensity curve to determine the third predicted noise image; and determining the second predicted noise image based on the first predicted noise image and the third predicted noise image.
[0009] Optionally, the method further includes: determining a second predicted noise image based on the first predicted noise image, the third predicted noise image and the noise reduction strength parameter.
[0010] Optionally, the denoising strength curve is determined based on pixels of the original image and a preset denoising style.
[0011] Optionally, generating a first denoised image based on the image to be processed and the second predicted noise image includes: performing a difference operation on the image to be processed and the second predicted noise image to determine the first denoised image.
[0012] In a second aspect, an image denoising device comprises: an input unit for inputting an image to be processed into a preset noise neural network model to determine a first predicted noise image; a model building unit for building a noise neural network model, including: collecting multiple original images, where the multiple original images are multiple static frames of the same picture; performing pixel-by-pixel averaging on the multiple original images to determine a noise-free image; selecting any frame of original image from the multiple original images as a noisy image; determining a true value pair based on the noisy image and the noise-free image; generating a noise neural network model based on the training of the true value pair; a determination unit for determining a second predicted noise image based on the first predicted noise image and a preset noise reduction intensity function; and an output unit for generating a first denoised image based on the image to be processed and the second predicted noise image.
[0013] In a third aspect, a computer-readable storage medium is provided, comprising instructions stored thereon, wherein when the instructions are executed by a processor, the image denoising method provided in the first aspect is executed. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The following is a brief introduction to the drawings used in describing the embodiments of this application:
[0015] Figure 1 A schematic diagram showing a process of an image noise reduction method provided in some embodiments of the present application is shown;
[0016] Figure 2 A schematic diagram of pixel-by-pixel averaging of multiple original images provided in some embodiments of the present application is shown;
[0017] Figure 3 A schematic diagram of a process for generating a noise neural network model based on true value pair training provided in some embodiments of the present application is shown;
[0018] Figure 4 A schematic diagram of a noise neural network model provided in some embodiments of the present application is shown;
[0019] Figure 5 A schematic diagram showing an image noise reduction process provided in some embodiments of the present application is shown;
[0020] Figure 6 A schematic flow chart of a method for determining a second predicted noise image provided in some embodiments of the present application is shown;
[0021] Figure 7 A schematic diagram showing a noise reduction intensity curve provided in some embodiments of the present application is shown;
[0022] Figure 8 A schematic diagram showing another image noise reduction process provided in some embodiments of the present application is shown;
[0023] Figure 9 A schematic structural diagram of an image noise reduction device provided in some embodiments of the present application is shown. DETAILED DESCRIPTION
[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, examples of implementation of the present application will be described below with reference to the accompanying drawings. The drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings and other implementations can be obtained based on these drawings without any creative work. Adjustments and improvements made without departing from the concept of the present application are all within the scope of protection of the present application.
[0025] To simplify the drawings, the figures schematically illustrate only the portions relevant to the embodiments and do not represent the actual structure of the products. Furthermore, to simplify the drawings and facilitate understanding, in some figures, only a portion of components with the same structure or function are schematically depicted; in practice, more or fewer components with the same structure or function may exist.
[0026] In this application, unless otherwise expressly specified and limited, ordinal numbers such as "first" and "second" are only used to distinguish and describe related objects, and cannot be understood as indicating or implying the relative importance or order between related objects; in addition, they do not represent the number of related objects. "Multiple" includes two or more, and other quantifiers are similar. " / " is used to describe the relationship between related objects, which indicates the "or" relationship between related objects. "And / or" is used to describe the relationship between related objects, which includes any combination relationship between related objects, for example, "a and / or b" includes: "alone a", "alone b", or "a and b". "One or more" or "at least one" in multiple objects refers to any object or any combination of multiple objects, for example, "one or more of a1, a2, a3" or "at least one of a1, a2, a3" includes: "alone a1", "alone a2", "alone a3", "a1 and a2", "a1 and a3", "a2 and a3", or "a1, a2 and a3".
[0027] The development of artificial intelligence (AI) technology has provided new methods and solutions to image denoising in the field of image processing. The emergence of neural network models, in particular, has significantly improved the efficiency and quality of image denoising. Neural network models are a relatively early step in the image processing process, and their output quality is crucial to the performance of subsequent image processing steps. Image denoising is a key first-level module in the image signal processing pipeline (ISP pipeline), typically located at the front end of the image processing process. However, due to their inherent characteristics, applying neural network models to image denoising can lead to challenges such as excessive computational resource consumption, insufficient generalization, poor tunability of the denoised image, and poor interoperability with other modules in the image processing pipeline. This poor tunability is primarily due to the fact that existing AI denoising models generally rely on predictive noise addition, limiting the controllable parameters to the predicted noise intensity. Traditional denoising algorithms, on the other hand, require manual adjustment of the denoising effect using a variety of parameters, such as noise level and texture preservation. On the other hand, the poor synergy is manifested in that the artificial intelligence denoising module is at the front-end position of the image processing flow, and its output quality will affect the effect of the image data in the subsequent image processing flow. If the artificial intelligence denoising intensity is too weak, the image processed by the subsequent module will have noise, thereby lowering the algorithm effect in the subsequent processing flow (such as sharpening modules); if the intensity is too strong, it will be difficult for the subsequent stage to restore details such as textures, and may cause color output distortion. In view of this, an image denoising method, device, and storage medium of the present application, through reasonable training of a neural network model for image denoising, generates a predicted noise image after the image to be processed is input into the model, and combines the predicted noise image with the preset noise reduction intensity function to determine a new predicted noise image, and finally generates a denoised image in combination with the image to be processed. The preset noise reduction intensity function can process pixel by pixel according to the pixel situation of the image to be processed, combined with the setting of the denoising style, thereby improving the adjustability of image noise reduction.
[0028] Figure 1 A flow chart of an image noise reduction method provided in some embodiments of the present application is shown, and the method includes at least the following steps:
[0029] S110: Inputting the image to be processed into a preset noise neural network model to determine a first predicted noise image; wherein establishing the noise neural network model includes: collecting multiple original images, where the multiple original images are multiple static frames of the same picture; performing pixel-by-pixel averaging on the multiple original images to determine a noise-free image; selecting any one original image from the multiple original images as a noisy image; determining a true value pair based on the noisy image and the noise-free image; and training and generating the noise neural network model based on the true value pair;
[0030] S120: Determine a second predicted noise image based on the first predicted noise image and a preset noise reduction strength function;
[0031] S130: Generate a first denoised image based on the image to be processed and the second predicted noise image.
[0032] In the above image denoising method, the establishment of a noise neural network model can separate the noise from the image to be processed, resulting in a first predicted noise image. This predicted noise image only contains the noise of the image to be processed. This allows for pixel-by-pixel denoising of the image to be processed during subsequent denoising of the image to be processed, thereby improving the scalability of the denoising. When establishing the noise neural network model, multiple original images are simultaneously acquired. These original images are multiple static frames of the same scene, such as 10 to 20 frames captured of the same object. After acquiring these multiple original images, they can be pixel-by-pixel averaged. Pixel-by-pixel averaging involves taking the average of each pixel across multiple frames (e.g., the aforementioned 10 or 20 frames) captured consecutively from the same static scene. Noise in an image can appear irregularly, but can be offset across multiple captures. For example, when a camera sensor captures an image, some random noise (such as thermal noise and readout noise) is inevitably superimposed. However, when capturing multiple static frames (with an unchanged scene and sensor), the true content of each frame remains the same, but the noise components of each frame are random and independent. Therefore, by collecting multiple frames of original images, the real content reflected is constant in each frame, but the noise is different. Figure 2 A schematic diagram of pixel-by-pixel averaging of multiple original images provided in some embodiments of the present application is shown. Figure 2 When averaging multiple frames at each pixel, the constant real content will be retained, and the randomly fluctuating noise components will cancel each other out and gradually approach zero. For example, refer to formula 1 for averaging:
[0033]
[0034] Among them, I i is the value of the i-th frame image at pixel (x, y), N is the total number of frames, I cleanThe averaged image, i.e., the noise-free image, is obtained. The averaging process can be performed pixel by pixel, with the average value of each pixel across all frames taken to generate a noise-free image. This noise-free image and a noisy frame selected from multiple original images are used as a ground-truth pair. During the training phase, this ground-truth pair serves as the training dataset to design a noisy neural network model. Through model training, the noisy neural network model can be trained to achieve the desired noise prediction effect. For example, a noisy image (the original image containing noise) can be input into the model, and the network layer by layer calculates the output prediction value for forward propagation. The loss function is then used to calculate the quantitative difference between the predicted value and the noise-free image corresponding to the noisy image, and the error is calculated. The chain rule is used to calculate the gradient of the loss function with respect to the network parameters, thereby transferring the error signal from the output layer to the input layer for backpropagation. Finally, by updating the parameters of the optimizer (such as the Adam optimizer) during training, the network weights are adjusted based on the gradient, and the parameters of the noisy neural network model are updated so that when the image to be processed is input, the ideal first predicted noise image is obtained. The preset noise reduction intensity function can also perform pixel-by-pixel processing. For example, after determining a first predicted noise image, if there are multiple pixels in the image, the preset noise intensity function can assign weights to each pixel based on its pixel value, adjusting the overall noise of the first predicted noise image and improving the efficiency of the noise reduction process. Alternatively, based on the multiple different pixel values presented in the first predicted noise image, new pixel values can be determined according to the preset noise intensity function, thereby adjusting the noise reduction intensity of different pixels in the same first predicted noise image and generating a second predicted noise image. The noise in the new predicted noise image has a certain correspondence with the noise in the image to be processed. Based on this correspondence, the noise position of the current image can be determined from the noise position of the second predicted noise image in the image to be processed, and the noise can be removed to achieve the purpose of image noise reduction. For example, the first denoised image can be determined by subtracting the image to be processed from the second predicted noise image, or the pixel value can be re-determined based on the ratio of each pixel value of the second predicted noise image to each pixel value of the image to be processed, thereby forming the first denoised image. The above image denoising method can obtain a noisy image (a first predicted noise image) of the image to be processed through a pre-set noise neural network model, and then adjust the first predicted noise image to a second predicted noise image in combination with a preset noise reduction intensity function to improve the adjustability of image noise reduction. On the basis of improving the adjustability of image noise reduction, because the image noise reduction process is more controllable, although image noise reduction is at the front end of the image processing flow, the degree of noise reduction can continue to be dynamically adjusted based on the image quality obtained in the subsequent stages, thereby improving the synergy of image processing.
[0035] Figure 3 A schematic diagram of a process for generating a noise neural network model based on truth value pair training provided in some embodiments of the present application is shown, including:
[0036] S310: Using the noisy image as a training sample, forward propagating the noise neural network model to determine a predicted image;
[0037] S320: Calculating a loss function value based on a difference between the predicted image and a noise-free image corresponding to the predicted image using the loss function;
[0038] S330: Calculating the gradient of the noisy neural network model through a back propagation algorithm based on the loss function value;
[0039] S340: Based on the gradient, use the optimizer to update the parameters of the noisy neural network model.
[0040] In the above embodiments, a noisy image can be input as a training sample into a noisy neural network model. A forward propagation operation is performed, causing the network to sequentially pass through modules such as convolution, normalization, activation, and feature mapping, outputting a predicted image with the same dimensions as the input image. This predicted image can be represented as the noise prediction result of the network for the input noisy image under the current parameter state. Next, the difference between the network's predicted image and its corresponding noise-free image is calculated, and a loss function (such as the L1 loss function or the L2 loss function) is used to quantify this difference to obtain a loss function value. This value measures the degree of deviation between the network's current output and the true target, with smaller values indicating better network performance. Subsequently, the loss function is derived using a backpropagation algorithm. The partial derivatives of the loss function with respect to each network parameter are calculated layer by layer according to the chain rule, and the gradient of each network parameter is obtained by backpropagating from the output layer to the input layer. The loss function can include mean absolute error (MAE) or mean squared error (MSE). The mean absolute error (MAE) is a loss function that measures the absolute difference between the predicted image and the noise-free image at each pixel and averages the difference across all pixels, reflecting the degree of linear deviation of the overall error. This loss function is insensitive to outliers and can be used for image data with relatively uniform noise distribution. The mean squared error (MSE) emphasizes pixel regions with large errors by averaging the squared differences between the predicted and true values. It is more sensitive to outliers and can be used for high-precision image restoration tasks. During training, different loss functions can be selected based on the specific image noise type and target accuracy requirements to guide the optimization direction of the network weights and improve the denoising performance of the neural network. Finally, based on the aforementioned gradients, the optimizer is called to update the network parameters, enabling the network to more accurately complete the image noise prediction task in the next training iteration. Figure 4The figure shows a working diagram of a noise neural network model provided in some embodiments of the present application. The training process of the noise neural network model is processed by the convolution layer Conv, the normalization layer Norm and the activation function Act each time, so as to extract features through the convolution layer Conv, maintain numerical stability by using the normalization layer Norm, and enhance the nonlinear expression ability by the activation function Act, thereby alleviating the difficulty of deep network training, accelerating convergence, and improving performance, and finally obtaining a noisy image containing only noise, such as Figure 4 The butterfly image shown above, after passing through the noise neural network model, produces the first predicted noise image containing only noise. This training process can be repeated on multiple image samples, gradually improving the model's noise prediction capabilities through iterative optimization.
[0041] In some embodiments of the present application, determining the second predicted noise image based on the first predicted noise image and a preset noise reduction strength function includes: performing weighted processing on the first predicted noise image based on a noise reduction strength parameter to determine the second predicted noise image.
[0042] In the present application, the preset noise reduction strength function may be in parameter form, such as k=0.7, and the first predicted noise image is weighted by the noise reduction strength parameter, and the first predicted noise image is processed pixel by pixel during weighting. Figure 5 FIG2 shows a schematic diagram of an image noise reduction process provided in some embodiments of the present application. For example, Figure 5 As shown, the image to be processed may be an image consisting of 3×3 pixels, with pixel values from the first row to the third row being 32, 64, 96, 128, 160, 192, 200, 220, and 240, respectively. After inputting the image into the noise neural network model for processing, a first predicted noise image is obtained, with pixel values varying from the first row to the third row being 10, 10, 10, 20, 20, 20, 30, 30, and 30. The first predicted noise image is then weighted pixel-by-pixel processed using a noise reduction intensity parameter to generate a second predicted noise image, with pixel values from the first row to the third row being 7, 7, 7, 14, 14, 14, 21, 21, and 21, respectively. After determining the second predicted noise image, pixel-by-pixel subtraction is performed between the image to be processed and the second predicted noise image to determine a first denoised image, with pixel values from the first row to the third row being 25, 57, 89, 114, 146, 178, 179, 199, and 219, respectively. Thus, the noise reduction of the image to be processed is completed.
[0043] Figure 6 A schematic flowchart of a method for determining a second predicted noise image provided in some embodiments of the present application is shown, the method comprising:
[0044] S610: Determine a second denoised image based on the first predicted noise image and the image to be processed;
[0045] S620: Process the second denoised image pixel by pixel based on the noise reduction strength curve to determine a third predicted noise image;
[0046] S630: Determine a second predicted noise image based on the first predicted noise image and the third predicted noise image.
[0047] Figure 7 A schematic diagram of a noise reduction intensity curve provided in some embodiments of the present application is shown. The preset noise reduction intensity function of the present application can be a curve based on a functional form, and the noise reduction intensity curve can be a function that associates pixels with intensities. The noise reduction intensity curve can be determined based on the pixels of the original image and a preset denoising style. This function constructs a curve that associates pixels with intensities based on the characteristic that the noise type of the image varies with pixel intensity. It can be used to determine the denoising intensity corresponding to different pixel positions during the noise reduction process, thereby achieving a more refined and controllable image restoration effect. The determination of the noise reduction intensity curve can be based on the training image sensor noise being approximated as a combination of multiple noise components, such as Poisson noise related to the pixel value of the noise-free image and the sensor type, and Gaussian noise related to the sensitivity and sensor type. Typically, dark areas (i.e., areas with low pixel intensity) are mainly affected by Gaussian noise, appearing as high-frequency color noise, while bright areas are more affected by Poisson noise, appearing as texture interference or blurred details. Based on this characteristic, the noise reduction intensity curve can dynamically set noise reduction parameters according to the pixel brightness value. When designing the noise reduction strength curve, when the pixel brightness is low (for example, lower than the first threshold, such as a pixel value less than 30), in order to suppress obvious Gaussian noise, the noise reduction strength can be set to a higher value (such as 0.8 to 0.9) to achieve a stronger noise reduction effect; when the pixel brightness is high (for example, higher than the second threshold, such as a pixel value greater than 200), considering that there may be more image structure information and details in this area, in order to avoid excessive denoising leading to texture loss, the noise reduction strength can be set to a lower value (such as 0.1 to 0.2). In the intermediate brightness range, the noise reduction strength can be determined by linear interpolation, piecewise function or empirical fitting curve to achieve a smooth transition. The noise reduction strength curve can be obtained through manual parameter adjustment or collaborative training with image quality evaluation engineers based on the response characteristics of the specific image sensor and the target image style, helping to improve the adjustability of image noise reduction and image processing quality. Figure 8A schematic diagram of another image denoising process provided in some embodiments of the present application is shown. In an embodiment of the present application, the image to be processed can be first input into a noise neural network model to generate a first predicted noise image, and then the image to be processed and the first predicted noise image are subtracted pixel by pixel to obtain a second denoised image. The second denoised image is not the final desired denoising result, but a relatively rough denoising process. Some real details may be destroyed during the subtraction, affecting the subsequent image processing flow. However, the figure can be considered to have the smallest difference in loss function from the noise-free image, and it can be considered that the pixel value can be used to predict the size of the noise somewhere. Therefore, the second denoised image can be used as a basis, combined with a preset denoising intensity curve, and the second denoised image can be processed again according to the changing relationship of the curve. Figure 7 Taking the noise reduction intensity curve formed as an example, its function expression refers to Formula 2:
[0048]
[0049] Based on the pixel value of each pixel and the value corresponding to the noise reduction intensity curve, a third predicted noise image is determined, and then the first predicted noise image is multiplied by the third predicted noise image to determine the second predicted noise image. This method effectively improves image purity in low-light areas while preserving detailed textures in highlight areas, resulting in better overall image visual quality. The adjustable noise reduction intensity curve makes the image noise reduction process more controllable, further improving the synergy of the image processing process.
[0050] In some embodiments of the present application, the second predicted noise image may be determined based on the first predicted noise image, the third predicted noise image, and the noise reduction strength parameter. That is, after determining the third predicted noise image by the noise reduction strength curve in the above embodiment, the noise reduction strength parameter may be further combined, such as Figure 8 The process shown generates a new second predicted noise image, and adjusts the overall image prediction noise by changing the noise reduction intensity parameter, thereby improving the image noise reduction processing efficiency.
[0051] In some embodiments of the present application, generating a first denoised image based on the image to be processed and the second predicted noise image includes: performing a difference operation between the image to be processed and the second predicted noise image to determine the first denoised image. Figure 8As shown, the image to be processed is a 3×3 pixel image, with the pixel values from the first row to the third row being 32, 64, 96, 128, 160, 192, 200, 220, and 240, respectively. The pixel values of the second predicted noise image from the first row to the third row are 0.7, 0.7, 0.7, 7, 7, 7, 10.5, 18.9, and 18.9, respectively. The pixel values of the first denoised image determined after subtraction are 31.3, 63.3, 95.3, 121, 153, 185, 189.5, 201.1, and 221.1, respectively, from the first row to the third row. Further processing can make each pixel value positive to comply with conventional image pixel processing rules.
[0052] Figure 9 A structural schematic diagram of an image denoising device provided in some embodiments of the present application is shown, including: an input unit 910, used to input an image to be processed into a preset noise neural network model to determine a first predicted noise image; a model building unit 920, used to build a noise neural network model, including: collecting multiple original images, where the multiple original images are multiple static frames of the same picture; performing pixel-by-pixel averaging on the multiple original images to determine a noise-free image; selecting any frame of original image from the multiple original images as a noisy image; determining a true value pair based on the noisy image and the noise-free image; and generating a noise neural network model based on the true value pair training; a determination unit 930, used to determine a second predicted noise image based on the first predicted noise image and a preset noise reduction intensity function; and an output unit 940, used to generate a first denoised image based on the image to be processed and the second predicted noise image.
[0053] The division of the above units is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. In addition, the above units can be implemented in the form of a processor calling software; for example, the detection device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU), and the memory is a memory within the device or a memory outside the device. Alternatively, the above units can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in some embodiments, the hardware circuit is an application specific integrated circuit (ASIC), and the functions of some or all units above can be realized by designing the logical relationship between the components within the circuit; for example, in another implementation, the hardware circuit can be implemented by a programmable logic device (PLD), which can include a large number of logic gate circuits, and the logical relationship between the logic gate circuits is configured by a configuration file, thereby realizing the functions of some or all units above. All units of the above apparatus may be implemented entirely in the form of a processor calling a program, or entirely in the form of a hardware circuit, or partially in the form of a processor calling a program and the rest in the form of a hardware circuit.
[0054] In addition, an embodiment of the present application further provides a computer-readable storage medium, including instructions stored thereon, which, when called by a processor, execute any of the image denoising methods in the above embodiments. An embodiment of the present application further provides a computer program (or computer program product), including instructions, which, when called by a processor, executes any of the image denoising methods in the above embodiments.
[0055] The above-mentioned computer-readable storage medium can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.
[0056] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. In addition, the above embodiments can be freely combined as needed.
Claims
1. An image denoising method, characterized in that: include: Inputting the image to be processed into a preset noise neural network model to determine a first predicted noise image; The establishment of the noise neural network model includes: collecting multiple original images, where the multiple original images are multiple static frames of the same picture; averaging the plurality of original images pixel by pixel to determine a noise-free image; Selecting any one frame of the original image from the plurality of original images as a noisy image; Determining a ground truth pair based on the noisy image and the noise-free image; Generate the noise neural network model based on the true value pair training; determining a second predicted noise image based on the first predicted noise image and a preset noise reduction strength function; A first denoised image is generated based on the image to be processed and the second predicted noise image.
2. The image denoising method according to claim 1, wherein: Generating the noise neural network model based on the true value pair training includes: Using the noisy image as a training sample, forward propagating the noise neural network model to determine a predicted image; Calculating a loss function value based on a difference between the predicted image and a noise-free image corresponding to the predicted image using a loss function; Based on the loss function value, calculating the gradient of the noisy neural network model through a back propagation algorithm; Based on the gradient, the parameters of the noisy neural network model are updated using an optimizer.
3. The image denoising method according to claim 2, wherein: The loss function includes: mean absolute error and mean square error.
4. The image denoising method according to claim 1, wherein: The determining of a second predicted noise image based on the first predicted noise image and a preset noise reduction strength function includes: The first predicted noise image is weighted based on the noise reduction strength parameter to determine the second predicted noise image.
5. The image denoising method according to claim 1, wherein: The determining of a second predicted noise image based on the first predicted noise image and a preset noise reduction strength function further includes: Determining a second denoised image based on the first predicted noise image and the image to be processed; performing pixel-by-pixel processing on the second denoised image based on the noise reduction strength curve to determine a third predicted noise image; The second predicted noise image is determined based on the first predicted noise image and the third predicted noise image.
6. The image denoising method according to claim 5, wherein: Also includes: The second predicted noise image is determined based on the first predicted noise image, the third predicted noise image, and a noise reduction strength parameter.
7. The image denoising method according to claim 5 or 6, wherein: The noise reduction strength curve is determined based on pixels of the original image and a preset denoising style.
8. The image denoising method according to any one of claims 1 to 6, wherein: The step of generating a first denoised image based on the image to be processed and the second predicted noise image includes: A difference operation is performed between the image to be processed and the second predicted noise image to determine the first denoised image.
9. An image noise reduction device, characterized in that: include: An input unit, configured to input an image to be processed into a preset noise neural network model to determine a first predicted noise image; The model building unit is configured to build the noise neural network model, comprising: collecting a plurality of original images, wherein the plurality of original images are multiple static frames of the same picture; performing pixel-by-pixel averaging on the plurality of original images to determine a noise-free image; selecting any one of the plurality of original images as a noisy image; determining a true value pair based on the noisy image and the noise-free image; and training and generating the noise neural network model based on the true value pair. a determining unit, configured to determine a second predicted noise image based on the first predicted noise image and a preset noise reduction strength function; An output unit is configured to generate a first denoised image based on the image to be processed and the second predicted noise image.
10. A computer-readable storage medium, characterized in that The device comprises instructions stored thereon, wherein when the instructions are executed by a processor, the image denoising method according to any one of claims 1 to 8 is executed.