Image noise reduction method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to the technical field of image processing, and provides an image noise reduction method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring a to-be-processed image; performing color noise reduction processing on the to-be-processed image to obtain a preliminary noise-reduced image; fusing the preliminary noise reduction image and the to-be-processed image to obtain a detail reduction image; generating a noise reduction enhanced image based on the detail reduction image and the color noise area image of the preliminary noise reduction image; and determining a target noise reduction image corresponding to the to-be-processed image based on the noise reduction enhanced image and the to-be-processed image. By adopting the scheme of the invention, the color noise in the finally obtained target noise reduction image is accurately removed, the problem of edge color overflow is avoided, and the image quality is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image noise reduction method, device, electronic device, storage medium and program product. Background Art

[0002] Research on the human visual system shows that the human eye has different spatial resolution capabilities for brightness and chromaticity. The resolution of chromaticity in the low-frequency band is greater than that of high-frequency bright noise, which means that the amount of low-frequency color noise has a greater impact on the overall quality of the image.

[0003] At present, the commonly used methods to remove low-frequency color noise mainly include using multiple frames of images or using digital noise reduction algorithms to remove low-frequency color noise. Existing methods for removing color noise are difficult to control the intensity of noise reduction. If the intensity of color noise reduction is too large, it is easy to cause edge overflow problems in the image after noise reduction, affecting image quality. Summary of the invention

[0004] Based on this, it is necessary to provide an image denoising method, device, electronic device, storage medium and program product that can effectively remove image color noise while avoiding image edge overflow in response to the above technical problems.

[0005] The present disclosure provides an image noise reduction method, the method comprising:

[0006] Get the image to be processed;

[0007] Performing color noise reduction processing on the image to be processed to obtain a preliminary noise-reduced image;

[0008] Fusing the preliminary denoised image and the image to be processed to obtain a detail restored image;

[0009] Generate a noise reduction enhanced image based on the detail restoration image and the color noise region image of the preliminary noise reduction image;

[0010] Based on the denoised and enhanced image and the image to be processed, a target denoised image corresponding to the image to be processed is determined.

[0011] In one embodiment, the generating of the noise reduction enhanced image based on the detail restored image and the color noise region image of the preliminary noise reduction image includes:

[0012] Performing color noise detection on a target image using a pre-trained color noise detection model to obtain a color noise detection result, wherein the target image is the image to be processed or the preliminary denoised image;

[0013] Determining the color noise region image from the preliminary noise reduction image based on the color noise detection result;

[0014] The detail restoration image and the color noise area image are fused to obtain the noise reduction and enhanced image.

[0015] In one embodiment, the step of fusing the detail restored image and the color noise region image to obtain the noise reduction enhanced image includes:

[0016] Get the image quality requirements set by the user;

[0017] In a case where the image quality requirement is a first requirement parameter, obtaining a first weight and a second weight, wherein the first requirement parameter is used to indicate that the priority of the user's requirement for image details is higher than the priority of the user's requirement for noise reduction effect, and the first weight is greater than the second weight;

[0018] Based on the first weight and the second weight, a weighted sum of pixel values ​​of each pixel point at a corresponding position in the detail restoration image and the color noise area image is calculated to obtain the noise reduction and enhanced image.

[0019] In one embodiment, the step of fusing the detail restored image and the color noise region image to obtain the noise reduction enhanced image further includes:

[0020] When the image quality requirement is the second requirement parameter, updating the pixel values ​​of the pixels corresponding to the color noise region image in the detail restoration image to the pixel values ​​of the pixels at the corresponding positions in the color noise region image;

[0021] The second requirement parameter is used to indicate that the priority of the user's requirement for image details is not higher than the priority of the user's requirement for noise reduction effect.

[0022] In one embodiment, determining a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed includes:

[0023] Performing face detection on the image to be processed based on a pre-trained face detection model to obtain a face detection result;

[0024] When the face detection result indicates that there is no face in the image to be processed, the denoised and enhanced image is determined as a target denoised image corresponding to the image to be processed.

[0025] In one embodiment, the step of determining a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed further includes:

[0026] When the face detection result indicates that a face exists in the image to be processed, determining a face region image from the image to be processed based on the face detection result;

[0027] The denoised and enhanced image is fused with the face region image to obtain a target denoised image corresponding to the image to be processed.

[0028] The present disclosure also provides an image noise reduction device, the device comprising:

[0029] An image acquisition module, used for acquiring an image to be processed;

[0030] A noise reduction module, used for performing color noise reduction processing on the image to be processed to obtain a preliminary noise reduction image;

[0031] A first fusion module, used for fusing the preliminary denoised image and the image to be processed to obtain a detail restored image;

[0032] A second fusion module, configured to generate a noise reduction enhanced image based on the detail restoration image and the color noise region image of the preliminary noise reduction image;

[0033] The image determination module is used to determine a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed.

[0034] An embodiment of the present disclosure further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the image noise reduction method provided in any embodiment of the present disclosure are implemented.

[0035] An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the image denoising method provided by any embodiment of the present disclosure are implemented.

[0036] An embodiment of the present disclosure provides a computer program product, which is used to execute the steps of the image denoising method provided by any embodiment of the present disclosure.

[0037] The image denoising method, device, electronic device and storage medium provided by the embodiments of the present disclosure obtain an image to be processed; perform color noise reduction processing on the image to be processed to obtain a preliminary denoised image; fuse the preliminary denoised image and the image to be processed to obtain a detail restored image; generate a denoised enhanced image based on the detail restored image and the color noise region image of the preliminary denoised image; and determine a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed. By adopting the scheme of the present disclosure, the image to be processed can be subjected to color noise reduction processing and then fused with the image to be processed to restore the detail information in the image, and then fused with the color noise region image of the image after the color noise reduction processing to enhance the color noise reduction effect in the color noise region, so that the target denoised image obtained can accurately remove the color noise in the image while avoiding the edge overflow problem and improving the image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic diagram of a flow chart of an image noise reduction method in one embodiment;

[0039] Figure 2 is a schematic flow chart of an image noise reduction method in another embodiment;

[0040] Figure 3 is a structural block diagram of an image noise reduction device in one embodiment;

[0041] Figure 4 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0043] Before explaining the solution of the present disclosure, the professional terms that may be involved in the present disclosure are explained as follows:

[0044] Camera: A camera (or WEBCAM), also known as a computer camera, computer eye, electronic eye, etc., is a video input device that is widely used in video conferencing, telemedicine, and real-time monitoring.

[0045] YUV: refers to the pixel format in which the brightness parameter and the chrominance parameter are expressed separately, where "Y" represents brightness (Luminance or Luma), that is, the grayscale value; while "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel;

[0046] Chroma Noise: The digital imaging system (Image Signal Processor, ISP) uses YUV space to process pixels, where Y represents the brightness of the pixel, and UV represents the chromaticity of blue and red respectively. When the ambient illumination is insufficient, the expected values ​​(absolute values) of the Y, U, and V signals are very small. At this time, the random noise of the UV component becomes quite significant, which appears as colored noise or noise spots on the screen.

[0047] Studies have shown that the human eye has different spatial resolution capabilities for brightness and chromaticity. The resolution of chromaticity in the low-frequency band is greater than that of high-frequency bright noise. That is to say, when an image contains both brightness noise and color noise, the human eye is more sensitive to color noise. This means that the amount of low-frequency color noise has a greater impact on the overall quality of the image.

[0048] Camera image processing is a very popular technology at present. For the problem of low-frequency ground color spots (color noise) in images of medium and low-brightness scenes, the noise in the image can be reduced in the frequency domain, spatial domain, time domain, etc., thereby improving the overall image quality. However, the existing method of reducing color noise in images is likely to cause edge overflow in the denoised image, and the existing noise reduction algorithm consumes a lot of power, which is likely to affect the debugging progress of the camera's Auto White Balance (AWB).

[0049] In response to the above problems, the present disclosure provides an image denoising method, which can accurately locate the color spot area while identifying the ground, and can effectively reduce the low-frequency color noise on the ground while protecting the color of the face, thereby achieving accurate denoising of the low-frequency color noise on the ground in medium and low brightness environments, greatly reducing the impact on the overall saturation of the image in the dark environment, and will not affect the overall AWB performance of the image, thereby improving the overall image quality.

[0050] The specific implementation process of the image noise reduction method, device, electronic device, storage medium and program product provided by the present disclosure is described in detail below with reference to the accompanying drawings.

[0051] In one embodiment, Figure 1 As shown, a method for image noise reduction is provided. This embodiment mainly uses the image noise reduction method applied to an electronic device as an example, and the electronic device can be, but is not limited to, a mobile phone, a tablet computer, and other devices.

[0052] like Figure 1 As shown, the image noise reduction method may include the following steps:

[0053] Step 102: Obtain an image to be processed.

[0054] The image to be processed may be an image currently captured by the camera of the electronic device, or the image to be processed may be a frame of image in a video captured by the camera of the electronic device.

[0055] As an example, each time a camera of an electronic device is used to take a picture, an image taken or an image in a video is obtained as an image to be processed and subjected to noise reduction processing.

[0056] As another example, each time the camera of an electronic device is used to take a picture, the current ambient brightness is first detected. When it is detected that the ambient brightness is lower than a preset brightness, the captured image or image in the video is obtained as an image to be processed to perform noise reduction processing using the scheme of the present invention. When the ambient brightness is not lower than the preset brightness, the scheme of the present invention is not used for noise reduction processing.

[0057] Step 104: Perform color noise reduction processing on the image to be processed to obtain a preliminary noise-reduced image.

[0058] In this embodiment, for the acquired image to be processed, the currently commonly used color noise reduction method can be used to reduce the color noise of the image to be processed to obtain a noise-reduced image (called a preliminary noise-reduced image). This disclosure does not limit the color noise reduction method used.

[0059] Step 106: Fusing the preliminary denoised image and the image to be processed to obtain a detail restored image.

[0060] In this embodiment, for the preliminary denoised image obtained after color noise reduction, the preliminary denoised image can be fused with the image to be processed to obtain a detail restoration image. Since the image to be processed is an unprocessed original image, which contains rich detail information, the preliminary denoised image is fused with the image to be processed to restore the image details, so that the fused detail restoration image has reduced color noise compared to the original image to be processed, and has restored the image detail information compared to the preliminary denoised image.

[0061] As an example, when fusing the preliminary denoised image and the image to be processed, the pixel values ​​of the corresponding pixel points in the two images can be averaged, and the obtained average value can be used as the new pixel value of the pixel point. For example, for the pixel point in the first row and first column of the preliminary denoised image, the pixel value of the pixel point is averaged with the pixel value of the first row and first column of the image to be processed, and then the pixel value of the pixel point in the preliminary denoised image is updated to the obtained average value. In this fusion method, each pixel point in the preliminary denoised image is traversed in turn, and after traversing all the pixel points, a detailed restored image is obtained.

[0062] As an example, the weight corresponding to the image to be processed and the weight corresponding to the preliminary denoised image during fusion can be pre-set, and the image to be processed and the preliminary denoised image are fused by weighted summation to obtain a detailed restored image. It can be understood that when fusing two images, the pixel values ​​of two pixel points at corresponding positions in the two images are weighted summed to obtain a new pixel value at the corresponding position as the fusion result.

[0063] Step 108: Generate a noise reduction enhanced image based on the detail restoration image and the color noise region image of the preliminary noise reduction image.

[0064] Since there is a lot of color noise in the image to be processed, after the image to be processed is fused with the preliminary denoised image, although the detail information in the image is restored, the color noise is also introduced. In order to obtain a better noise reduction effect and achieve accurate noise reduction, in this embodiment, after obtaining the detail restored image, the detail restored image can be fused with the color noise area image of the preliminary denoised image to obtain a noise reduction enhanced image, so as to enhance the color noise reduction effect in the color noise area and improve the noise reduction accuracy.

[0065] As an example, a pre-trained color noise detection model can be used to perform color noise detection on the preliminary denoised image, and the color noise area can be identified from the preliminary denoised image and cropped to obtain a color noise area image, and then the color noise area image is fused with the detail restoration image to obtain a denoised and enhanced image.

[0066] As an example, a pre-trained color noise detection model can be used to perform color noise detection on the image to be processed, identify the color noise area in the image to be processed, and based on the identified color noise area, crop the color noise area image from the preliminary denoised image, and then fuse the color noise area image with the detail restoration image to obtain a denoised and enhanced image.

[0067] It should be noted that, in this embodiment, for the training of the color noise detection model, a large number of images containing color noise and not containing color noise can be collected and marked whether they contain color noise and the location of the color noise to obtain a large number of training samples, and then the training samples are used to train commonly used target detection models (such as regional convolutional neural networks, fast regional convolutional neural networks, etc.) to obtain the color noise detection model. The present disclosure does not limit the network structure and iteration method of the color noise detection model.

[0068] It can be understood that when the color noise area image and the detail restoration image are fused to obtain the noise reduction and enhanced image, each pixel in the color noise area image is fused with the pixel at the corresponding position in the detail restoration image, and each pixel in the color noise area image retains its position information in the preliminary noise reduction image, that is, the position information in the image to be processed. For example, the pixel in the first row and first column of the color noise area image is the pixel in the 10th row and 15th column of the preliminary noise reduction image, then the position information of the pixel in the first row and first column of the color noise area image is still (10,15), and according to the position information, the pixel in the 10th row and 15th column can be found from the detail restoration image to be fused with it.

[0069] Step 110: determining a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed.

[0070] In this embodiment, after obtaining the denoised enhanced image, the target denoised image corresponding to the image to be processed can be determined in combination with the image to be processed. The image to be processed is used to detect whether it contains a face, and different methods are used to determine the target denoised image for the two situations where the image to be processed contains a face or does not contain a face.

[0071] In an optional embodiment of the present disclosure, face detection can be performed on the image to be processed based on a pre-trained face detection model to obtain a face detection result. When the face detection result indicates that there is no face in the image to be processed, the denoised and enhanced image is determined as the target denoised image corresponding to the image to be processed; when the face detection result indicates that there is a face in the image to be processed, a face area image is determined from the image to be processed based on the face detection result, and then the denoised and enhanced image and the face area image are fused to obtain the target denoised image corresponding to the image to be processed.

[0072] As an example, the face detection result output by the face detection model may include a first probability that the image to be processed contains a face, a second probability that the image to be processed does not contain a face, and the position information of the detection frame corresponding to the face area. If the first probability is greater than the second probability, it is determined that the image to be processed contains a face, and the position information includes the coordinates of the four corner points of the detection frame; if the first probability is not greater than the second probability, it is considered that the image to be processed does not contain a face, and the position information is empty. Thus, based on the face recognition result, the electronic device can determine whether the image to be processed contains a face, and if it contains a face, it can crop the face area image from the image to be processed based on the position information of the detection frame corresponding to the face area in the face detection result, and then fuse the face area image with the noise reduction and enhancement image to obtain the target noise reduction image. If the electronic device determines that the image to be processed does not contain a face, it directly determines the noise reduction and enhancement image as the target noise reduction image.

[0073] It can be understood that when the face area image is fused with the denoising and enhanced image to obtain the target denoised image, each pixel in the face area image is fused with the pixel at the corresponding position in the denoising and enhanced image, and each pixel in the face area image retains its position information in the image to be processed. For example, if the pixel in the first row and first column of the face area image is the pixel in the 6th row and 13th column of the image to be processed, then the position information of the pixel in the first row and first column of the face area image is still (6,13). Based on this position information, the pixel in the 6th row and 13th column can be found from the denoising and enhanced image to be fused with it.

[0074] As an example, when fusing a face region image and a noise reduction enhanced image, the fusion can be performed by averaging the pixel values ​​of the corresponding pixel points in the two images, and the obtained average is used as the new pixel value of the corresponding pixel point.

[0075] As an example, when fusing a face area image and a noise reduction enhanced image, the weights corresponding to the two images can be obtained, and the new pixel values ​​of the pixels at the corresponding positions in the two images are obtained by taking the weighted sum of the pixel values.

[0076] As an example, in order to retain the color of the face area to the greatest extent, when fusing the face area image and the denoising and enhanced image, the pixel values ​​of the pixels in the face area in the denoising and enhanced image can be replaced with the pixel values ​​of the pixels at the corresponding positions in the face area image, that is, the face area in the denoising and enhanced image is restored to the face area in the image to be processed.

[0077] It should be noted that, in this embodiment, for the training of the face detection model, a large number of images containing faces and not containing faces can be collected and annotated to determine whether faces are contained and the positions of faces, so as to obtain a large number of training samples, and then the training samples are used to train the commonly used target detection models (such as regional convolutional neural networks, fast regional convolutional neural networks, etc.) to obtain the face detection model. The present disclosure does not limit the network structure and iteration method of the face detection model. Optionally, in order to reduce the number of models, a target detection model can be trained using the training samples obtained by collecting and annotating images containing faces and / or color noise, and a face-color noise detection model that can simultaneously detect the face area and the color noise area in the image can be obtained. The acquired image to be processed is input into the model, and it can be identified whether the image to be processed contains color noise and faces.

[0078] In this embodiment, by identifying whether the image to be processed contains a face, different methods are used to determine the target denoised image according to whether it contains a face. When a face is contained, the denoised enhanced image is fused with the face area image of the image to be processed to obtain the target denoised image. In this way, the face protection function during color noise reduction is implemented, the face discoloration problem caused by color noise reduction is effectively avoided, and the quality of the image containing the face is improved.

[0079] The image denoising method provided by the embodiment of the present disclosure is as follows: acquiring an image to be processed; performing color noise reduction processing on the image to be processed to obtain a preliminary denoised image; fusing the preliminary denoised image and the image to be processed to obtain a detail restored image; generating a denoised enhanced image based on the detail restored image and the color noise region image of the preliminary denoised image; and determining a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed. The scheme of the present disclosure is adopted, by performing color noise reduction processing on the image to be processed and then fusing it with the image to be processed, the detail information in the image can be restored, and then fusing it with the color noise region image of the image after the color noise reduction processing, the color noise reduction effect of the color noise region can be enhanced, and finally the target denoised image obtained can accurately remove the color noise in the image while avoiding the edge overflow problem, thereby improving the image quality.

[0080] In an optional embodiment of the present disclosure, Figure 2 As shown, in Figure 1 Based on the illustrated embodiment, step 108 may include the following sub-steps:

[0081] Step 202: Perform color noise detection on the target image using a pre-trained color noise detection model to obtain a color noise detection result, wherein the target image is an image to be processed or a preliminary denoised image.

[0082] Among them, the color noise detection model can be obtained in advance by training a currently commonly used neural network model.

[0083] As an example, a color noise detection model can be obtained based on Fast Region-based Convolutional Neural Networks (Fast R-CNN) training. The Fast R-CNN algorithm mainly includes the following four parts:

[0084] (1) Convolutional layer, i.e., feature extraction network, is used to extract image features. Specifically, the input image is passed through a series of convolutional layers, activation functions, and pooling layers for feature extraction to obtain the corresponding feature map. The purpose of this step is to extract key information from the image and provide a basis for subsequent candidate region generation and classification;

[0085] (2) Region Proposal Network (RPN), used to generate candidate boxes. RPN is the core part of the Faster R-CNN algorithm, which is responsible for generating candidate regions. RPN generates a series of candidate boxes by sliding a small window on the last layer of the feature map of the full convolutional network, and performs binary classification (foreground or background) and bounding box regression on each candidate box to obtain accurate candidate regions. This step greatly reduces the number of candidate regions and improves the running speed of the algorithm;

[0086] (3) Region of Interest Pooling (ROIPooling): Its function is to obtain the feature region corresponding to the region of interest in the input image on the feature map through the coordinate projection method, and perform maximum pooling on the region, so as to obtain the features of the region of interest and unify the feature size for subsequent classification and regression operations. Through ROIPooling, the algorithm can process input images and candidate regions of any size, enhancing the robustness of the algorithm;

[0087] (4) Classification and regression layer: The fixed-size feature map output by the ROIPooling layer is input into the fully connected layer for classification to obtain the category to which each candidate region belongs. At the same time, the position of the candidate region can be fine-tuned through bounding box regression to further improve the accuracy of target detection.

[0088] In this embodiment, the image to be processed or the preliminary denoised image can be input into the trained color noise detection model to obtain the color noise detection result. Exemplarily, the color noise detection result includes the probability of the image to be processed or the preliminary denoised image containing the color noise area and the probability of not containing the color noise area, as well as the position information of the detection frame corresponding to the color noise area. When the probability of containing the color noise area is greater than the probability of not containing the color noise area, it is determined that the image to be processed or the preliminary denoised image contains the color noise area, and the position information of the detection frame corresponding to the color noise area is obtained; otherwise, it is determined that the image to be processed or the preliminary denoised image does not contain the color noise area, and the position information of the detection frame corresponding to the color noise area is empty.

[0089] Step 204: determine a color noise region image from the preliminary noise reduction image based on the color noise detection result.

[0090] In this embodiment, after obtaining the color noise detection result output by the color noise detection model, it can be determined whether there is a color noise area in the image based on the color noise detection result. When there is a color noise area, the color noise area image is determined from the preliminary denoised image based on the position information of the detection box corresponding to the color noise area in the color noise detection result. It can be understood that the color noise area image is the color noise area after the color noise reduction processing.

[0091] When the target image is the image to be processed, since the image to be processed is the original image that has not been subjected to noise reduction processing, the image to be processed is input into the color noise detection module for color noise detection, and the obtained color noise detection result is more accurate, and the color noise area image determined from the preliminary noise reduction image based on the color noise detection result is more accurate and has a larger range. When the target image is a preliminary noise reduction image, the color noise area located in the obtained color noise detection result is smaller or the same as the color noise area in the image to be processed, which helps to reduce the processing amount of image fusion. Whether to select the image to be processed for color noise detection or the preliminary noise reduction image for color noise detection can be selected according to needs. When more attention is paid to noise reduction accuracy, the image to be processed can be selected for color noise detection. When more attention is paid to processing efficiency, the preliminary noise reduction image can be selected for color noise detection. Optionally, when the color noise area is not detected by the preliminary noise reduction image, the image to be processed can be further input into the color noise detection model for color noise detection to obtain a color noise area image.

[0092] Step 206: Fusing the detail restored image and the color noise region image to obtain a noise reduction and enhanced image.

[0093] In this embodiment, after the color noise region image is obtained, the color noise region image and the detail restoration image may be fused to obtain a noise reduction and enhanced image.

[0094] It can be understood that the fusion of the color noise area image and the detail restoration image is to fuse each pixel in the color noise area image with the pixel at the corresponding position in the detail restoration image, and each pixel in the color noise area image retains its position information in the image to be processed. For example, the pixel in the first row and first column of the color noise area image is the pixel in the 23rd row and 13th column of the image to be processed, then the position information of the pixel in the first row and first column of the color noise area image is still (23,13), and according to the position information, the pixel in the 23rd row and 13th column can be found in the detail restoration image to be fused with it.

[0095] As an example, when fusing a color noise area image and a detail restoration image, the fusion can be performed by averaging the pixel values ​​of the pixel points at corresponding positions in the two images, and the obtained average is used as the new pixel value of the pixel point at the corresponding position.

[0096] As an example, the corresponding fusion method can be selected according to the user's actual image quality requirements to fuse the color noise area image and the detail restoration image. Specifically, the image quality requirement parameters set by the user can be obtained. For example, the user can set the image quality requirement parameters required by the user through the Camera settings page. The settable image quality requirement parameters include a first requirement parameter and a second requirement parameter, wherein the first requirement parameter is used to indicate that the user's demand for image details has a higher priority than the demand for noise reduction effects, that is, the user pays more attention to the detailed information of the image; the second requirement parameter is used to indicate that the user's demand for image details does not have a higher priority than the demand for noise reduction effects, that is, the user is more concerned about the noise reduction effect of the image and hopes that the color noise in the image is as small as possible.

[0097] Optionally, when the image quality requirement parameter set by the user is a first requirement parameter, a pre-set first weight and a second weight are further obtained, wherein the first weight is greater than the second weight. Then, based on the first weight and the second weight, a weighted sum of the pixel values ​​of each pixel point at a corresponding position in the detail restoration image and the color noise area image is calculated to obtain a noise reduction and enhanced image. When performing the weighted summation, the first weight is used to weight the detail restoration image, and the second weight is used to weight the color noise area image. The detail restoration image has a higher weight, so that the obtained noise reduction and enhanced image can retain more detail information, thereby obtaining an image that meets the user's image quality requirements.

[0098] Optionally, when the image quality requirement parameter set by the user is the second requirement parameter, the pixel value of the pixel point corresponding to the color noise region image in the detail restoration image can be updated to the pixel value of the pixel point at the corresponding position in the color noise region image. For example, if the pixel point at the 34th row and 15th column in the detail restoration image corresponds to the pixel point at the 1st row and 2nd column in the color noise region image, the pixel value of the pixel point at the 34th row and 15th column in the detail restoration image is updated to the pixel value of the pixel point at the 1st row and 2nd column in the color noise region image, thereby replacing the color noise region in the detail restoration image with the color noise region image to obtain a better noise reduction effect.

[0099] The image denoising method of the disclosed embodiment performs color noise detection on a target image by using a pre-trained color noise detection model to obtain a color noise detection result, wherein the target image is an image to be processed or a preliminary denoising image, and then determines a color noise area image from the preliminary denoising image based on the color noise detection result, and then fuses the detail restoration image and the color noise area image to obtain a denoising enhanced image, thereby effectively removing the color noise introduced into the detail restoration image due to the fusion of the image to be processed, and enhancing the color noise reduction effect on the color noise area in the detail restoration image.

[0100] It should be understood that although Figure 1-2The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0101] In one embodiment, Figure 3 As shown, an image denoising device is provided, the image denoising device 30 comprises an image acquisition module 302, a denoising module 304, a first fusion module 306, a second fusion module 308 and an image determination module 310. Among them:

[0102] An image acquisition module 302 is used to acquire an image to be processed;

[0103] A noise reduction module 304 is used to perform color noise reduction processing on the image to be processed to obtain a preliminary noise reduction image;

[0104] A first fusion module 306 is used to fuse the preliminary denoised image and the image to be processed to obtain a detail restored image;

[0105] A second fusion module 308, configured to generate a noise reduction enhanced image based on the detail restoration image and the color noise region image of the preliminary noise reduction image;

[0106] The image determination module 310 is used to determine a target denoised image corresponding to the image to be processed based on the denoised and enhanced image and the image to be processed.

[0107] In one embodiment, the second fusion module 308 includes:

[0108] A color noise detection unit, used to perform color noise detection on a target image using a pre-trained color noise detection model to obtain a color noise detection result, wherein the target image is the image to be processed or the preliminary denoised image;

[0109] a color noise image determining unit, configured to determine the color noise region image from the preliminary noise reduction image based on the color noise detection result;

[0110] The fusion unit is used to fuse the detail restoration image and the color noise area image to obtain the noise reduction and enhanced image.

[0111] In one embodiment, the fusion unit is further configured to:

[0112] Get the image quality requirement parameters set by the user;

[0113] When the image quality requirement parameter is a first requirement parameter, obtaining a first weight and a second weight, wherein the first requirement parameter is used to indicate that the user's requirement for image details has a higher priority than the requirement for noise reduction effect, and the first weight is greater than the second weight;

[0114] Based on the first weight and the second weight, a weighted sum of pixel values ​​of each pixel point at a corresponding position in the detail restoration image and the color noise area image is calculated to obtain the noise reduction and enhanced image.

[0115] Furthermore, in one embodiment, the fusion unit is further configured to:

[0116] When the image quality requirement is the second requirement parameter, updating the pixel values ​​of the pixels corresponding to the color noise region image in the detail restoration image to the pixel values ​​of the pixels at the corresponding positions in the color noise region image;

[0117] The second requirement parameter is used to indicate that the priority of the user's requirement for image details is not higher than the priority of the user's requirement for noise reduction effect.

[0118] In one embodiment, the image determination module 310 is further configured to:

[0119] Performing face detection on the image to be processed based on a pre-trained face detection model to obtain a face detection result;

[0120] When the face detection result indicates that there is no face in the image to be processed, the denoised and enhanced image is determined as a target denoised image corresponding to the image to be processed.

[0121] In one embodiment, the image determination module 310 is further configured to:

[0122] When the face detection result indicates that a face exists in the image to be processed, determining a face region image from the image to be processed based on the face detection result;

[0123] The denoised and enhanced image is fused with the face region image to obtain a target denoised image corresponding to the image to be processed.

[0124] The image denoising device provided by the embodiment of the present disclosure obtains an image to be processed; performs color noise reduction processing on the image to be processed to obtain a preliminary denoised image; fuses the preliminary denoised image and the image to be processed to obtain a detail restoration image; generates a denoised enhanced image based on the detail restoration image and the color noise region image of the preliminary denoised image; and determines a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed. The scheme of the present disclosure can restore the detail information in the image by performing color noise reduction processing on the image to be processed and then fusing it with the image to be processed, and then fusing it with the color noise region image of the image after the color noise reduction processing to enhance the color noise reduction effect of the color noise region, so that the target denoised image can accurately remove the color noise in the image while avoiding the edge overflow problem and improving the image quality.

[0125] For the specific definition of the image denoising device, please refer to the definition of the image denoising method above, which will not be repeated here. Each module in the above-mentioned image denoising device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0126] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, an image noise reduction method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0127] Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] In one embodiment, the image noise reduction device provided by the present disclosure can be implemented in the form of a computer program. The computer program can be Figure 4 The memory of the electronic device may store various program modules constituting the image noise reduction device, for example, Figure 3 The image acquisition module, noise reduction module, first fusion module, second fusion module and image determination module shown in the figure. The computer program composed of various program modules enables the processor to execute the steps of the image noise reduction method of each embodiment of the present disclosure described in this specification.

[0129] For example, Figure 4 The electronic device shown can be Figure 3 The image acquisition module in the image denoising device shown executes the step of acquiring the image to be processed. The electronic device may perform the step of reducing color noise on the image to be processed through the denoising module to obtain a preliminary denoised image. The electronic device may perform the step of fusing the preliminary denoised image and the image to be processed through the first fusion module to obtain a detail restored image. The electronic device may perform the step of generating a denoised enhanced image based on the detail restored image and the color noise area image of the preliminary denoised image through the second fusion module. The electronic device may perform the step of determining the target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed through the image determination module.

[0130] In one embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0131] In one embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0132] In one embodiment, a computer program product is further provided, wherein the computer program product is used to execute the steps in the above method embodiments.

[0133] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static random access memory (SRAM) and dynamic random access memory (DRAM).

[0134] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above-mentioned embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the patent of the present disclosure shall be subject to the attached claims.

Claims

1. An image denoising method, characterized in that: The method comprises: Get the image to be processed; Performing color noise reduction processing on the image to be processed to obtain a preliminary noise-reduced image; Fusing the preliminary denoised image and the image to be processed to obtain a detail restored image; Generate a noise reduction enhanced image based on the detail restoration image and the color noise region image of the preliminary noise reduction image; Based on the denoised and enhanced image and the image to be processed, a target denoised image corresponding to the image to be processed is determined.

2. The method according to claim 1, characterized in that The step of generating a noise reduction enhanced image based on the detail restoration image and the color noise region image of the preliminary noise reduction image comprises: Performing color noise detection on a target image using a pre-trained color noise detection model to obtain a color noise detection result, wherein the target image is the image to be processed or the preliminary denoised image; Determining the color noise region image from the preliminary noise reduction image based on the color noise detection result; The detail restoration image and the color noise area image are fused to obtain the noise reduction and enhanced image.

3. The method according to claim 2, characterized in that The step of fusing the detail restored image and the color noise region image to obtain the noise reduction enhanced image includes: Get the image quality requirement parameters set by the user; When the image quality requirement parameter is a first requirement parameter, obtaining a first weight and a second weight, wherein the first requirement parameter is used to indicate that the user's requirement for image details has a higher priority than the requirement for noise reduction effect, and the first weight is greater than the second weight; Based on the first weight and the second weight, a weighted sum of pixel values ​​of each pixel point at a corresponding position in the detail restoration image and the color noise area image is calculated to obtain the noise reduction and enhanced image.

4. The method according to claim 3, characterized in that The fusing of the detail restored image and the color noise region image to obtain the noise reduction enhanced image further includes: When the image quality requirement is the second requirement parameter, updating the pixel values ​​of the pixels corresponding to the color noise region image in the detail restoration image to the pixel values ​​of the pixels at the corresponding positions in the color noise region image; The second requirement parameter is used to indicate that the priority of the user's requirement for image details is not higher than the priority of the user's requirement for noise reduction effect.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed includes: Performing face detection on the image to be processed based on a pre-trained face detection model to obtain a face detection result; When the face detection result indicates that there is no face in the image to be processed, the denoised and enhanced image is determined as a target denoised image corresponding to the image to be processed.

6. The method according to claim 5, characterized in that The step of determining a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed further comprises: When the face detection result indicates that a face exists in the image to be processed, determining a face region image from the image to be processed based on the face detection result; The denoised and enhanced image is fused with the face region image to obtain a target denoised image corresponding to the image to be processed.

7. An image noise reduction device, characterized in that: include: An image acquisition module, used for acquiring an image to be processed; A noise reduction module, used for performing color noise reduction processing on the image to be processed to obtain a preliminary noise reduction image; A first fusion module, used for fusing the preliminary denoised image and the image to be processed to obtain a detail restored image; A second fusion module, configured to generate a noise reduction enhanced image based on the detail restoration image and the color noise region image of the preliminary noise reduction image; The image determination module is used to determine a target denoised image corresponding to the image to be processed based on the denoised enhanced image and the image to be processed.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the image noise reduction method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image noise reduction method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product is used to implement the steps of the image noise reduction method according to any one of claims 1 to 6.

Images