Image noise reduction method, device, equipment, and storage medium

Abstract

The embodiment of the present invention discloses a method and apparatus, a device and a storage medium for image denoising. The method comprises: performing edge detection on a color image to obtain a preprocessed image; acquiring a depth image of the same scene as the color image; and performing a first denoising process on the preprocessed image according to the depth image to obtain a first image.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing, and relate to but are not limited to a method and apparatus, device, and storage medium for image noise reduction. Background Art

[0002] With the continuous development of digital multimedia technology, people are increasingly pursuing high definition and high resolution of images. However, images are often affected by noise interference from imaging devices and the external environment during digitization and transmission, thereby reducing the clarity of the image. The process of reducing noise in an image is called image denoising. The existing common noise reduction methods mainly filter signals through various filters to achieve partition noise reduction, but this noise reduction method cannot avoid the phenomenon of blurred object boundaries during noise reduction. In the related art, the use of multi-frame noise reduction technology can ensure the clarity of details or contours, but it is necessary to continuously capture multiple photos for processing after shooting, and the shutter will be locked during shooting. The processing time is long, which affects the effect of continuous shooting. Summary of the invention

[0003] In view of this, embodiments of the present invention provide a method, apparatus, device, and storage medium for image noise reduction to solve at least one problem existing in the prior art.

[0004] An embodiment of the present invention provides a method for image noise reduction, the method comprising:

[0005] Perform edge detection on the color image to obtain a preprocessed image;

[0006] Acquire a depth image having the same scene as the color image;

[0007] A first denoising process is performed on the pre-processed image according to the depth image to obtain a first image.

[0008] The embodiment of the present invention further provides a device for image noise reduction, the device comprising:

[0009] A detection unit configured to perform edge detection on the color image to obtain a preprocessed image;

[0010] An acquisition unit configured to acquire a depth image having the same scene as the color image;

[0011] The first denoising unit is configured to perform a first denoising process on the pre-processed image according to the depth image to obtain a first image.

[0012] An embodiment of the present invention further provides an image noise reduction device, the terminal comprising: a memory and a processor, the memory storing a computer program executable on the processor, and the processor implementing the above-mentioned image noise reduction method when executing the computer program.

[0013] An embodiment of the present invention further provides a computer-readable storage medium, in which computer-executable instructions are stored. The computer-executable instructions are configured to execute the above-mentioned image denoising method.

[0014] The technical solution provided by the embodiment of the present invention can retain most edge information and details when performing noise reduction processing on an image, thereby preventing blurring of edges and details. In addition, it is not necessary to obtain multiple frames of images, thereby improving processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a method for image noise reduction according to an embodiment of the present invention;

[0016] Figure 2 A schematic flow chart of another method for image noise reduction according to an embodiment of the present invention;

[0017] Figure 3A A schematic diagram of another method flow for image noise reduction according to an embodiment of the present invention;

[0018] Figure 3B A schematic diagram of another method flow for image noise reduction according to an embodiment of the present invention;

[0019] Figure 3C is a schematic diagram of an original image in an embodiment of the present invention;

[0020] Figure 3D is a schematic diagram of an image that has been sharpened in an embodiment of the present invention;

[0021] Figure 3E is a depth map corresponding to the image in an embodiment of the present invention;

[0022] Figure 3F Schematic diagram of an image processed by edge detection in an embodiment of the present invention;

[0023] Figure 3G A schematic diagram of performing noise reduction processing on an image in an embodiment of the present invention;

[0024] Figure 3H A schematic diagram of performing noise reduction processing on image boundary points in an embodiment of the present invention;

[0025] Fig. 3I A schematic diagram of performing noise reduction processing on points near image boundary points in an embodiment of the present invention;

[0026] Figure 4 Schematic diagram of the structure of the device for image noise reduction in an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of a hardware entity of an image denoising device in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to better understand the embodiments of the present invention, the following terms are explained as follows:

[0029] Image noise is a random variation in brightness or color information in an image (not present in the photographed object itself), usually a manifestation of electronic noise. It is generally generated by the sensors and circuits of scanners or digital cameras, or by film grain or the inevitable shot noise in ideal photodetectors. Image noise is an undesirable byproduct of the image capture process, which brings errors and extra information to the image.

[0030] Noise: Pixels of image noise.

[0031] Image Denoising: In the process of digitization and transmission, digital images in reality are often affected by the interference of imaging equipment and external environmental noise, which are called noisy images or noisy images. The process of reducing noise in digital images is called image denoising.

[0032] Texture features: Texture is a visual feature that reflects homogeneous phenomena in an image. It reflects the slowly changing or periodically changing surface structure and organization properties of an object. Texture has three major characteristics: a certain local sequence that is repeated continuously, non-random arrangement, and a roughly uniform unity within the texture area.

[0033] Image foreground and background: The foreground is the person or object near the front or in front of the subject in the shot; the background is the person or object near the back or behind the subject in the shot. The foreground and background are the basic levels of photographic composition, which can make the picture rich in layers and depth.

[0034] Multi-frame noise reduction technology: Multi-frame noise reduction means that in night scenes or low-light environments, the camera will collect multiple / multi-frame photos or images from the time the shutter is pressed to the time the image is formed, find different pixels with noise properties at different frame rates, and obtain a cleaner and purer night scene or low-light photo through weighted synthesis.

[0035] The edge image is the image obtained by edge extraction of the original image. The most basic feature of an image is the edge. The edge is where the regional attributes suddenly change, where the uncertainty in the image is the greatest, and where the image information is most concentrated. The edge of the image contains rich information.

[0036] Edge detection technology is to extract edge pixels in an image by calculation to obtain an edge image.

[0037] A depth map is an image or image channel that contains information about the distance to the surface of scene objects. A depth map is similar to a grayscale image, except that each pixel value is the actual distance from the sensor to the object.

[0038] Connected region: If a simple closed curve is drawn arbitrarily in a region on the complex plane, and the interior of the simple closed curve always belongs to this region, then this region is called a simply connected region.

[0039] Image sharpening is a process that compensates for the contour of an image, enhances the edge and grayscale jump of the image, and makes the image clear. It is divided into two categories: spatial domain processing and frequency domain processing. Image sharpening is to highlight the edge, contour, or characteristics of certain linear target elements of the object on the image. This filtering method increases the contrast between the edge of the object and the surrounding pixels, so it is also called edge enhancement.

[0040] Dual camera: Use dual cameras to shoot. The dual cameras can measure distance, shoot, and obtain depth maps through calculation.

[0041] Grayscale value: Grayscale uses black tones to represent objects, that is, black is used as the base color and different saturation levels of black are used to display images. Each grayscale object has a brightness value from 0% (white) to 100% (black).

[0042] In the current image processing field, noise reduction is usually performed on the captured images. However, the noise reduction effect will be significantly reduced at night or in low-light environments. Generally speaking, the processing of noise will affect the display of details. The relevant technology still cannot meet the requirements of noise reduction in low-light conditions while ensuring the clarity of details. In particular, the processing of the boundaries between objects often leads to blurred boundaries while reducing noise.

[0043] In some related technologies, a partition noise reduction method is used for image noise reduction. The basic principle is to draw concentric circles with the center of the picture as the center. The closer the pixel is to the center, the smaller the noise reduction coefficient is, and the farther it is from the center, the larger the noise reduction coefficient is. The noise reduction coefficient here can be understood as the degree of noise reduction. The smaller the noise reduction coefficient is, the weaker the degree of noise reduction is. According to the imaging principle of a general camera, the captured image will appear bright in the center and dark around. At this time, the noise in the darker area may be greater than the brighter area in the center. Therefore, the above-mentioned partition noise reduction method can effectively reduce the noise of the entire image. However, this method still cannot avoid the problem of blurred boundaries between objects when reducing noise, that is, the processing effect on the edge of the image is not good.

[0044] In other related technologies, texture features are used to separate the foreground and background of a captured image during image denoising, and then the foreground and background are subjected to denoising respectively, and then synthesized after processing. This method requires training and distinguishing the texture features of the foreground and background, and the processing is relatively complicated. Although this method retains the clear boundary between the foreground and background, the contours between objects in the foreground and background will still be affected by denoising and become blurred.

[0045] In other related technologies, multi-frame noise reduction technology can also be used to reduce the noise of images. This method synthesizes the image data of multiple frames to reduce noise while ensuring the clarity of details and boundaries. However, in actual application, this method needs to continuously capture multiple images for processing during shooting. When shooting, the camera shutter will be locked, and the processing time is long. It is not suitable for continuous dynamic shooting, which affects the shooting effect.

[0046] The following is explained in conjunction with embodiments:

[0047] Embodiment 1

[0048] The embodiment of the present invention provides a method for image noise reduction, which is used in an image processing device. When the device executes the method, it can still retain most edge information and detail information after noise reduction processing, and does not need to obtain multiple frames of images, and will not lock the shutter, thereby improving the user experience. The function implemented by the method can be implemented by calling program code by a processor in the image processing device. Of course, the program code can be stored in a computer storage medium. It can be seen that the image processing device at least includes a processor and a storage medium.

[0049] Figure 1 FIG. 1 is a flow chart of a method for image noise reduction according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0050] Step S101, performing edge detection on the color image to obtain a preprocessed image;

[0051] The color image here can be an image captured directly by the camera, or it can be other images that need to be processed. It can be a grayscale image containing only grayscale, or it can be a color image. Edge detection is to extract edge pixels from an image by calculation to obtain an edge image. The preprocessed image here refers to the edge image obtained after edge detection. Since edge detection is achieved by detecting pixels with high-frequency characteristics, there are noise points in this preprocessed image, and the noise points will be distributed in the area near the edge and in the connected area of ​​non-edges. Images acquired under low light conditions may have more noise points.

[0052] Step S102, obtaining a depth image having the same scene as the color image;

[0053] Here, a depth image with the same scene as the color image is obtained by using a device that can obtain depth information, such as a dual camera function. The depth image has depth information and can distinguish areas of connected depth. In practical applications, the depth image and the color image can be collected by different devices or by the same device; they can be collected separately or at the same time; in addition, the color image itself can also be a depth image with depth information. Regardless of the method used, it is sufficient to ensure that the scene corresponding to the color image and the depth image is exactly the same.

[0054] Depth information can reflect the distance from the object in the image to the camera, and can be obtained through the ranging sensor of a dual-camera or other shooting device. Based on the depth information of the image, a depth image corresponding to the original image can be generated. In the depth image, the pixel value of each pixel reflects the distance from the object in the scene to the camera. Through the depth image, different objects and the boundaries between each surface of the object and other surfaces can be better distinguished.

[0055] Step S103: performing a first noise reduction process on the pre-processed image according to the depth image to obtain a first image;

[0056] By combining the preprocessed image after edge extraction with the depth image, the depth connected area in the preprocessed image can be found through the depth image. The high-frequency points in the depth connected area are noise points; and the part where the depth changes and the preprocessed image overlap is the edge. The high-frequency noise points are removed through the first denoising process, and the first image with clear edges can be obtained.

[0057] In other embodiments, performing a first denoising process on the pre-processed image according to the depth image to obtain a first image includes:

[0058] According to the connected areas of each continuous depth in the depth image, the first denoising process is performed on the preprocessed image to obtain the first image.

[0059] Here, combined with the depth image, the connected areas of each continuous depth can be distinguished, that is, the areas surrounded by each edge. The first noise reduction process is performed on these connected areas respectively, and the fragmentary noise points in each connected area can be removed to ensure that most of the pixels with high-frequency information in the first image are boundary points.

[0060] In other embodiments, performing the first denoising process on the preprocessed image according to the connected regions of each continuous depth in the depth image to obtain the first image includes:

[0061] Determine high-frequency noise points in the connected area according to each connected depth in the depth image;

[0062] According to the first noise reduction processing, the high-frequency noise points are removed to obtain the first image.

[0063] The first noise reduction process here is to perform noise reduction on the continuous depth area mentioned above, remove isolated noise points with high-frequency characteristics, and cause as little edge blur as possible to improve the accuracy of subsequent noise reduction processing. The first noise reduction process here can be implemented through a variety of noise reduction methods, for example, taking a point near a high-frequency noise point, and modifying the grayscale value of the high-frequency noise point to the grayscale value of the selected nearby point; or, taking all points in an area near the high-frequency noise point, finding the average value of these nearby points, and modifying the grayscale value of the high-frequency noise point to the average value; or, finding the average value of the grayscale values ​​of all points in the connected area, and modifying the grayscale values ​​of all high-frequency noise points in the connected area to the average value.

[0064] In other embodiments, performing edge detection on the color image to obtain a preprocessed image includes:

[0065] Step S11, performing sharpening processing on the color image to obtain a sharpened image;

[0066] Here, the terminal device first performs sharpening processing on the color image, which makes the edge more obvious and facilitates subsequent image edge detection. Sharpening processing will amplify the details and also amplify the noise.

[0067] Step S12: performing edge detection on the sharpened image to obtain the preprocessed image.

[0068] Here, after the color image is sharpened, edge detection is performed on the sharpened image, which can more accurately detect the edge of the image, and of course, the noise points amplified by sharpening will also be detected, thus forming a pre-processed image with a large number of fragmentary noise points. These fragmentary noise points need to be subjected to noise reduction processing in step S104 of this embodiment.

[0069] Embodiment 2

[0070] The embodiment of the present invention also provides a method for image noise reduction, such as Figure 2 As shown, the method includes:

[0071] Step S201, performing edge detection on the color image to obtain a preprocessed image;

[0072] Step S202: acquiring a depth image having the same scene as the color image;

[0073] Step S203: performing a first noise reduction process on the pre-processed image according to the depth image to obtain a first image;

[0074] The above steps S201 to S203 are the same as steps S101 to S103 in the first embodiment, and are not described again here.

[0075] Step S204: extracting boundary points in the first image;

[0076] Here, the boundary points in the first image, which are used as input for subsequent processing, are extracted through the terminal device. Boundary points refer to the pixel points that constitute the edge in the image, including the boundary between objects and the pixel points on the edge of the object itself. Here, the boundary points corresponding to the edge image in step S204 can be directly extracted from the first image. In different images, the edge may be expressed differently. For example, in an image, the boundary between objects is black, which appears as a black edge line, while the edge on an object appears white due to the illumination of light; it is also possible to distinguish the edge by the color change between objects. In practical applications, the edge, that is, the pixel with high-frequency characteristics, can be identified by the brightness value, grayscale value or color mutation of the pixel.

[0077] The terminal device here can be a shooting device for acquiring the original image, such as a mobile phone, a digital camera, a video camera, etc.; it can also be a related computer device for post-processing.

[0078] Step S205, modifying the initial parameter values ​​of the boundary points in the first image to obtain a second image;

[0079] Here, the terminal device modifies the parameter values ​​of the extracted boundary points in order to achieve de-marginalization, that is, to modify the parameter values ​​of the edge pixels, such as the brightness value or grayscale value of the boundary points, to the values ​​of the pixels without high-frequency features. Because in the subsequent noise reduction process, an area is generally selected and noise reduction is performed on this area. For example, an area of ​​n×n pixels (n is a natural number greater than 1) is selected, and the parameter values ​​of the pixels in this area are averaged to achieve the effect of removing high-frequency points. Since the noise points usually have similar high-frequency characteristics to the boundary points on the edge, in the process of this noise reduction process, not only the high-frequency characteristics of the noise points will be removed, but also the high-frequency characteristics of the boundary points. In this way, while reducing the noise, it will also cause blurring of the edges.

[0080] Here, the parameter values ​​of the boundary points are modified so that they do not have high-frequency characteristics, so that when the image is subjected to noise reduction, the parameter values ​​of the boundary points will not affect the pixels near them. At the same time, the original values ​​of the boundary points are saved first and will not be affected by noise reduction. In this way, the points near the edge and the boundary points on the edge will not become blurred due to noise reduction.

[0081] Step S206: performing a second noise reduction process on the second image to obtain a third image;

[0082] Here, the terminal device performs noise reduction processing on the second image after the de-margining processing in step S102 to obtain a third image after noise reduction. Since the second image here has been de-margined, it will not be affected by high-frequency points at details such as edges during the noise reduction process. Therefore, a stronger noise reduction processing can be used in this step, such as selecting a higher noise reduction coefficient. In this way, slight noise points in the image can also be removed, achieving better noise reduction processing.

[0083] Step S207: modify the parameter values ​​of the boundary points in the third image to initial parameter values ​​to obtain a fourth image.

[0084] The image after the noise reduction process in step S206 is an image without edge and detail information. Here, it is necessary to restore the original edge and detail information in the image after the noise reduction process. The terminal device calls out the stored initial parameter values ​​of the boundary points, modifies the parameter values ​​of the boundary points in the image after the noise reduction process back to the initial parameter values, and obtains a fourth image with edge and detail information and after the noise reduction process is completed.

[0085] In other embodiments, modifying the initial parameter values ​​of the boundary points in the first image to obtain the second image includes:

[0086] Step S21, determining a reference pixel point for the boundary point;

[0087] Step S22, determining the parameter value of the reference pixel;

[0088] Step S23: modify the initial parameter values ​​of the boundary points in the first image to the parameter values ​​of the reference pixel points to obtain a second image.

[0089] In this embodiment, the modification of the initial parameter value of the boundary point is based on the reference point. That is, when the initial parameter value of the boundary point is modified, the parameter value of the selected reference point replaces the initial parameter value of the boundary point. The reference point here is a point that does not have high-frequency characteristics relative to other points in the image, and can be a point near each boundary point in the first image, or a uniformly set point.

[0090] In other embodiments, determining a reference pixel point for the boundary point includes: determining a pixel point within a preset distance range from the boundary point as a reference pixel point;

[0091] The determining the parameter value of the reference pixel includes: if the number of the reference pixel points is greater than 1, determining the average brightness value of the reference pixel points as the parameter value of the reference pixel points.

[0092] Here, the points near the boundary point are used as reference pixels. The number of reference pixels can be one or more. If the number of pixels within the preset distance range is greater than 1, the average brightness value of these pixels can be used as the parameter value of the reference point, and the initial parameter value of the corresponding boundary point can be modified to the parameter value of this reference point.

[0093] In this embodiment, the brightness value is used as the parameter value. In practical applications, various characteristic values ​​of image pixels such as grayscale value and color value can also be used as parameter values, which can be determined according to the characteristics of the actual image. In other embodiments, any of the above methods further includes the following steps:

[0094] Step S24, performing a third noise reduction process on the boundary points to obtain a fifth image; wherein the noise reduction degree of the third noise reduction process is weaker than the noise reduction degree of the second noise reduction process.

[0095] Since the boundary points on the edge are points with high-frequency characteristics, if the noise reduction degree of the boundary points is too strong, the edges will be blurred. Therefore, when performing noise reduction on the boundary points here, a weaker noise reduction process needs to be used. For example, the noise reduction coefficient is adjusted to half of the noise reduction coefficient in the second noise reduction process to remove the noise points in the boundary points.

[0096] Through the steps in the above method, the noise of the entire image can be reduced, and as many details and edge information as possible can be retained.

[0097] Embodiment 3

[0098] The embodiment of the present invention provides a method for image noise reduction, such as Figure 3A As shown, the method includes:

[0099] Step S301: Use edge detection technology to process a low-illuminance and high-noise image to obtain an edge image after high-frequency filtering.

[0100] Here we are dealing with low-light and high-noise images, that is, images taken in low light conditions with a lot of noise. After high-frequency filtering using edge detection technology, the grayscale value mutation points in the image can be screened out, and the edge image is obtained. The noise points are also the grayscale value mutation points, so this edge image contains not only the edge pixels, but also the noise points in the connected area.

[0101] Step S302: Combine the depth map and remove independent high noise points in the connected area at the same depth to improve the accuracy of the object boundary and obtain a boundary map;

[0102] Here, combined with the depth map, the depth of each connected area can be distinguished, and the noise points in each connected area of ​​the same depth can be processed separately. Independent high noise points are high-frequency noise points that are not continuous, that is, relatively obvious noise points.

[0103] Step S303: Based on the boundary map, a low noise reduction coefficient is selected to process the boundary points;

[0104] Here, only the boundary points in the boundary map are subjected to noise reduction processing, and a low noise reduction coefficient is selected, that is, a weaker noise reduction processing is performed. The purpose is to remove the noise points in the boundary, make the boundary clearer, and retain more details.

[0105] Step S304, using a compensation elimination method to reduce noise on points near the boundary points;

[0106] The compensation elimination method here means replacing the boundary points with points near the boundary points, that is, the points near the boundary points for noise reduction. After the replacement is completed, the points near the boundary points are subjected to noise reduction. In this way, when the points near the boundary points are subjected to noise reduction, the influence caused by the boundary points can be avoided, and the blurring near the boundary points can be avoided.

[0107] Step S305: After processing the points near the boundary point, replace the boundary point back to the boundary point in the boundary map;

[0108] Step S306: For the connected areas surrounded by the same boundary, a larger noise reduction coefficient is selected to perform noise reduction processing.

[0109] Since the connected area here will not be disturbed by the boundary, a larger noise reduction coefficient is selected, that is, a stronger noise reduction processing is performed, and the low-frequency noise points in the connected area, that is, the weaker and less obvious noise points, are also removed to ensure the noise reduction effect.

[0110] In other embodiments, the image noise reduction method provided is as follows: Figure 3B As shown, the method comprises the following steps:

[0111] Step S311: The terminal device uses a camera module to capture a scene, obtain an image, and transmit the image to a central processing chip.

[0112] The central processing chip here can be a CPU (central processing unit) or a GPU (graphics processing unit).

[0113] Step S312: After receiving the digital image data, the processing chip performs image sharpening processing;

[0114] Sharpening can highlight details, make them clearer, and make boundaries more obvious, which is convenient for subsequent edge detection. This can be achieved through the sharpening filtering method in the relevant technology.

[0115] Step S313, using edge detection technology on the sharpened image to obtain an edge image;

[0116] Sharpening will amplify the noise in the image, and this problem will be more prominent in low illumination conditions. Figure 3C is the original image, Figure 3D This is an image obtained after edge detection after sharpening. Since high-frequency pixels are over-sharpened, there are a lot of fragmentary noise points in the edge image.

[0117] Step S314: Combine the depth map to remove fragmentary noise points in the connected area at the same depth, improve the accuracy of the object boundary, and obtain a boundary map;

[0118] In order to solve these fragmented noise problems, the depth map of the scene is obtained through the dual-camera function, such as Figure 3E As shown in , the individual high-frequency points falling within the connected area at the same depth are removed to obtain the final edge map, as shown in Figure 3F As shown in the figure. Since the simple dual-camera function can only distinguish the depth of the front and back fields, two objects at the same depth cannot be directly distinguished, and the boundary between them will be affected by the clarity during the noise reduction process. Therefore, edge detection technology is first used here to obtain the edge image, and then combined with the depth map to obtain the boundary map.

[0119] Step S315: Based on the boundary map, a low noise reduction coefficient is selected to process the boundary points;

[0120] After obtaining the boundary map, the whole image is subjected to denoising by partition. The image denoising is performed by multiplying the matrix template. Figure 3GAs shown, each square represents a pixel, and the number inside each pixel represents the brightness value or grayscale value of this pixel. The black pixel 31 is a continuous point of a boundary between objects, and there are two objects in the image on both sides, which have different brightness values ​​or grayscale values.

[0121] Generally, a pixel point 33 at the center of the matrix 32 is processed according to the 3x3 matrix 32. At this time, the brightness value or grayscale value of the boundary may be calculated to reduce the difference of the boundary, thus causing the boundary to be blurred. In this embodiment, a boundary map composed of boundary points is obtained in advance, and the boundary map is first subjected to noise reduction processing separately. For example, the noise reduction coefficient is adjusted to half of the normal noise reduction coefficient, such as Figure 3H In the case shown, the boundary point 31 is subjected to noise reduction processing.

[0122] Step S316, using a compensation elimination method to reduce noise on points near the boundary points;

[0123] When performing noise reduction processing on the point 34 near the boundary point, the boundary point 31 is removed and replaced with the point 34 near the boundary point, such as Fig. 3I After the replacement, the point 34 near the boundary point is subjected to noise reduction processing.

[0124] Step S317, after processing the points near the boundary point, replace the boundary point back to the boundary point in the boundary map;

[0125] After the noise reduction process of the point 34 near the boundary point is completed, the replaced boundary point 31 is replaced back to restore the value of the boundary point, and the noise reduction process of the entire boundary and the points near the boundary point is completed.

[0126] Step S318: For the connected areas surrounded by the same boundary, a larger noise reduction coefficient is selected to perform noise reduction processing.

[0127] Since the connected area here will not be disturbed by the boundary, a larger noise reduction coefficient is selected, that is, a stronger noise reduction processing is performed, and the low-frequency noise points in the connected area are also removed to ensure the noise reduction effect.

[0128] In other embodiments, when performing edge detection, in addition to combining the depth map, the edges may be distinguished by combining texture features or color distribution, and the boundary points may be extracted.

[0129] In this embodiment, when the above method is actually applied to a terminal, a prompt "Intelligent noise reduction in progress" may be displayed on the interface of the terminal's shooting application, and a switch control may be provided on the viewfinder interface to turn on or off the above intelligent noise reduction function.

[0130] Through the processing of the above method, most of the boundary information and details can be retained. And there is no need to obtain multiple frames of images, which improves the processing efficiency. It can be applied to the improvement of ordinary shooting quality and the optimization of details in video surveillance.

[0131] Embodiment 4

[0132] Based on the foregoing embodiments, an embodiment of the present invention provides an image denoising device, which includes the units included and the modules included in the units, which can be implemented by a processor in a terminal; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0133] This embodiment provides an image noise reduction device 400, such as Figure 4 As shown, the device comprises:

[0134] The detection unit 401 is configured to perform edge detection on the color image to obtain a preprocessed image;

[0135] An acquisition unit 402 is configured to acquire a depth image having the same scene as the color image;

[0136] The first denoising unit 403 is configured to perform a first denoising process on the pre-processed image according to the depth image to obtain a first image.

[0137] In other embodiments, the detection unit includes:

[0138] A sharpening module, configured to perform sharpening processing on the color image to obtain a sharpened image;

[0139] The detection module is configured to perform edge detection on the sharpened image to obtain the preprocessed image.

[0140] In other embodiments, the first noise reduction unit is further configured as:

[0141] According to the connected areas of each continuous depth in the depth image, the first denoising process is performed on the preprocessed image to obtain the first image.

[0142] In other embodiments, the first noise reduction unit includes:

[0143] A first determination module is configured to determine high-frequency noise points in a connected area according to each connected area of ​​the depth image;

[0144] The noise reduction module is configured to remove the high-frequency noise points according to the first noise reduction process to obtain the first image.

[0145] In other embodiments, the device further comprises:

[0146] an extraction unit, configured to extract boundary points in the first image;

[0147] A first modifying unit is configured to modify the initial parameter values ​​of the boundary points in the first image to obtain a second image;

[0148] A second noise reduction unit is configured to perform a second noise reduction process on the second image to obtain a third image;

[0149] A second modifying unit is configured to modify the parameter values ​​of the boundary points in the third image to initial parameter values ​​to obtain a fourth image;

[0150] The third noise reduction unit is configured to perform a third noise reduction process on the boundary points in the fourth image to obtain a fifth image; wherein the noise reduction degree of the third noise reduction process is weaker than the noise reduction degree of the second noise reduction process.

[0151] In other embodiments, the first modification unit includes:

[0152] A second determination module is configured to determine a reference pixel point for the boundary point;

[0153] A third determination module is configured to determine a parameter value of the reference pixel point;

[0154] The modification module is configured to modify the initial parameter values ​​of the boundary points in the first image to the parameter values ​​of the reference pixel points to obtain a second image.

[0155] In other embodiments, the second determination module is further configured to determine a pixel point within a preset distance range from the boundary point as a reference pixel point;

[0156] The third determination module is further configured to determine the average brightness value of the reference pixels as the parameter value of the reference pixels if the number of the reference pixels is greater than 1.

[0157] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0158] It should be noted that, in the embodiment of the present invention, if the above-mentioned image denoising method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a camera, a mobile terminal, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0159] Correspondingly, an embodiment of the present invention provides an image denoising device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps in the image denoising method provided in the above embodiment are implemented.

[0160] Correspondingly, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the image denoising method provided in the above embodiment are implemented.

[0161] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.

[0162] It should be noted that Figure 5 FIG. 1 is a schematic diagram of a hardware entity of an image noise reduction device (eg, a terminal device) according to an embodiment of the present invention. Figure 5 As shown, the hardware entity of the image noise reduction device 500 includes: a processor 501, a communication interface 502 and a memory 503, wherein

[0163] The processor 501 generally controls the overall operation of the apparatus 500 for image noise reduction.

[0164] The communication interface 502 enables the image noise reduction device to communicate with other terminals or servers through a network.

[0165] The memory 503 is configured to store instructions and applications executable by the processor 501, and can also cache data to be processed or processed by the processor 501 and the modules in the image denoising 500 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0166] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0167] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0168] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0169] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0170] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0171] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0172] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a terminal device (which can be a personal computer, mobile phone, camera, video camera, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0173] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for image denoising, characterized in that: The method comprises: Perform edge detection on the color image to obtain a preprocessed image; Acquire a depth image having the same scene as the color image; Performing a first noise reduction process on the preprocessed image according to the depth image to obtain a first image; The method further comprises: extracting boundary points in the first image; Modifying the initial parameter values ​​of the boundary points in the first image to obtain a second image; performing a second noise reduction process on the second image to obtain a third image; Modifying the parameter values ​​of the boundary points in the third image to initial parameter values ​​to obtain a fourth image; Performing a third noise reduction process on the boundary points in the fourth image to obtain a fifth image; wherein the noise reduction degree of the third noise reduction process is weaker than the noise reduction degree of the second noise reduction process.

2. The method according to claim 1, characterized in that The step of performing edge detection on the color image to obtain a preprocessed image includes: Performing sharpening processing on the color image to obtain a sharpened image; Perform edge detection on the sharpened image to obtain the preprocessed image.

3. The method according to claim 2, characterized in that The performing a first denoising process on the pre-processed image according to the depth image to obtain a first image includes: According to the connected areas of each continuous depth in the depth image, the first denoising process is performed on the preprocessed image to obtain the first image.

4. The method according to claim 3, characterized in that The step of performing the first denoising process on the preprocessed image according to the connected regions of each continuous depth in the depth image to obtain the first image includes: Determine high-frequency noise points in the connected area according to each connected depth in the depth image; According to the first noise reduction processing, the high-frequency noise points are removed to obtain the first image.

5. The method according to claim 1, characterized in that: Modifying the initial parameter values ​​of the boundary points in the first image to obtain a second image includes: Determining reference pixel points for the boundary points; Determining a parameter value of the reference pixel; The initial parameter values ​​of the boundary points in the first image are modified to the parameter values ​​of the reference pixel points to obtain a second image.

6. The method according to claim 5, characterized in that Determining reference pixel points for the boundary points includes: determining pixel points within a preset distance range from the boundary points as reference pixel points; The determining the parameter value of the reference pixel includes: if the number of the reference pixel points is greater than 1, determining the average brightness value of the reference pixel points as the parameter value of the reference pixel points.

7. A device for image noise reduction, characterized in that: The device comprises: A detection unit configured to perform edge detection on the color image to obtain a preprocessed image; An acquisition unit configured to acquire a depth image having the same scene as the color image; A first denoising unit, configured to perform a first denoising process on the pre-processed image according to the depth image to obtain a first image; The device also includes: an extraction unit, configured to extract boundary points in the first image; A first modifying unit is configured to modify the initial parameter values ​​of the boundary points in the first image to obtain a second image; A second noise reduction unit is configured to perform a second noise reduction process on the second image to obtain a third image; A second modifying unit is configured to modify the parameter values ​​of the boundary points in the third image to initial parameter values ​​to obtain a fourth image; The third noise reduction unit is configured to perform a third noise reduction process on the boundary points in the fourth image to obtain a fifth image; wherein the noise reduction degree of the third noise reduction process is weaker than the noise reduction degree of the second noise reduction process.

8. A device for image noise reduction, characterized in that: The device comprises: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the image denoising method provided in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are configured to execute the image denoising method provided in any one of claims 1 to 6.

Images