Image noise reduction method and device based on MEMS stripe structured light

Abstract

The present invention discloses a method and device for image denoising based on MEMS stripe structured light. The method first performs logarithmic operations on pixel values ​​to convert multiplicative noise into additive noise that is easier to filter, so as to obtain a better noise reduction effect. Then, by finding similar points in search boxes in multiple images and using an improved weight calculation function to calculate the corresponding weights, each pixel is restored. The method of the present invention makes full use of the redundant information in the MEMS stripe structured light image sequence, improves the weight calculation function, greatly improves the accuracy of the traditional non-local mean denoising algorithm, and provides more accurate image information for subsequent image processing and analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image noise reduction method and device based on MEMS stripe structured light. Background Art

[0002] The 3D reconstruction technology of stripe structured light based on MEMS belongs to active optical measurement, which has the advantages of fast speed, high precision and non-contact, and has been widely used in industrial inspection, face recognition, medical health, cultural relics protection and many other aspects. It projects sinusoidal stripes onto the surface of the object, and then uses a single or multiple cameras to shoot the measured surface to obtain a structured light image; obtains key information through phase unwrapping and other methods, and finally calculates the 3D distance of the object through 3D image analysis based on the principle of triangulation. MEMS light sources are widely used because of their advantages such as stable performance and precise measurement. However, the speckle noise generated by the mutual interference of the lasers emitted by the MEMS light source seriously damages the quality of the pictures obtained by the camera, and then seriously affects the accuracy of 3D reconstruction. Therefore, it is very necessary to perform noise reduction processing on the 3D imaging pictures of MEMS stripe structured light patterns and suppress the noise signal while retaining the real details of the image as much as possible.

[0003] Image denoising is also called image filtering, and its purpose is to suppress noise signals and restore the essential information of the image while retaining as much original information as possible. The effect of image denoising will directly affect the subsequent image analysis and even the quality of the entire project, which is particularly important in MEMS stripe structured light 3D reconstruction technology.

[0004] Image denoising methods are mainly divided into three categories: spatial pixel feature denoising algorithm, transform domain denoising algorithm and neural network-based denoising algorithm. Due to the particularity of MEMS structured light technology, it is not easy to obtain noise-free original signals as training sets, so neural network-based denoising algorithms are rarely used. The transform domain-based denoising algorithm has the disadvantages of complex calculations and loss of a large number of edge details. Therefore, when processing MEMS structured light images, traditional spatial or transform domain denoising algorithms are usually used. At present, simple Gaussian filtering is widely used in industry to process original structured light images. Although this method is simple, fast and low in complexity, it is not ideal for denoising MEMS stripe structured light images contaminated by speckle noise. The non-local mean filtering (NLM) algorithm proposed by Buades et al. makes full use of the redundant information in the image and uses the non-local weighted average method to filter the noise, which has better denoising effect on the current structured light images.

[0005] The non-local means algorithm assumes that each pixel in the image does not exist alone, but forms different geometric structures such as edges and textures together with the surrounding pixels. Generally speaking, the image information possessed by similar structures is highly overlapping, that is, pixels at different positions in the image have a strong correlation. The non-local means algorithm utilizes the redundant information in the image. The target pixel and its surrounding pixels are collectively referred to as image blocks. During operation, each pixel in the image is traversed, and similar image blocks are searched in the area centered on each pixel. The higher the similarity, the greater the weight obtained. After normalizing the weight, the value of each pixel is the sum of the products of all pixels in the search box area and their corresponding weights. The traditional NLM algorithm only utilizes the redundant information of a single image, while structured light images are usually multiple images with the same background and slightly different phases. Therefore, information in other images will be wasted, resulting in reduced image denoising accuracy.

[0006] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0007] The object of the present invention is to provide a method and device for image noise reduction based on MEMS stripe structured light, which can overcome the problem of low noise reduction accuracy in the prior art.

[0008] To achieve the above object, an embodiment of the present invention provides an image noise reduction method based on MEMS stripe structured light, comprising:

[0009] The MEMS light source projects the stripe structured light onto the surface of the object being measured;

[0010] A plurality of images of the surface of the object to be measured at different times are acquired by the same camera, wherein the plurality of images have the same background and different phase information;

[0011] The non-local mean filtering method is used to calculate the similarity Wi(x,y) between the original pixel x and the pixel y of the i-th image;

[0012] Restore each pixel in the image:

[0013]

[0014] N represents the number of images, and v(y) represents the grayscale value of the pixel.

[0015] In one or more embodiments of the present invention, the non-local means filtering method includes:

[0016] Selecting one of the multiple images as a main image and the other images as auxiliary images;

[0017] Select a similarity box of size d*d and a search box of size D*D in the main image;

[0018] Calculate the similarity Wi(x,y) between the original pixel x and the pixel y of the i-th image

[0019]

[0020] in:

[0021]

[0022]

[0023] Among them, N(x) and N(y) represent the rectangular neighborhood centered on the initial pixel x and the pixel y in the search box; ||N(x)-N(y)|| 2 represents the Gaussian weighted Euclidean distance between two neighborhoods; v(x+z) and v(y+z) represent the grayscale values ​​of the pixels; Z(x) represents the normalization coefficient, which is D in the search box. 2 The sum of the weights of the points; h is the smoothing parameter.

[0024] In one or more embodiments of the present invention, before the non-local mean filtering method is used to calculate the similarity Wi(x, y) between the original pixel point x and the pixel point y of the i-th image, the method further includes:

[0025] Perform logarithmic operation on the grayscale value v of all images to convert multiplicative noise into additive noise.

[0026] In one or more embodiments of the present invention, v=ln(v+1).

[0027] In one or more embodiments of the present invention, after restoring each pixel in the image, the method further includes:

[0028] The gray value of the pixel is restored by exponential calculation.

[0029] In one or more embodiments of the present invention, the gray value v after performing exponential calculation to restore the gray value of the pixel point satisfies:

[0030] v=exp(v)-1

[0031] To achieve the above object, an embodiment of the present invention further provides an image noise reduction processing device, comprising:

[0032] MEMS light source projects stripe structured light onto the surface of the object being measured;

[0033] A camera is used to obtain multiple images of the surface of the object under test at different times, wherein the multiple images have the same background and different phase information;

[0034] The processing unit calculates the similarity Wi(x,y) between the original pixel x and the pixel y of the i-th image using a non-local mean filtering method;

[0035] Restore each pixel in the image:

[0036]

[0037] N represents the number of images, v(x) and v(y) represent the grayscale values ​​of pixels.

[0038] In one or more embodiments of the present invention, the non-local means filtering method includes:

[0039] Selecting one of the multiple images as a main image and the other images as auxiliary images;

[0040] Select a similarity box of size d*d and a search box of size D*D in the main image;

[0041] Calculate the similarity Wi(x,y) between the original pixel x and the pixel y of the i-th image

[0042]

[0043] in:

[0044]

[0045]

[0046] Among them, N(x) and N(y) represent the rectangular neighborhood centered on the initial pixel x and the pixel y in the search box; ||N(x)-N(y)|| 2 represents the Gaussian weighted Euclidean distance between two neighborhoods; v(x+z) and v(y+z) represent the grayscale values ​​of the pixels; Z(x) represents the normalization coefficient, which is D in the search box. 2 The sum of the weights of the points; h is the smoothing parameter.

[0047] To achieve the above object, an embodiment of the present invention further provides a computer-readable storage medium, characterized in that:

[0048] The invention comprises a computer program which, when executed by a processor, implements the method described.

[0049] To achieve the above object, an embodiment of the present invention further provides an electronic device, characterized in that:

[0050] include:

[0051] processor;

[0052] A memory, configured to store executable instructions of the processor;

[0053] The processor is configured to implement the method by executing the executable instructions.

[0054] Compared with the prior art, the present invention proposes a non-local mean algorithm for denoising an image by utilizing redundant information in multiple images. The present invention makes full use of the redundant information in the MEMS stripe structured light image sequence, improves the weight calculation function, and greatly improves the accuracy of the traditional non-local mean denoising algorithm, providing more accurate image information for subsequent image processing and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1a is a schematic diagram of statistical characteristics of an additive noise image according to an embodiment of the present invention;

[0056] Figure 1b is a schematic diagram of statistical characteristics of a multiplicative noise image according to an embodiment of the present invention;

[0057] Figure 2a is a schematic diagram of comparing statistical characteristics of images according to an embodiment of the present invention;

[0058] Figure 2b is a schematic diagram of a local enlargement of image statistical characteristic comparison according to an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram comparing a weight calculation function according to an embodiment of the present invention and an algorithm function in the prior art;

[0060] Figure 4a It is a DLP stripe structured light simulation image;

[0061] Figure 4b Denoised image after denoising by the algorithm in Comparative Example 1;

[0062] Figure 4c The denoised image after denoising in Example 1. DETAILED DESCRIPTION

[0063] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0064] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0065] The embodiments of the present application can be applied to three-dimensional reconstruction technology of MEMS-based stripe structured light, and can be applied to industrial inspection, face recognition, medical health, cultural relics protection and other aspects, but the embodiments disclosed in the present application are not limited to this.

[0066] In order to better understand the solutions of the embodiments of the present application, the possible application scenarios of the embodiments of the present application are briefly introduced below with reference to the accompanying drawings.

[0067] Example 1

[0068] According to a preferred embodiment of the present invention, a method for reducing image noise based on MEMS stripe structured light comprises the following steps:

[0069] Step 1: The MEMS light source projects stripe structured light onto the surface of the object to be measured.

[0070] Step 2: The same camera is used to obtain multiple images of the surface of the object under test at different times. The multiple images have the same background and different phase information. One of them is selected as the main image to be subjected to noise reduction operation, and the other images in the same group are set as auxiliary images that provide information.

[0071] Step 3: Perform logarithmic operation on the grayscale values ​​v of all input images to convert the multiplicative noise into additive noise that can be better processed and obtain new grayscale values.

[0072] v=ln(v+1)

[0073] Step 4: Select a similarity box of the main image size of d*d and a search box of D*D.

[0074] Step 5: Calculate the different weights of the same pixel x in multiple images, that is, the weight (similarity) Wi(x,y) corresponding to the original pixel x and the pixel y in the search box of the i-th image. The specific weight calculation formula is as follows:

[0075]

[0076] in:

[0077]

[0078]

[0079] Among them, N(x) and N(y) represent the rectangular neighborhood centered on the initial pixel x and the pixel y in the search box; ||N(x)-N(y)|| 2 represents the Gaussian weighted Euclidean distance between two neighborhoods; v(x+z) and v(y+z) represent the grayscale values ​​of the pixels; Z(x) represents the normalization coefficient, which is D in the search box. 2 The sum of the weights of the points; h is the smoothing parameter, which controls the strength of the noise reduction effect. The larger h is, the stronger the noise reduction is, which may cause the image to become blurred; the smaller h is, the weaker the noise reduction is, and the noise reduction may not be thorough; N represents the number of input images involved in noise reduction.

[0080] Step 6: Multiply the grayscale value of each pixel in the search box of multiple images by its corresponding weight to restore the grayscale value of the pixel in the main image. The specific restoration formula is as follows:

[0081]

[0082] Among them, v(x) and v(y) represent the grayscale values ​​of the pixels.

[0083] Step 7: Repeat steps 5-6 for each pixel in the main image in turn until all pixels are processed.

[0084] Step 8: Perform exponential calculation on the gray value of the pixel to restore it.

[0085] v=exp(v)-1

[0086] Figure 1a and Figure 1b The image statistical characteristics of additive noise and multiplicative noise are compared with those of MEMS stripe structured light. Figure 2a and Figure 2b As shown, Figure 2a For the comparison of image statistical characteristics, Figure 2b The figure is a partial enlarged schematic diagram of the comparison of image statistical characteristics. It can be seen from the figure that the image statistical characteristics of MEMS structured light basically coincide with the image statistical characteristics of multiplicative noise, so it can be considered that the main noise type of MEMS structured light image is multiplicative noise. Therefore, when calculating step 3 of this embodiment, the pixel value is first logarithmically operated to convert the multiplicative noise into additive noise that is easier to filter, so as to obtain a better noise reduction effect.

[0087] In step 5 of this embodiment, different weights of the same pixel point x in multiple images are calculated, which can make full use of multiple sets of image information. A set of MEMS structured light images is usually composed of multiple images with the same background but different phase information. These images have the same structural information, and even at certain angles, this information is more than that contained in the original image. By making full use of this information, the information masked by noise in the original image can be restored using the information in other images.

[0088] When the difference between two pixels is too large, the calculated weight is preferably close to 0, so as to prevent the introduction of too much noise error. The weight calculation function of the algorithm of the prior art (which only uses the redundant information of a single picture) is an exponential function, which is too rough. Even when the two pixels are very different, a certain weight will still be assigned. And the function is too sensitive when the pixels are close, and it drops too fast. All of these will lead to a decrease in the noise reduction effect. Therefore, the weight function in step 5 of this embodiment is improved: a high weight is obtained when the error is small, and a low weight is obtained when the error is large. The weight can even be made close to 0 to reduce the introduction of noise. At the same time, the weight drops rapidly within a certain range, making the weight calculation more accurate.

[0089] Combination Figure 3 It can be seen that, compared with the weight calculation function in comparative example 1 (which only uses the redundant information of a single image), the weight calculation function in step 3 of this embodiment has at least the following advantages:

[0090] (1) When pixels are similar, they receive higher weights;

[0091] (2) When the pixels are dissimilar, they receive a lower weight, and when they are too dissimilar, the weight approaches 0;

[0092] (3) The weight can drop rapidly within a certain range, which can increase the accuracy of weight calculation;

[0093] Therefore, the weight calculation function in step 3 of this embodiment can greatly improve the accuracy of the entire algorithm.

[0094] Comparative Example 1

[0095] The traditional non-local mean filtering algorithm is used to reduce the noise of the DLP stripe structured light image with multiplicative noise of different intensities. This algorithm uses the redundant information of a single image. The traditional non-local mean filtering algorithm process is as follows:

[0096] (1) First, with the pixel of the MEMS structured light image as the center, a similarity box of size d*d and a search box of size D*D are selected.

[0097] (2) The similar box slides in the search box and traverses the D 2Pixel points, calculate the sum of the Euclidean distances between the similarity box of each pixel point y in the search box and the corresponding point in the similarity box of the original pixel point x, and then use the weight calculation formula to obtain the weight value w(x,y) between the original pixel point x and the corresponding point y in the search box.

[0098] The weight calculation formula is as follows:

[0099]

[0100] Where: w(x,y) represents the similarity between pixel points x and y, N(x) and N(y) represent the rectangular neighborhood centered on the initial pixel point x and the midpoint y in the search box, ||N(x)-N(y)|| 2 represents the Gaussian weighted Euclidean distance between two neighborhoods; v represents the gray value of the current pixel, Z(x) represents the normalization coefficient, and is the D 2 The sum of the weights of the points. h is a smoothing parameter that controls the strength of the noise reduction effect. The larger h is, the stronger the noise reduction is, which may cause the image to become blurred; the smaller h is, the weaker the noise reduction is, and the noise reduction may not be thorough.

[0101]

[0102]

[0103] (3) The final value of each pixel is the sum of the grayscale value of each pixel in the search box multiplied by its corresponding weight.

[0104]

[0105] The algorithms of Example 1 and Comparative Example 1 were respectively used to simulate adding multiplicative noise with different noise variances to a DLP-based stripe structured light image (the image contains almost no noise and can be considered as a noise-free original image) to test the noise reduction effect.

[0106] Table 1 lists the peak signal-to-noise ratio (PSNR) values ​​of the DLP stripe structured light images with different noise variances after denoising using the denoising methods of Example 1 and Comparative Example 1. It can be seen from the table that regardless of the size of the noise variance, the method of Example 1 has a better denoising effect than the algorithm of Comparative Example 1.

[0107] Table 1 Comparison of PSNR values ​​after denoising of stripe structured light simulation images

[0108]

[0109] Figure 4a It is a DLP stripe structured light simulation image. Figure 4b The denoised image after denoising by the algorithm in Example 1 is: Figure 4cDenoised image after denoising in Example 1. From the direct visual effect, compared with the method in Comparative Example 1, Example 1 retains more image details and has a better effect in eliminating speckle noise.

[0110] In summary, the purpose of this embodiment is to address the imperfections of the non-local mean denoising algorithm in the application of MEMS stripe structured light images, and propose a non-local mean algorithm that uses redundant information in multiple images to reduce the noise of an image. First, because the speckle noise of the MEMS structured light image is a multiplicative noise, during the calculation, the pixel value is first logarithmically operated to convert the multiplicative noise into additive noise that is easier to filter, so as to obtain a better noise reduction effect. After that, by finding similar points in the search boxes of multiple images, the corresponding weights are calculated using the improved weight calculation function to restore each pixel point. The method of this embodiment makes full use of the redundant information in the MEMS stripe structured light image sequence, improves the weight calculation function, greatly improves the accuracy of the traditional non-local mean denoising algorithm, and provides more accurate image information for subsequent image processing and analysis.

[0111] An image noise reduction processing device according to a preferred embodiment of the present invention comprises:

[0112] MEMS light source projects stripe structured light onto the surface of the object being measured;

[0113] A camera is used to obtain multiple images of the surface of the object under test at different times, wherein the multiple images have the same background and different phase information;

[0114] The processing unit calculates the similarity Wi(x,y) between the original pixel x and the pixel y of the i-th image using a non-local mean filtering method;

[0115] Restore each pixel in the image:

[0116]

[0117] N represents the number of images, and v(y) represents the grayscale value of the pixel.

[0118] In one embodiment, the non-local means filtering method includes:

[0119] Selecting one of the multiple images as a main image and the other images as auxiliary images;

[0120] Select a similarity box of size d*d and a search box of size D*D in the main image;

[0121] Calculate the similarity Wi(x,y) between the original pixel x and the pixel y of the i-th image

[0122]

[0123] in:

[0124]

[0125]

[0126] Among them, N(x) and N(y) represent the rectangular neighborhood centered on the initial pixel x and the pixel y in the search box; ||N(x)-N(y)|| 2 represents the Gaussian weighted Euclidean distance between two neighborhoods; v(x+z) and v(y+z) represent the grayscale values ​​of the pixels; Z(x) represents the normalization coefficient, which is D in the search box. 2 The sum of the weights of the points; h is the smoothing parameter.

[0127] An embodiment of the present application also provides a computer-readable storage medium, including a computer program, which implements the method of the above embodiment when executed by a processor.

[0128] The present application also provides an electronic device, including:

[0129] processor;

[0130] A memory, configured to store executable instructions of the processor;

[0131] The processor is configured to implement the method of the above embodiment by executing the executable instructions.

[0132] The memory may be a read only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory may store a program, and when the program stored in the memory is executed by the processor, the processor and the communication interface are used to execute the various steps of the image denoising method of the embodiment of the present application.

[0133] The processor can be a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits, which are used to execute relevant programs to implement the functions required to be performed by the modules in the image denoising device of the embodiment of the present application, or to execute the image denoising method of the method embodiment of the present application.

[0134] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A method for image noise reduction based on MEMS stripe structured light, characterized in that: include: The MEMS light source projects the stripe structured light onto the surface of the object being measured; A plurality of images of the surface of the object to be measured at different times are acquired by the same camera, wherein the plurality of images have the same background and different phase information; The non-local mean filtering method is used to calculate the similarity between the original pixel x and the pixel y of the i-th image. ; Restore each pixel in the image: , N represents the number of images, and Represents the gray value of a pixel. The non-local mean filtering method comprises: Selecting one of the multiple images as a main image and the other images as auxiliary images; Select a similarity box of size d×d and a search box of size D×D in the main image; Calculate the similarity between the original pixel x and the pixel y of the i-th image in: in, Represents a rectangular neighborhood centered on the initial pixel x and the pixel y in the search box; represents the Gaussian weighted Euclidean distance between two neighbors; and represents the gray value of the pixel; Z(x) represents the normalization coefficient, which is D in the search box 2 The sum of the weights of the points; h is the smoothing parameter.

2. The image noise reduction method based on MEMS stripe structured light according to claim 1, characterized in that: In the above, the non-local mean filtering method is used to calculate the similarity between the original pixel point x and the pixel point y of the i-th image. Previously, it also included: Perform logarithmic operation on the grayscale value v of all images to convert multiplicative noise into additive noise.

3. The image noise reduction method based on MEMS stripe structured light according to claim 2, characterized in that: 。 4. The image noise reduction method based on MEMS stripe structured light according to claim 1, characterized in that: After restoring each pixel in the image, the method further includes: The gray value of the pixel is restored by exponential calculation.

5. The image noise reduction method based on MEMS stripe structured light according to claim 4, characterized in that: The gray value v after exponential calculation of the gray value of the pixel satisfies: 。 6. An image noise reduction processing device, characterized in that: include: MEMS light source projects stripe structured light onto the surface of the object being measured; A camera is used to obtain multiple images of the surface of the object under test at different times, wherein the multiple images have the same background and different phase information; The processing unit uses the non-local mean filtering method to calculate the similarity between the original pixel x and the pixel y of the i-th image. ; Restore each pixel in the image: , N represents the number of images, and Represents the gray value of a pixel. The non-local mean filtering method comprises: Selecting one of the multiple images as a main image and the other images as auxiliary images; Select a similarity box of size d×d and a search box of size D×D in the main image; Calculate the similarity between the original pixel x and the pixel y of the i-th image , in: in, Represents a rectangular neighborhood centered on the initial pixel x and the pixel y in the search box; represents the Gaussian weighted Euclidean distance between two neighbors; and represents the gray value of the pixel; Z(x) represents the normalization coefficient, which is D in the search box 2 The sum of the weights of the points; h is the smoothing parameter.

7. A computer-readable storage medium, characterized in that: The invention comprises a computer program, which implements the method according to any one of claims 1 to 5 when being executed by a processor.

8. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor is configured to implement the method according to any one of claims 1 to 5 by executing the executable instructions.

Images