A depth estimation method for microscopic discrete noise scenes

Abstract

The present invention relates to a depth estimation method for microscopic discrete noise scenes. The method comprises the following steps: 1, using a micron-level stepping motor to collect a multi-depth image sequence of the microscopic discrete noise scene; 2, using a multi-directional Laplacian operator to perform a convolution operation with the image sequence to obtain multiple focal volume results; 3, obtaining multiple initial depth maps based on the location of the maximum focal volume result; 4, screening the initial depth maps based on constraints proposed from the perspective of statistical data stability; 5, fusing the screened depth maps; 6, combining the image sequence with the fused depth map to obtain a fused image; and 7, performing weighted median filtering on the fused depth map and the fused image to obtain a final depth map of the microscopic scene. The method proposed in the present invention can accurately estimate the depth information of microscopic discrete noise scenes.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision, and in particular relates to a depth estimation method for microscopic discrete noise scenes. Background Art

[0002] Existing micro- and nanoscale 3D reconstruction methods fall into three main categories: laser confocal 3D reconstruction, white-light interferometry 3D reconstruction, and optical image-based 3D reconstruction methods. Laser confocal 3D reconstruction aims to scan and image the microscopic scene to be measured point by point, line by line, and surface by surface using a laser beam emitted by a laser. The laser point in the focal plane of the scene is reflected back to the detection pinhole for imaging, and finally converges to form the 3D structure of the microscopic scene. However, due to the high cost of the laser transmitter and the high degree of hardware integration, it lacks applicability in multiple scenarios. White-light interferometry 3D reconstruction primarily utilizes the low interference property of white light. Since interference only occurs near zero optical path difference, the contrast of the interference fringes decreases with increasing optical path difference. By using an algorithm to locate the zero optical path difference position of each point on the surface of the microscopic scene, the relative height of each point can be determined, thereby obtaining the 3D topography of the microscopic scene. Due to its nanometer-level precision, its scanning efficiency is low. Furthermore, white-light interferometry 3D reconstruction cannot obtain the scene's texture information. In summary, micro- and nanoscale 3D reconstruction systems such as laser confocal 3D reconstruction and white light interferometry 3D reconstruction typically require expensive hardware, resulting in costs of hundreds of thousands or even millions, which prevents their widespread application. Optical image-based 3D reconstruction aims to explore the different focal planes of a scene using clues from two-dimensional images, obtain relative depth information of the microscopic scene by aggregating different focal planes, and then reconstruct the 3D structure of the microscopic scene. Replacing the point scanning methods of the first two methods with surface scanning can significantly increase the reconstruction speed of microscopic scenes, and optical imaging can obtain rich texture information of microscopic scenes. However, existing optical image 3D reconstruction is prone to a large number of reflective noise points during the reconstruction process of microscopic scenes with high reflectivity, which leads to a decrease in reconstruction accuracy. We believe that the cause of this problem is the lack of effective filtering of noise information during the reconstruction process. Therefore, how to effectively distinguish normal information from noise information is the key to achieving high-precision 3D reconstruction of microscopic discrete noise scenes.

[0003] In summary, the method of the present invention proposes a noise-robust microscopic scene depth estimation method from the perspective of data stability in a statistical sense, thereby realizing high-precision three-dimensional reconstruction of microscopic scenes with discrete noise. Summary of the Invention

[0004] In order to overcome the problems existing in the above technologies, an object of the present invention is to provide a depth estimation method for microscopic discrete noise scenes.

[0005] The technical solution adopted by the present invention is: a depth estimation method for microscopic discrete noise scenes, comprising the following steps:

[0006] Step 1: Use a micron-level stepper motor to collect a multi-depth image sequence of a microscopic discrete noise scene. n ,1≤n≤N, where n is the subscript of the image sequence and N is the total number of image sequences;

[0007] Step 2: The image sequence I obtained in step 1 n ,1≤n≤N and C-directional Laplace operators CDML c ,1≤c≤C According to formula (1), the convolution operation is performed to obtain the focal volume results of the C group image sequence

[0008]

[0009] Where c is the subscript of the directional Laplace operator, C is the total number of operators, is the convolution operator, the directional Laplacian operator CDML c ,1≤c≤C is shown in formula (2).

[0010]

[0011] Where sin(·) and cos(·) are sine and cosine functions respectively, θ is the angle parameter, R x With R y are the horizontal Laplace operator and the vertical Laplace operator, respectively, and their expressions are shown in formula (3).

[0012]

[0013] The pixel coordinates (ψ,ζ) are the neighborhood coordinates of the pixel (x,y), s is the step size, and I n (ψ,ζ) is the nth image I n Gray value at the position (ψ,ζ);

[0014] Step 3: The focal volume results of the image sequence of group C obtained in step 2 are According to formula (4), we can get the C initial depth map D c ,1≤c≤C,

[0015]

[0016] in The function representing the subscript n of the focal volume solution;

[0017] Step 4: For the C initial depth maps D obtained in step 3 c ,1≤c≤C each depth map Dk ,1≤k≤C is selected according to the data stability constraint of formula (5), and the depth maps that meet the conditions are combined into R depth maps D i ,1≤i≤R,

[0018]

[0019] Among them S 2 Represents C initial depth map D c ,1≤c≤C variance,D j ,1≤j≤C is the jth depth map among the C initial depth maps, D i ,1≤i≤R is the i-th depth map in the initial depth map after R selections;

[0020] Step 5: The R initial depth map D obtained in step 4 is i ,1≤i≤R According to formula (6), the fused depth map D is obtained F ;

[0021]

[0022] Among them D i ,1≤i≤R is the i-th depth map in the initial depth map after R selections;

[0023] Step 6: The fused depth map D obtained in step 5 is F Combine the image sequence I in step 1 n ,1≤n≤N According to formula (7), the corresponding fusion image F is obtained.

[0024]

[0025] Where D2F(·) is the function of mapping the depth image to the grayscale image;

[0026] Step 7: The fused depth map D obtained in step 5 is F The final depth image D of the microscopic discrete noise scene is obtained by combining the fused image F obtained in step 6 according to formula (8).

[0027] D=jointWMF(D F ,F) (8)

[0028] where jointWMF(·) represents the weighted median filter function.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] (1) The depth estimation method proposed in the present invention filters out abnormal data in various depth maps from the perspective of statistical data stability, thereby effectively filtering out discrete noise in microscopic scenes, and obtaining more accurate depth estimation for microscopic scenes;

[0031] (2) The depth estimation method proposed in the present invention has high computational efficiency and wide applicability in various scenarios. It is particularly suitable for complex microscopic scenes containing many highly reflective surfaces, and can effectively reduce the noise depth points caused by reflections. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of a depth estimation method for a microscopic discrete noise scene according to the present invention;

[0033] Figure 2 Schematic diagram of a depth estimation method for a microscopic discrete noise scene according to the present invention;

[0034] Figure 3 The 80-frame image sequence I collected in step 1 of embodiment 1 of the present invention is n ,1≤n≤80;

[0035] Figure 4 The eight initial depth maps calculated in step 3 of Example 1 of the present invention;

[0036] Figure 5 The six initial depth images selected and obtained in step 4 of Example 1 of the present invention;

[0037] Figure 6 The depth map obtained by fusion in step 5 of embodiment 1 of the present invention;

[0038] Figure 7 The fused image calculated in step 6 of Example 1 of the present invention;

[0039] Figure 8 The depth image of the microscopic discrete noise scene calculated in step 7 of Example 1 of the present invention;

[0040] Figure 9 Depth image of microscopic discrete noise scene obtained by other methods. DETAILED DESCRIPTION

[0041] like Figure 1 、 Figure 2 As shown, a depth estimation method for a microscopic discrete noise scene includes the following steps:

[0042] Step 1: Use a micron-level stepper motor to collect a multi-depth image sequence of a microscopic discrete noise scene. n ,1≤n≤N, where n is the subscript of the image sequence and N is the total number of image sequences;

[0043] Step 2: The image sequence I obtained in step 1 n ,1≤n≤N and C-directional Laplace operators CDML c ,1≤c≤C According to formula (1), the convolution operation is performed to obtain the focal volume results of the C group image sequence

[0044]

[0045] Where c is the subscript of the directional Laplace operator, C is the total number of operators, is the convolution operator, the directional Laplacian operator CDML c ,1≤c≤C is shown in formula (2).

[0046]

[0047] Where sin(·) and cos(·) are sine and cosine functions respectively, θ is the angle parameter, R x With R y are the horizontal Laplace operator and the vertical Laplace operator, respectively, and their expressions are shown in formula (3).

[0048]

[0049] The pixel coordinates (ψ,ζ) are the neighborhood coordinates of the pixel (x,y), s is the step size, and I n (ψ,ζ) is the nth image I n Gray value at the position (ψ,ζ);

[0050] Step 3: The focal volume results of the image sequence of group C obtained in step 2 are According to formula (4), we can get the C initial depth map D c ,1≤c≤C,

[0051]

[0052] in The function representing the subscript n of the focal volume solution;

[0053] Step 4: For the C initial depth maps D obtained in step 3 c ,1≤c≤C each depth map D k ,1≤k≤C is selected according to the data stability constraint of formula (5), and the depth maps that meet the conditions are combined into R depth maps D i ,1≤i≤R,

[0054]

[0055] Among them S 2 Represents C initial depth map D c ,1≤c≤C variance,D j ,1≤j≤C is the jth depth map among the C initial depth maps, D i ,1≤i≤R is the i-th depth map in the initial depth map after R selections;

[0056] Step 5: The R initial depth map D obtained in step 4 is i ,1≤i≤R According to formula (6), the fused depth map D is obtained F ;

[0057]

[0058] Among them D i ,1≤i≤R is the i-th depth map in the initial depth map after R selections;

[0059] Step 6: The fused depth map D obtained in step 5 is F Combine the image sequence I in step 1 n ,1≤n≤N According to formula (7), the corresponding fusion image F is obtained.

[0060]

[0061] Where D2F(·) is the function of mapping the depth image to the grayscale image;

[0062] Step 7: The fused depth map D obtained in step 5 is F The final depth image D of the microscopic discrete noise scene is obtained by combining the fused image F obtained in step 6 according to formula (8).

[0063] D=jointWMF(D F ,F) (8)

[0064] where jointWMF(·) represents the weighted median filter function.

[0065] Example 1

[0066] A depth estimation method for microscopic discrete noise scenes, such as Figure 1 and Figure 2 As shown, the following steps are included:

[0067] Step 1: Use a micron-level stepper motor to collect a multi-depth image sequence of a microscopic discrete noise scene. n ,1≤n≤80, where n is the image sequence subscript, N is the total number of image sequences and is set to 80 in this embodiment, and the image resolution is 800×600 pixels, as shown in Figure 3 As shown;

[0068] Step 2: The image sequence I obtained in step 1 n ,1≤n≤80 and 8-directional Laplace operator CDML c ,1≤c≤8 According to formula (1), the convolution operation is performed to obtain the focal volume results of 8 groups of image sequences

[0069]

[0070] Where c is the directional Laplace operator subscript, C is the total number of operators and is set to 8 in this embodiment. is the convolution operator, the directional Laplacian operator CDML c ,1≤c≤C is shown in formula (2).

[0071]

[0072] Where sin(·) and cos(·) are sine and cosine functions respectively, θ is the angle parameter and is set to 22.5 in this embodiment, R x With R y are the horizontal Laplace operator and the vertical Laplace operator, respectively, and their expressions are shown in formula (3).

[0073]

[0074] The pixel coordinates (ψ, ζ) are the neighborhood coordinates of the pixel (x, y), s is the step size and is set to 1 in this embodiment, I n (ψ,ζ) is the nth image I n Gray value at the position (ψ,ζ);

[0075] Step 3: The focal volume result FV of the image sequence of group C obtained in step 2 is n c ,1≤n≤80,1≤c≤8 According to formula (4), 8 initial depth maps D are calculated. c ,1≤c≤8, such as Figure 4 As shown,

[0076]

[0077] in The function representing the subscript n of the solution focus level result;

[0078] Step 4: For the 8 initial depth maps D obtained in step 3 c , each depth map D in 1≤c≤8 k ,1≤k≤8 According to the data stability constraint of formula (5), the depth maps that meet the conditions are combined into 6 depth maps D i,1≤i≤R, such as Figure 5 As shown,

[0079]

[0080] Among them S 2 Represents 8 initial depth maps D c ,1≤c≤8 variance,D j ,1≤j≤8 is the jth depth map among the 8 initial depth maps, D i ,1≤i≤6 is the i-th depth map among the 6 selected initial depth maps;

[0081] Step 5: The 6 depth maps D obtained in step 4 are i ,1≤i≤6According to formula (6), the fused depth map D is obtained F ,like Figure 6 As shown,

[0082]

[0083] Among them D i ,1≤i≤6 is the i-th depth map among the 6 depth maps;

[0084] Step 6: The fused depth map D obtained in step 5 is F Combine the image sequence I in step 1 n ,1≤n≤80According to formula (7), the corresponding fusion image F is obtained, as follows Figure 7 As shown,

[0085]

[0086] Where D2F(·) is the function of mapping the depth image to the grayscale image;

[0087] Step 7: The fused depth map D obtained in step 5 is F The final depth image D of the microscopic discrete noise scene is obtained by combining the fused image F obtained in step 6 according to formula (8), as follows: Figure 8 As shown,

[0088]

[0089] where jointWMF(·) represents the weighted median filter function.

[0090] Figure 8 The depth image obtained by the method of the present invention is compared with Figure 9 Compared with the depth image obtained by other methods, the method proposed in the present invention can accurately estimate the depth information of microscopic discrete noise scenes.

Claims

1. A depth estimation method for microscopic discrete noise scenes, characterized by the following steps: Step 1: Use a micron-level stepper motor to collect a multi-depth image sequence of a microscopic discrete noise scene. n ,1≤n≤N, where n is the subscript of the image sequence and N is the total number of image sequences; Step 2: The image sequence I obtained in step 1 n ,1≤n≤N and C-directional Laplace operators CDML c ,1≤c≤C According to formula (1), the convolution operation is performed to obtain the focal volume results of the C group image sequence Where c is the subscript of the directional Laplace operator, C is the total number of operators, is the convolution operator, the directional Laplacian operator CDML c ,1≤c≤C is shown in formula (2). CDML c =R x sin((c-1)θ)+R y cos((c-1)θ),1≤c≤C (2) Where sin(·) and cos(·) are sine and cosine functions respectively, θ is the angle parameter, R x With R y are the horizontal Laplace operator and the vertical Laplace operator, respectively, and their expressions are shown in formula (3). The pixel coordinates (ψ,ζ) are the neighborhood coordinates of the pixel (x,y), s is the step size, and I n (ψ,ζ) is the nth image I n Gray value at the position (ψ,ζ); Step 3: The focal volume results of the image sequence of group C obtained in step 2 are According to formula (4), we can get the C initial depth map D c ,1≤c≤C, in The function representing the subscript n of the focal volume solution; Step 4: For the C initial depth maps D obtained in step 3 c ,1≤c≤C each depth map D k ,1≤k≤C is selected according to the data stability constraint of formula (5), and the depth maps that meet the conditions are combined into R depth maps D i ,1≤i≤R, Among them S 2 Represents C initial depth map D c ,1≤c≤C variance,D j ,1≤j≤C is the jth depth map among the C initial depth maps, D i ,1≤i≤R is the i-th depth map in the initial depth map after R selections; Step 5: The R initial depth map D obtained in step 4 is i ,1≤i≤R According to formula (6), the fused depth map D is obtained F ; Among them D i ,1≤i≤R is the i-th depth map in the initial depth map after R selections; Step 6: The fused depth map D obtained in step 5 is F Combine the image sequence I in step 1 n ,1≤n≤N According to formula (7), the corresponding fusion image F is obtained. Where D2F(·) is the function of mapping the depth image to the grayscale image; Step 7: The fused depth map D obtained in step 5 is F The final depth image D of the microscopic discrete noise scene is obtained by combining the fused image F obtained in step 6 according to formula (8). D=jointWMF(D F ,F) (8) where jointWMF(·) represents the weighted median filter function.

Images