An adaptive depth information calibration method for a tilted micro three-dimensional reconstruction result

Abstract

The application discloses a kind of self-adapting depth information calibration methods for oblique micro three-dimensional reconstruction results;Including the following steps: first, the multi-focus image set of the micro object to be measured is collected;Then, using the corrected Laplacian operator and local area focus statistics to solve the focal level of each image;Second, the uncalibrated three-dimensional reconstruction result is derived by the position of the maximum focal level;Finally, the reference area is adaptively selected according to the division area, and the corresponding horizontal and vertical coefficient matrix is obtained by selecting four point coordinates randomly in the reference area, and the corrected three-dimensional reconstruction result is obtained by matrix multiplication with the uncalibrated three-dimensional reconstruction result.The method of the application can realize the adaptive correction of the depth information of the oblique three-dimensional reconstruction result, and provide protection for the fine measurement of micro objects in the field of precision manufacturing.

Description

Technical Field

[0001] This invention belongs to the field of quality control in precision manufacturing, and specifically relates to an adaptive depth information calibration method for tilted microscopic three-dimensional reconstruction results. Background Technology

[0002] 3D topography reconstruction technology based on image focusing information is widely used in fields such as microfabrication and biomedicine. As the scale of research objects gradually decreases, the requirements for the accuracy of 3D topography reconstruction and the imaging quality of acquisition equipment become increasingly stringent. For high-precision 3D reconstruction of some immovable precision equipment, handheld devices are required for measurement. Since the measurement accuracy reaches the micro-nano level, and the placement of the handheld device inevitably results in a slight offset from the microscopic object being measured, this slight offset will cause the overall 3D reconstruction result of the microscopic object to be tilted, leading to errors in subsequent fine measurement.

[0003] Therefore, adaptive depth information correction of tilted microscopic 3D reconstruction results is crucial for accurate measurement of microscopic 3D information. Currently, existing image tilt correction methods mainly fall into three categories: Hough transform, affine transform, and perspective transform. Hough transform-based image tilt correction algorithms first detect straight lines in the image, average the rotation angles of all horizontal lines, and then perform tilt correction based on the rotation angle. This method is suitable for tilt correction of tabular images. However, when there is interference from other information at the image edges, the bounding rectangle's border will cover the entire image, and a zero-degree rotation angle will cause the algorithm to fail. Affine transform-based image tilt correction algorithms mainly use linear transformations in vector space to achieve tilt, translation, rotation, and scaling, but this method is only applicable to transformations in two-dimensional image space. Perspective transform-based image tilt correction algorithms utilize the collinearity of the perspective center, image point, and target point, and rotate the perspective plane around the perspective axis by a specific angle according to the perspective rotation law to achieve correction. However, for the depth information correction problem of tilted microscopic 3D reconstruction results, since there is no globally effective transformation of depth information, the above methods cannot effectively solve this problem.

[0004] Based on the above analysis, we believe that the depth information correction of tilted microscopic 3D reconstruction results faces the following challenges: existing tilt correction methods have a clear correction direction, that is, they predetermine certain information correspondences between the uncorrected image and the corrected image. However, the tilt direction of tilted microscopic 3D reconstruction results is randomly generated. Therefore, how to determine the tilt rate of the overall microscopic 3D reconstruction results through adaptive methods is the key to solving this problem.

[0005] In summary, this patent proposes an adaptive depth information calibration method for tilted microscopic 3D reconstruction results. This method first obtains the uncalibrated 3D reconstruction result of the microscopic object under test through a typical focusing evaluation operator. Then, it adaptively finds a calibration reference area by dividing the area. Four points are randomly selected from the calibration reference area to establish the tilt rate of the global microscopic 3D reconstruction result, thereby achieving accurate calibration of the tilted microscopic 3D reconstruction result and providing a guarantee for subsequent precision measurement. Summary of the Invention

[0006] To overcome the problems existing in the above-mentioned technologies, the purpose of this invention is to provide an adaptive depth information calibration method for tilted microscopic three-dimensional reconstruction results, which can effectively improve the accuracy of three-dimensional measurement of microscopic objects in the field of precision manufacturing quality control.

[0007] The technical solution adopted in this invention is: an adaptive depth information calibration method for tilted microscopic three-dimensional reconstruction results, comprising the following steps:

[0008] Step 1: By adjusting the distance between the micron-level optical imaging device and the microscopic object under test, images of the microscopic object at different focal lengths are acquired at equal intervals. Where P is the total number of images in the image set, p represents the number of images, and its value range is 1≤p≤P, i and j represent the coordinate position of a single image, and their range is 1≤i≤M, 1≤j≤N, M and N are the width and height of a single image in the image set;

[0009] Step 2, extract each image I from the image set obtained in Step 1. p (i,j), 1≤i≤M, 1≤j≤N. Solve for its focal level ML according to the modified Laplace operator of equation (1). p (i,j), 1≤i≤M, 1≤j≤N, and then I is obtained according to the local region focusing statistics of equation (2). p The final focus level F of (i,j), 1≤i≤M, 1≤j≤N p (i,j), 1≤i≤M, 1≤j≤N,

[0010]

[0011]

[0012] Where step represents the spacing, T1 is the threshold of the modified Laplacian operator, and s represents the local pixel range;

[0013] Step 3, adjust the final focus level F obtained in Step 2. p(i,j),1≤i≤M,1≤j≤N. Calculate the uncalibrated 3D reconstruction result D(i,j),1≤i≤M,1≤j≤N of the microscopic object to be measured according to equation (3).

[0014]

[0015] in This indicates the solution for the maximum focal level F. p The function corresponding to the subscript p for (i,j), 1≤i≤M, 1≤j≤N;

[0016] Step 4: Calculate the distance from each point (i,j) to the four vertices in the uncalibrated 3D reconstruction result D(i,j) obtained in Step 3 according to Equation (4). distance dist n (i,j), where n represents the distance ordinal number, and its value range is 1≤n≤4. Then, the diagonal distance diagdist of the uncalibrated 3D reconstruction result D(i,j) is calculated according to Equation (5).

[0017]

[0018]

[0019] Step 5: Divide the uncalibrated 3D reconstruction result D(i,j) obtained in Step 3 into four parts Ds according to Equation (6). k (r,c), 1≤k≤4, then calculate the region Cr where the sum of the gray-level histograms of the four regions is the minimum according to equation (7), and use it as the reference region.

[0020]

[0021]

[0022] Where hist_sum(·) is the histogram summation function. This represents solving for the four-part image Ds. k (r,c), 1≤k≤4 minimum histogram sum hist_sum(Ds) k (r,c)) corresponds to the function with index k;

[0023] Step 6: Randomly select four points from the reference region Cr obtained in Step 5, and calculate the horizontal and vertical distances between the four points respectively to obtain the horizontal distance. and vertical distance Calculate the level weight coefficient according to equation (8). and vertical weight coefficient

[0024]

[0025] Step 7, based on the distance dist obtained in Step 4 n (i,j) and the level weight coefficient obtained in step 6 with vertical weight coefficient According to equation (9), the horizontal direction coefficient matrix tmat(i,j), 1≤i≤M, 1≤j≤N and the vertical direction coefficient matrix kmat(i,j), 1≤i≤M, 1≤j≤N are obtained.

[0026]

[0027] Where * represents the matrix multiplication symbol;

[0028] Step 8: Based on the uncalibrated 3D reconstruction result D(i,j), 1≤i≤M, 1≤j≤N obtained in Step 3, and the horizontal direction coefficient matrix tmat(i,j), 1≤i≤M, 1≤j≤N and the vertical direction coefficient matrix kmat(i,j), 1≤i≤M, 1≤j≤N obtained in Step 7, the corrected 3D reconstruction result flat(i,j), 1≤i≤M, 1≤j≤N is obtained according to Equation (10).

[0029] flat(i,j)=D(i,j)*tmat(i,j)*kmat(i,j),1≤i≤M,1≤j≤N (10)

[0030] The asterisk (*) represents the matrix multiplication symbol.

[0031] The method of the present invention can achieve adaptive correction of tilted three-dimensional reconstruction results. Attached Figure Description

[0032] Figure 1 This is a flowchart of an adaptive depth information calibration method for tilted microscopic 3D reconstruction results;

[0033] Figure 2 This is a schematic diagram of the framework for an adaptive depth information calibration method for tilted microscopic 3D reconstruction results. Detailed Implementation

[0034] like Figure 1 , Figure 2 As shown, an adaptive depth information calibration method for tilted microscopic 3D reconstruction results includes the following steps:

[0035] Step 1: By adjusting the distance between the micron-level optical imaging device and the microscopic object under test, images of the microscopic object at different focal lengths are acquired at equal intervals. Where P is the total number of images in the image set, p represents the number of images, and its value range is 1≤p≤P, i and j represent the coordinate position of a single image, and their range is 1≤i≤M, 1≤j≤N, M and N are the width and height of a single image in the image set;

[0036] Step 2, extract each image I from the image set obtained in Step 1. p (i,j), 1≤i≤M, 1≤j≤N. Solve for its focal level ML according to the modified Laplace operator of equation (1). p (i,j), 1≤i≤M, 1≤j≤N, and then I is obtained according to the local region focusing statistics of equation (2). p The final focus level F of (i,j), 1≤i≤M, 1≤j≤N p (i,j), 1≤i≤M, 1≤j≤N,

[0037]

[0038]

[0039] Where step represents the spacing, T1 is the threshold of the modified Laplacian operator, and s represents the local pixel range;

[0040] Step 3, adjust the final focus level F obtained in Step 2. p (i,j),1≤i≤M,1≤j≤N. Calculate the uncalibrated 3D reconstruction result D(i,j),1≤i≤M,1≤j≤N of the microscopic object to be measured according to equation (3).

[0041]

[0042] in This indicates the solution for the maximum focal level F. p The function corresponding to the subscript p for (i,j), 1≤i≤M, 1≤j≤N;

[0043] Step 4: Calculate the distance from each point (i,j) to the four vertices in the uncalibrated 3D reconstruction result D(i,j) obtained in Step 3 according to Equation (4). distance dist n (i,j), where n represents the distance ordinal number, and its value range is 1≤n≤4. Then, the diagonal distance diagdist of the uncalibrated 3D reconstruction result D(i,j) is calculated according to Equation (5).

[0044]

[0045]

[0046] Step 5: Divide the uncalibrated 3D reconstruction result D(i,j) obtained in Step 3 into four parts Ds according to Equation (6). k (r,c), 1≤k≤4, then calculate the region Cr where the sum of the gray-level histograms of the four regions is the minimum according to equation (7), and use it as the reference region.

[0047]

[0048]

[0049] Where hist_sum(·) is the histogram summation function. This represents solving for the four-part image Ds. k (r,c), 1≤k≤4 minimum histogram sum hist_sum(Ds) k (r,c)) corresponds to the function with index k;

[0050] Step 6: Randomly select four points from the reference region Cr obtained in Step 5, and calculate the horizontal and vertical distances between the four points respectively to obtain the horizontal distance. and vertical distance Calculate the level weight coefficient according to equation (8). and vertical weight coefficient

[0051]

[0052] Step 7, based on the distance dist obtained in Step 4 n (i,j) and the level weight coefficient obtained in step 6 with vertical weight coefficient According to equation (9), the horizontal direction coefficient matrix tmat(i,j), 1≤i≤M, 1≤j≤N and the vertical direction coefficient matrix kmat(i,j), 1≤i≤M, 1≤j≤N are obtained.

[0053]

[0054] Where * represents the matrix multiplication symbol;

[0055] Step 8: Based on the uncalibrated 3D reconstruction result D(i,j), 1≤i≤M, 1≤j≤N obtained in Step 3, and the horizontal direction coefficient matrix tmat(i,j), 1≤i≤M, 1≤j≤N and the vertical direction coefficient matrix kmat(i,j), 1≤i≤M, 1≤j≤N obtained in Step 7, the corrected 3D reconstruction result flat(i,j), 1≤i≤M, 1≤j≤N is obtained according to Equation (10).

[0056] flat(i,j)=D(i,j)*tmat(i,j)*kmat(i,j),1≤i≤M,1≤j≤N (10)

[0057] The asterisk (*) represents the matrix multiplication symbol.

Claims

1. An adaptive depth information calibration method for tilted microscopic 3D reconstruction results, comprising the following steps: Step 1: By adjusting the distance between the micron-level optical imaging device and the microscopic object under test, images of the microscopic object at different focal lengths are acquired at equal intervals. ,in The total number of images in the set. This represents the number of images, and its value range is... , and The coordinates of a single image are defined within a range. , and The width and height of a single image in the image set; Step 2: Extract each image from the image set obtained in Step 1. The focus level is solved using the modified Laplace operator according to equation (1). Then, based on the local region focusing statistics of equation (2), we obtain... final focus level , （1） （2） in Indicates spacing, To correct the threshold of the Laplace operator, Indicates a local pixel range; Step 3, adjust the final focus level obtained in Step 2. The uncalibrated 3D reconstruction result of the microscopic object to be measured is calculated according to equation (3). , （3） in This indicates the solution for the maximum focal level. Corresponding subscript The function; Step 4: Calculate the uncalibrated 3D reconstruction results obtained in Step 3 according to Equation (4). Every point To the four vertices distance ,in This represents the distance ordinal number, and its value range is... Then, the uncalibrated 3D reconstruction result is calculated according to equation (5). diagonal distance ; （4） （5） Step 5: The uncalibrated 3D reconstruction results obtained in Step 3 are... Divide into four parts according to formula (6). Then, according to equation (7), calculate the region where the sum of the gray-level histograms of the four regions is the minimum. and use it as the reference area. （6） （7） in It is a summation function of histograms. This represents solving a four-part image. minimum histogram sum Corresponding subscript The function; Step 6, the reference area obtained in Step 5 Four points are randomly selected, and the horizontal and vertical distances between the four points are calculated respectively to obtain the horizontal distance. and vertical distance Calculate the level weight coefficient according to equation (8). and vertical weight coefficient ; （8） Step 7, based on the distance obtained in Step 4 The level weight coefficients obtained in step 6 with vertical weight coefficient The horizontal direction coefficient matrix was calculated. and vertical direction coefficient matrix , （9） in Represents the matrix multiplication symbol; Step 8: Based on the uncalibrated 3D reconstruction results obtained in Step 3 And the horizontal direction coefficient matrix obtained in step 7 and vertical direction coefficient matrix The corrected 3D reconstruction results were calculated. , （10） in Represents the matrix multiplication symbol.

Images