Three-dimensional surface topography reconstruction method based on zoom measurement for complex high-steep structure
Patent Information
- Application Number
- CN202610981546.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-02
- Publication Date
- 2026-10-20
AI Technical Summary
[0005]本发明的目的在于提供一种面向复杂高陡度结构的基于变焦测量的三维表面形貌重构方法，以克服现有变焦测量在高陡度复杂结构区域缺乏适用性、单峰假设失效等问题
[0033]本发明方法适用于高陡度轮廓、复杂纹理、高纵横比等难以直接轴向深度重建的目标样品，同时不显著增加测量的时间和硬件成本。
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention belongs to the field of precision optical microscopy measurement technology, specifically relating to a three-dimensional surface topography reconstruction method based on zoom measurement for complex, steep structures. Background Technology
[0002] Focus variation measurement, also known as multifocal plane stacking measurement, is a non-contact, high-resolution optical detection method that has been widely used in the three-dimensional topography measurement of microscale structures. Without increasing the complexity of the hardware, this technology utilizes the finite depth-of-field characteristics of the optical system combined with controlled axial scanning to recover the three-dimensional topography by solving the sharpness evolution law of spatial pixels in the image sequence. At the same time, multifocal plane stacking measurement technology has the characteristics of diverse illumination methods, flexible system structure and strong parameter adjustability, and has good adaptability to different types of samples. Especially in the three-dimensional topography reconstruction of high steep surfaces, this method can meet the detection needs of micro, complex and high-precision parts, and has been widely used in automobile manufacturing, aerospace and other fields in recent years [1].
[0003] The core idea of zoom measurement is to traverse the image sequence and calculate the focus evaluation value, match the corresponding optimal focus depth for each pixel, and combine the depth estimation strategy to compensate for the step error introduced by discrete sampling, so as to achieve high-precision focus positioning. Therefore, the selection of focus evaluation function and depth estimation strategy has an important impact on the final three-dimensional reconstruction accuracy. Traditional multi-focal plane stacking measurement methods usually use Gaussian interpolation [2] or polynomial interpolation [3] to estimate the depth based on the focus response curve. However, the existing depth interpolation-based estimation strategy often shows certain limitations in the measurement of complex and steep structures. In near-vertical structural regions, the projection of a single pixel may cover multiple depth levels at the same time, causing the focus response curve to deviate from the classic single-peak distribution shape, showing multi-peak stacking or plateau saturation phenomena, resulting in peak positioning failure, and thus causing the reconstruction results to be missing or distorted.
[0004] To address this issue, this invention proposes a spatial neighborhood weighted interpolation algorithm for complex, steep structures without significantly increasing hardware costs or system complexity. This method improves the continuity and accuracy of 3D topography reconstruction results by introducing neighborhood spatial information to weight the estimation of the focus position, thereby enhancing its applicability and practical value in complex structure measurement scenarios. Summary of the Invention
[0005] The purpose of this invention is to provide a three-dimensional surface topography reconstruction method based on zoom measurement for complex and steep structures, so as to overcome the problems of lack of applicability of existing zoom measurement in complex and steep structural regions and the failure of the single-peak assumption.
[0006] This invention proposes a 3D surface topography reconstruction method based on zoom measurement for complex, steep structures. The method includes constructing a zoom measurement system under microscopic imaging conditions to continuously acquire image sequences under different focusing states; calculating a focus criterion value for each image during image sequence processing; constructing a spatial three-dimensional neighborhood model, using the focus criterion value as weights, weighted fitting a plane to the neighborhood data and calculating its normal vector direction; extracting the corresponding discrete focus evaluation curve along the coordinate axis direction corresponding to the maximum component of the normal vector, and performing sub-voxel-level interpolation calculation on the focus position along this coordinate axis direction to obtain the precise focus position of the target pixel. This method combines high sensitivity, high robustness, and good reconstruction continuity, and can be widely applied to high-precision 3D topography measurement scenarios for complex, steep structures without significantly increasing hardware costs or measurement time. The specific steps are as follows:
[0007] (i) Constructing a zoom measurement system under microscopic imaging conditions; the measurement system includes a microscopic measurement system and a motion control mechanism; the microscopic measurement system includes: a camera, a tube mirror, a first beam splitter, a second beam splitter, an objective lens, a micro-displacement stage, a lens group, and a light source; wherein, the camera, tube mirror, second beam splitter, microscopic objective lens, sample to be measured, and micro-displacement stage are arranged coaxially from top to bottom in the optical path to form an imaging optical path; the micro-displacement stage is used to place the sample to be measured; the light source and lens group are sequentially distributed in the optical path of the first beam splitter to form an illumination optical path; at the same time, the micro-displacement stage is used to control the axial movement of the sample to be measured to achieve relative movement with respect to the microscopic objective lens.
[0008] (II) Axial scanning image acquisition:
[0009] (1) Fix the sample to be tested on the stage of the measurement system, adjust the objective lens or the entire microscopic imaging system, and determine the axial position where the highest and lowest points of the sample to be tested can be clearly imaged, so as to set the scanning range and scanning step size of the measurement.
[0010] (2) Based on the scanning range and scanning step size, the camera will capture a sequence of n images, denoted as I. i , i = 1, 2, ..., n;
[0011] (III) Constructing the focus evaluation function:
[0012] (1) Calculate the focus criterion value for each frame in the image sequence based on the selected focus criterion;
[0013] (2) Based on the focus criterion value, set an appropriate threshold to locate the focus area in each frame of the image. Specifically, for each image, obtain the maximum value of its focus criterion, and multiply the value by a preset scaling factor as the threshold. The scaling factor is 0.01 to 0.1.
[0014] (iv) Spatial neighborhood weighted interpolation:
[0015] (1) For the effective focus pixel points (x,y) determined by threshold screening in step (iii), construct a three-dimensional spatial neighborhood window and obtain the corresponding focus criterion value of each pixel point in the neighborhood calculated in step (iii);
[0016] (2) Using the focus criterion values corresponding to the pixels in the neighborhood as weights, perform weighted principal component analysis plane fitting on the neighborhood point set, and calculate the normal vector of the local fitting plane. ;
[0017] (3) Based on the absolute value of the normal vector on the three-dimensional spatial coordinate axis components, determine the coordinate axis direction corresponding to the component with the largest absolute value as the interpolation direction; sample the pixel points in the three-dimensional spatial neighborhood on the interpolation direction, extract the focus evaluation curve of the correspondence between the spatial position and the focus criterion value on the direction; and perform sub-voxel level interpolation calculation on the focus evaluation curve to obtain the precise focus position of the target pixel position;
[0018] (V) 3D topography restoration and point cloud construction:
[0019] By combining the depth information obtained through interpolation with the pixel position, and mapping the actual pixel size, objective lens magnification, and scanning step size to the real physical space, a three-dimensional point cloud model is reconstructed, thereby achieving high-precision three-dimensional morphological reconstruction of the target structure.
[0020] Furthermore:
[0021] The "calculation of focus criterion value based on the selected focus criterion" in step (iii) specifically includes the following process: performing sharpness evaluation calculation on each pixel of each focal plane image in the image sequence to obtain the focus response value of each pixel at each focal plane position; wherein, the focus criterion value is used to characterize the sharpness of the pixel in the focal plane direction, and its calculation method is not limited to a specific operator.
[0022] The three-dimensional spatial neighborhood window mentioned in step (iv) is a three-dimensional window constructed with the target pixel as the center, and the neighborhood range is a preset size. A cube window, where m is odd and satisfies .
[0023] The weighted principal component analysis plane fitting described in step (iv) specifically includes the following process: using the focus criterion value of each neighboring pixel as the weight, the larger the focus criterion value, the greater the weight of the corresponding pixel in the plane fitting calculation; calculating the weighted centroid of the neighboring point set, see formula (1); and constructing the weighted covariance matrix, see formula (2), and performing eigenvalue decomposition on it. The eigenvector corresponding to its smallest eigenvalue represents the normal vector of the local fitting plane in geometric space. Even in the presence of noise interference, this normal vector can still stably characterize the surface features of the local geometry.
[0024] (1)
[0025] (2)
[0026] in, , m is the set of voxels within the window, F j Its corresponding focus criterion value, For normalized weights.
[0027] Furthermore, this invention identifies the largest component by comparing the absolute values of each component of the normal vector, and uses the coordinate axis direction corresponding to this largest component as the interpolation direction for the subsequent focus evaluation curve. This selection mechanism based on the direction of the largest component of the normal vector weakens the influence of random noise on the estimation of the main structural direction, and avoids the peak positioning failure caused by multi-peak or saturation when interpolating only along the z-axis in traditional methods. This significantly enhances the continuity of reconstruction in high-steep regions and the engineering operability of the algorithm. In particular, when |nz| is at its maximum, the local surface is close to horizontal, degenerating into the traditional interpolation along the z-axis direction; when |nx| or |ny| is at its maximum, the local surface is close to vertical (i.e., high-steep region), and interpolation is changed to the corresponding lateral coordinate axis direction. This effectively avoids the multi-peak superposition or curve saturation problem caused by high steepness in traditional z-axis interpolation, significantly improving the stability and accuracy of focus positioning.
[0028] Step (iv) "extracting the corresponding focus evaluation curve" specifically includes the following process: taking the coordinate axis direction corresponding to the maximum component of the normal vector determined by comparison as the interpolation direction of the region where the target pixel is located; sampling the pixel points in the three-dimensional spatial neighborhood along the interpolation direction to obtain the spatial position of each sampling point and the corresponding focus criterion value, and establishing the correspondence between the spatial position and the focus criterion value along the interpolation direction; using the spatial position as the independent variable and the corresponding focus criterion value as the dependent variable to construct a discrete focus evaluation curve.
[0029] Furthermore, the interpolation employs a log-Gaussian model, specifically transforming the Gaussian model into a quadratic function, as shown in equation (3). Three consecutive sampling points are selected near the peak value. A quadratic interpolation model is constructed in the logarithmic domain. The subvocetine offset of the focal peak is obtained by analytically solving for the extremum position of the quadratic function, as shown in formula (4).
[0030] (3)
[0031] (4)
[0032] in, The mean of the Gaussian distribution (corresponding to the optimal focus position). Standard deviation, , , These are the focus criterion values corresponding to three consecutive sampling points. The sampling step size is used; when the interpolation direction is the z-axis, the axial scanning step size is used; when the interpolation direction is the x-axis or y-axis, the actual physical pixel size in the corresponding direction is used, which is the value obtained by dividing the camera pixel size by the objective lens magnification.
[0033] The method of this invention is applicable to target samples with high steep contours, complex textures, and high aspect ratios that are difficult to reconstruct directly along the axial depth, without significantly increasing measurement time and hardware costs.
[0034] This invention, based on traditional multifocal imaging systems, achieves 3D reconstruction using only conventional microscopic acquisition equipment. In image sequence processing, unlike traditional methods that perform axial interpolation based on focus evaluation curves in the depth direction, this invention constructs a spatial three-dimensional neighborhood model and designs a spatial neighborhood weighted interpolation algorithm for complex, steep structures. This effectively improves the system's adaptability and reconstruction accuracy for steep and complex geometric structures. By introducing neighborhood spatial information for weighted estimation of the focus position, it does not rely on special hardware, improving the continuity and accuracy of the 3D topography restoration results, enhancing system sensitivity and robustness. It boasts advantages such as ease of implementation, low cost, and strong scalability, making it suitable for various scenarios including industrial inspection and microstructure measurement. Attached Figure Description
[0035] Figure 1 This is a schematic diagram of a microscopic measurement system.
[0036] Figure 2 This is a flowchart of the method of the present invention.
[0037] Figure 3 The image shows a sample taken by the microscopic measurement system in Example 1.
[0038] Figure 4 This is the result of the spatial neighborhood weighted interpolation section in Example 1.
[0039] Figure 5 This is the result of the three-dimensional reconstruction and restoration in Example 1.
[0040] In the figure, the numbers are as follows: 1 is the light source, 2 is the lens group, 3 is the first beam splitter, 4 is the camera, 5 is the tube mirror, 6 is the second beam splitter, 7 is the microscope objective, 8 is the sample to be tested, and 9 is the micro-displacement stage. Detailed Implementation
[0041] The present invention will be further described below with reference to specific embodiments and accompanying drawings. However, the embodiments do not constitute a limitation of the present invention. All other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention.
[0042] Example 1: 3D Topography Reconstruction Based on Spatial Neighborhood Weighted Interpolation
[0043] The microscopic measurement system designed in this invention is described in [reference needed]. Figure 1 As shown, the system includes a light source 1, a lens group 2, a first beam splitter 3, a camera 4, a tube mirror 5, a second beam splitter 6, a microscope objective 7, a sample to be tested 8, and a micro-displacement stage 9. The camera 4, tube mirror 5, second beam splitter 6, microscope objective 7, sample to be tested 8, and micro-displacement stage 9 are arranged coaxially from top to bottom in the optical path, forming the imaging optical path. The micro-displacement stage 9 is used to place the sample to be tested 8 and achieves axial scanning by changing the distance between the sample and the microscope objective. The light source 1 and lens group 2 are sequentially distributed along the optical path of the first beam splitter 3, forming the illumination optical path. The microscope objective 7 has a numerical aperture of 0.19, a working distance of 110 mm, and an objective magnification of 5×. The system acquires image sequences of the target under test at different focal planes by axially moving the objective lens or the entire imaging system. The image size is 1024×1024, the camera pixel size is 13μm, the number of frames acquired is n=97, the scanning interval is 20μm, and the total scanning depth range is 1940μm. When the interpolation direction is the x-axis or y-axis, the sampling step size in formula (4) is taken as the actual physical pixel size 13μm÷5=2.6μm.
[0044] See the flowchart of the method of this invention. Figure 2 As shown.
[0045] The target sample was a semi-porous microporous sample with a diameter of approximately 1 mm and a depth of approximately 2 mm. The acquired image sequence is illustrated below. Figure 3 As shown, focus criteria values are calculated for each frame in the image sequence to form focus criteria volume data, which provides a basis for subsequent interpolation.
[0046] For effectively focused pixels, further spatial neighborhood weighted interpolation is performed. For each pixel (x, y, z) to be processed, a neighborhood window of a preset size is constructed in three-dimensional space centered on that pixel. In this embodiment, the neighborhood window size is m=7, that is, a 7×7×7 three-dimensional neighborhood window is constructed centered on the pixel to be processed, thereby obtaining the local focus information distribution near that position. Each voxel point within the neighborhood window constitutes a local three-dimensional point set, and its corresponding focus criterion value is recorded. Using the focus criterion values of each point in the neighborhood as weights, a weighted plane fitting is performed on the local point set. Specifically, firstly, the weighted centroid of the neighborhood point set is calculated, then a weighted covariance matrix is constructed, and the covariance matrix is decomposed into eigenvalues. The eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the local fitting plane. Then, the absolute values of the normal vector on the three-dimensional coordinate axis components are compared, and the coordinate axis direction corresponding to the component with the largest absolute value is adaptively selected as the interpolation direction. A log-Gaussian model is used for interpolation to obtain a more accurate focal position for the current pixel. Figure 4 As shown, the inner wall contour obtained after interpolation is continuous, complete, and clear.
[0047] The interpolated coordinates are mapped onto the real 3D space to obtain a complete 3D reconstructed point cloud. For example... Figure 5 As shown, the restored three-dimensional topography is complete and clear, with natural edge transitions.
[0048] This embodiment effectively improves the 3D reconstruction accuracy of complex structural regions by using spatial neighborhood weighted interpolation of multi-focal plane image sequences. It has the advantages of requiring no additional hardware, high reconstruction efficiency, and strong scalability.
[0049] References
[0050] [1] Z.Qiu, J.Tang, "Three-dimensional information measurement and reconstruction for micro-hole array based on the modified shape from focus", Optics and Lasers in Engineering 2024;178:108246.
[0051] [2]S.K.Nayar,Y.Nakagawa,"Shape from focus: an effective approach forrough surfaces",Proceedings of the 1990 IEEE International Conference onRobotics and Automation 1990;218-225.
[0052] [3]M.Subbarao,T.Choi,"Accurate recovery of three-dimensional shapefrom image focus",IEEE Transactions on Pattern Analysis and MachineIntelligence ,1995;17(3):266-274。
Claims
1. A method for reconstructing three-dimensional surface topography based on zoom measurement for complex, steep structures, characterized in that, This includes constructing a zoom measurement system under microscopic imaging conditions to continuously acquire image sequences under different focusing states; calculating a focusing criterion value for each image during image sequence processing; constructing a spatial three-dimensional neighborhood model, using the focusing criterion value as weights to perform weighted fitting of the neighborhood data to a plane and calculate its normal vector direction; and interpolating the focusing evaluation curve along the coordinate axis direction corresponding to the maximum component of the normal vector. The specific steps are as follows: (i) Constructing a zoom measurement system under microscopic imaging conditions; the measurement system includes a microscopic measurement system and a motion control mechanism; the microscopic measurement system includes: a camera, a tube mirror, a first beam splitter, a second beam splitter, an objective lens, a micro-displacement stage, a lens group, and a light source; wherein, the camera, tube mirror, second beam splitter, microscope objective lens, sample to be measured, and micro-displacement stage are arranged coaxially from top to bottom in the optical path to form an imaging optical path; the micro-displacement stage is used to place the sample to be measured; the light source and lens group are sequentially distributed in the optical path of the first beam splitter to form an illumination optical path; at the same time, the micro-displacement stage is used to control the axial movement of the sample to be measured to achieve relative movement with respect to the microscope objective lens; (II) Axial scanning image acquisition: (1) Fix the sample to be tested on the stage of the measurement system, adjust the objective lens or the entire microscopic imaging system, and determine the axial position where the highest and lowest points of the sample to be tested can be clearly imaged, so as to set the scanning range and scanning step size of the measurement. (2) Based on the scanning range and scanning step size, the camera will capture a sequence of n images, denoted as I. i , i = 1, 2, ..., n; (III) Constructing the focus evaluation function: (1) Calculate the focus criterion value for each frame in the image sequence based on the selected focus criterion; (2) Based on the focus criterion value, set an appropriate threshold to locate the focus area in each frame of the image. Specifically, for each image, obtain the maximum value of its focus criterion, and multiply the value by a preset scaling factor as the threshold. The scaling factor is 0.01 to 0.
1. (iv) Spatial neighborhood weighted interpolation: (1) For the effective focus pixel points (x,y) determined by threshold screening in step (iii), construct a three-dimensional spatial neighborhood window and obtain the corresponding focus criterion value of each pixel point in the neighborhood calculated in step (iii); (2) Using the focus criterion values corresponding to the pixels in the neighborhood as weights, perform weighted principal component analysis plane fitting on the neighborhood point set, and calculate the normal vector of the local fitting plane. ; (3) Based on the absolute value of the normal vector on the three-dimensional spatial coordinate axis components, determine the coordinate axis direction corresponding to the component with the largest absolute value as the interpolation direction; sample the pixel points in the three-dimensional spatial neighborhood on the interpolation direction, extract the focus evaluation curve of the correspondence between the spatial position and the focus criterion value on the direction; and perform sub-voxel level interpolation calculation on the focus evaluation curve to obtain the precise focus position of the target pixel position; (V) 3D topography restoration and point cloud construction: By combining the depth information obtained through interpolation with the pixel position, and mapping the actual pixel size, objective lens magnification, and scanning step size to the real physical space, a three-dimensional point cloud model is reconstructed, thereby achieving high-precision three-dimensional morphological reconstruction of the target structure.
2. The three-dimensional surface topography reconstruction method according to claim 1, characterized in that, The process of calculating the focus criterion value based on the selected focus criterion in step (iii) is as follows: perform sharpness evaluation calculation on each pixel of each focal plane image in the image sequence to obtain the focus response value of each pixel at each focal plane position; wherein, the focus criterion value is used to characterize the sharpness of the pixel in the focal plane direction, and its calculation method is not limited to a specific operator.
3. The three-dimensional surface topography reconstruction method according to claim 1, characterized in that, The three-dimensional spatial neighborhood window mentioned in step (iv) is a three-dimensional window constructed with the target pixel as the center, and the neighborhood range is a preset size. A cube window, where m is odd and satisfies .
4. The three-dimensional surface topography reconstruction method according to claim 1, characterized in that, The weighted principal component analysis plane fitting described in step (iv) is as follows: the focus criterion value of each neighboring pixel is used as the weight. The larger the focus criterion value, the greater the weight of the corresponding pixel in the plane fitting calculation. The weighted centroid of the neighboring point set is calculated, as shown in formula (1). The weighted covariance matrix is constructed, as shown in formula (2). The eigenvalues are decomposed, and the eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the local fitting plane. The absolute values of the normal vectors on the three-dimensional spatial coordinate axis components are compared, and the coordinate axis direction corresponding to the component with the largest absolute value in the normal vector is taken as the interpolation direction of the subsequent focus evaluation curve of the region. (1) (2) in, j=1,2,…,m, where m is the set of voxels within the window, F j Its corresponding focus criterion value, For normalized weights.
5. The three-dimensional surface topography reconstruction method according to claim 1, characterized in that, The specific process of extracting the corresponding focus evaluation curve in step (iv) is as follows: take the coordinate axis direction corresponding to the maximum component of the normal vector determined by comparison as the interpolation direction of the region where the target pixel is located; sample the pixel points in the three-dimensional spatial neighborhood in the interpolation direction, obtain the spatial position of each sampling point and the corresponding focus criterion value, and establish the correspondence between the spatial position in the interpolation direction and the focus criterion value. Using spatial location as the independent variable and the corresponding focus criterion value as the dependent variable, a discrete focus evaluation curve is constructed.
6. The three-dimensional surface topography reconstruction method according to claim 5, characterized in that, The interpolation uses a log-Gaussian model. Specifically, the Gaussian model is transformed into a quadratic function, as shown in formula (3). Three consecutive sampling points are selected near the peak. A quadratic interpolation model is constructed in the logarithmic domain; the subvoc offset of the focal peak is obtained by analytically solving the extremum position of the quadratic function, as shown in formula (4); (3) (4) in, The mean is a Gaussian distribution. Standard deviation, , , These are the focus criterion values corresponding to three consecutive sampling points. The sampling step size is used; when the interpolation direction is the z-axis, the axial scanning step size is used; when the interpolation direction is the x-axis or y-axis, the actual physical pixel size in the corresponding direction is used, which is the value obtained by dividing the camera pixel size by the objective lens magnification.