Point cloud rapid splicing method and system for surface confocal surface topography

Abstract

The invention provides a point cloud rapid splicing method and system for surface confocal surface topography, and relates to the technical field of point cloud splicing, and the method comprises the steps: carrying out the regional scanning of a target object through a surface confocal microscope, so as to obtain a plurality of point cloud data sets; voxel filtering is sequentially carried out on the point cloud data set to obtain a transition point cloud set, radius filtering is carried out on the transition point cloud set to obtain a standard point cloud set, and the standard point cloud set comprises a source point cloud and a target point cloud; iCP registration is carried out on the source point cloud and the target point cloud, and a preliminary splicing point cloud is acquired; and converting the preliminary splicing point cloud into a depth map corresponding to the preliminary splicing point cloud, and performing inverse conversion operation on the depth map to obtain a final splicing point cloud. According to the invention, the integrity and geometric accuracy of the final spliced point cloud can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud stitching, and in particular to a method and system for fast stitching of point clouds of planar confocal surface topography. Background Art

[0002] Point cloud stitching is a crucial step in 3D point cloud processing and a critical task in fields such as reverse engineering, virtual reality, and cultural relic restoration. In actual measurement, due to the limited viewing angles of cameras and projectors, it's difficult to obtain the complete shape of an object from a single perspective. This necessitates measuring from multiple perspectives and stitching the resulting point cloud data into a complete 3D model. This process is known as point cloud stitching.

[0003] Chinese patent publication number CN117593204B discloses a point cloud instance segmentation method based on supervoxel neighbor clustering. This application first collects a 3D point cloud of the target scene and obtains a noise-free 3D point cloud through preprocessing. A cloth simulation optimization algorithm is used to segment the ground points of the 3D point cloud, dividing the target scene into a ground scene and a non-ground scene. A coarse segmentation of the target scene is achieved using 3D point cloud feature learning technology. The point cloud is voxelized using octree search technology, and on this basis, the voxels are initially clustered to achieve the conversion of the point cloud to a supervoxel cloud. Global features of the supervoxel cloud are extracted, and the distance matrix between global features is calculated. Using voxel resolution, the connectivity between supervoxels is determined to achieve instance segmentation of the 3D point cloud of the target type. However, this approach directly constructs supervoxels on the original point cloud, which is susceptible to noise and outliers, resulting in a decrease in the accuracy of subsequent clustering. Therefore, it is highly desirable to provide a method and system for rapid point cloud splicing of confocal surface topography to improve the integrity and geometric accuracy of the final spliced ​​point cloud. Summary of the Invention

[0004] In light of this, the present invention proposes a method and system for rapid point cloud stitching of confocal surface topography. Voxel filtering significantly reduces the subsequent computational burden without significantly losing geometric information. Radius filtering further removes isolated points with too few neighbors, effectively suppressing noise and outliers. The initial stitched point cloud is converted into a corresponding depth map, which is then inversely converted back into a 3D point cloud, thereby improving the integrity and geometric accuracy of the final stitched point cloud.

[0005] The present invention provides a method for quickly stitching point clouds of surface confocal topography, the method comprising:

[0006] Scan the target object in different areas using an area confocal microscope to obtain multiple point cloud data sets;

[0007] performing voxel filtering on the point cloud dataset in sequence to obtain a transition point cloud set, and performing radius filtering on the transition point cloud set to obtain a standard point cloud set, wherein the standard point cloud set includes a source point cloud and a target point cloud;

[0008] Performing ICP registration on the source point cloud and the target point cloud to obtain a preliminary spliced ​​point cloud;

[0009] The preliminary spliced ​​point cloud is converted into a depth map corresponding to the preliminary spliced ​​point cloud, and an inverse conversion operation is performed on the depth map to obtain a final spliced ​​point cloud.

[0010] On the basis of the above technical solution, preferably, the voxel filtering of the point cloud data set specifically includes:

[0011] Calculate the minimum and maximum values ​​of any point cloud in the point cloud dataset in the X, Y, and Z directions, and divide the grids along the X, Y, and Z directions according to the minimum value of the point cloud, the maximum value of the point cloud, and a preset voxel side length to construct a three-dimensional voxel array;

[0012] Dividing the coordinates of each original sampling point in the point cloud by the preset voxel side length in the corresponding direction to determine the voxel grid to which the original sampling point belongs in the three-dimensional voxel array;

[0013] For each non-empty voxel grid in the three-dimensional voxel array, sequentially accumulating the coordinates of all original sampling points in the non-empty voxel grid and dividing the sum by the number of original sampling points in the non-empty voxel grid to obtain the centroid of the non-empty voxel grid;

[0014] The centroids of all non-empty cells are integrated into a transition point cloud set for output.

[0015] On the basis of the above technical solution, preferably, performing radius filtering on the transition point cloud set specifically includes:

[0016] For each transition sampling point in the transition point cloud set, construct a sampling sphere with the transition sampling point as the sphere center and a preset length threshold as the radius;

[0017] Searching for all transition sampling points that meet a preset condition within the sampling sphere to obtain a neighborhood point set corresponding to the transition sampling point, wherein the preset condition is that the Euclidean distance of the transition sampling point is less than or equal to the preset length threshold point;

[0018] If the number of all transition sampling points in the neighborhood point set is less than a preset minimum number of neighbor points, the transition sampling point is marked as an outlier sampling point, and the outlier sampling point is removed from the transition point cloud set;

[0019] Determine whether each transition sampling point in the transition point cloud set is an outlier sampling point in sequence, and output the transition point cloud set after removing all outlier sampling points as a standard point cloud set.

[0020] Further preferably, the converting the preliminary spliced point cloud into a depth map corresponding to the preliminary spliced point cloud specifically comprises:

[0021] According to the minimum value and the maximum value of the preliminary spliced point cloud in the X and Y directions, three-dimensional pixel points corresponding to a preset pixel size are divided along the X and Y directions in combination with the preset pixel size;

[0022] Based on a first pixel conversion function, coordinate calculation is performed on each three-dimensional pixel point in the preliminary spliced point cloud to obtain a two-dimensional pixel point corresponding to the three-dimensional pixel point;

[0023] The depth value of each three-dimensional pixel point is mapped to a depth candidate value of a two-dimensional pixel point, and a mapping set of all three-dimensional pixel points mapped to the same two-dimensional pixel point is marked as a depth candidate value set;

[0024] For each two-dimensional pixel point, if the depth candidate value set corresponding to the two-dimensional pixel point is a non-empty set, the pixel depth value is calculated, and if the depth candidate value set corresponding to the two-dimensional pixel point is an empty set, the pixel depth value corresponding to the two-dimensional pixel point is set to a preset invalid depth value;

[0025] The pixel depth values of all two-dimensional pixel points are arranged in a preset row-column order to obtain a depth map corresponding to the preliminary spliced point cloud.

[0026] Further preferably, the inverse conversion operation is performed on the depth map to obtain a final spliced point cloud, specifically comprising:

[0027] The pixel depth value corresponding to each two-dimensional pixel point in the depth map is read, and if the pixel depth value is not equal to the preset invalid depth value, the coordinates of a standard three-dimensional pixel point corresponding to the two-dimensional pixel point are calculated based on a second pixel conversion function;

[0028] Mirror mapping is performed on each standard three-dimensional pixel point along the Y direction, and all mapped standard three-dimensional pixel points are collected to obtain a final spliced point cloud.

[0029] Further preferably, before arranging the pixel depth values of all two-dimensional pixel points in a preset row-column order, it further comprises:

[0030] If there is an unassigned two-dimensional pixel in the preliminary spliced ​​point cloud or the pixel depth value corresponding to the two-dimensional pixel is a preset invalid depth value, the two-dimensional pixel is interpolated by using the distance-weighted average of the valid two-dimensional pixels around the two-dimensional pixel, wherein the preset invalid depth value is 0 or the minimum grayscale value of the image.

[0031] More preferably, the source point cloud and the target point cloud are point clouds collected from adjacent areas on the surface of the same target object, and the source point cloud and the target point cloud have an overlapping area greater than or equal to 20% in the target plane.

[0032] In a second aspect of the present application, a point cloud rapid stitching system for surface confocal topography is provided, wherein the point cloud rapid stitching system comprises a data acquisition module, a point cloud registration module, and a point cloud stitching module, wherein:

[0033] The data acquisition module is used to scan the target object in different areas through an area confocal microscope to obtain multiple point cloud data sets;

[0034] The point cloud registration module is used to perform voxel filtering on the point cloud dataset in sequence to obtain a transition point cloud set, and perform radius filtering on the transition point cloud set to obtain a standard point cloud set, wherein the standard point cloud set includes a source point cloud and a target point cloud, and perform ICP registration on the source point cloud and the target point cloud to obtain a preliminary spliced ​​point cloud;

[0035] The point cloud stitching module is used to convert the preliminary stitched point cloud into a depth map corresponding to the preliminary stitched point cloud, and perform an inverse conversion operation on the depth map to obtain a final stitched point cloud.

[0036] In a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory.

[0037] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. The computer program is executed by a processor to implement the steps of a method for rapid point cloud stitching of planar confocal surface morphology.

[0038] The method and system for rapid point cloud stitching of confocal surface topography provided by the present invention have the following beneficial effects compared with the prior art:

[0039] (1) By scanning the target object in different areas with a confocal microscope, high-density point cloud data from different perspectives can be obtained, which can maximize the retention of tiny structural features and detail information. Voxel filtering can evenly downsample the point cloud without significantly losing geometric information, significantly reducing the subsequent computational burden. Radius filtering can further remove isolated points with too few neighboring points, effectively suppress noise and outliers, and improve the quality of the point cloud. At the same time, iterative IPC registration is performed on the source point cloud and the target point cloud. By minimizing the point-to-point / point-to-surface distance, sub-pixel precise alignment is achieved. The preliminary spliced ​​point cloud is converted into a corresponding depth map, and then the depth map is inversely converted back to a three-dimensional point cloud. This not only repairs holes and lost areas, but also further evens out the point cloud distribution and improves surface continuity, thereby improving the integrity and geometric accuracy of the final spliced ​​point cloud.

[0040] (2) By constructing a neighborhood sphere with a fixed radius around each sampling point and setting a minimum threshold for the number of neighbors, points with too sparse neighborhoods are directly identified as outliers and removed, effectively eliminating isolated points and abnormal points caused by noise, occlusion or scanning artifacts, and improving the purity of the point cloud. In addition, the radius filter can evenly retain the effective sampling points in each area based on the dynamic judgment of the local spatial density, avoiding the uneven density caused by simple downsampling, making the standard point cloud more evenly distributed in space. At the same time, by eliminating a small number of outliers with great influence at an early stage, the invalid data in the neighborhood search and matrix operation is reduced, which helps to accelerate the subsequent depth map conversion, image filtering and inverse conversion processes, and improve the overall processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of the process of a method for rapid point cloud stitching of surface confocal topography provided by the present invention;

[0043] Figure 2 A schematic diagram of the depth map inverse conversion operation provided by the present invention;

[0044] Figure 3 A schematic diagram of the structure of the point cloud rapid splicing system provided by the present invention;

[0045] Figure 4 This is a schematic structural diagram of the electronic device provided by the present invention.

[0046] Explanation of the accompanying symbols: 1. Point cloud rapid stitching system; 11. Data acquisition module; 12. Point cloud registration module; 13. Point cloud stitching module; 2. Electronic device; 21. Processor; 22. Communication bus; 23. User interface; 24. Network interface; 25. Memory. DETAILED DESCRIPTION

[0047] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] The present invention discloses a method for fast splicing of point clouds of confocal surface topography, referring to Figure 1 , the steps of the method include S1 to S4.

[0049] Step S1: Scan the target object in different areas using a confocal microscope to obtain multiple point cloud data sets.

[0050] In this step, the total size of the area to be measured in the XY plane is determined, and the subarea size and overlap ratio are set. Based on the total size and the subarea plus overlap ratio, an M×N scanning grid is generated; each node in the grid serves as the starting point for a subarea scan. An objective lens with an appropriate magnification (5×–50×) is selected, taking into account both field of view size and resolution. Confocal scanning or laser point scanning is used to ensure that Z-direction depth information is obtained in a single XY scan. The XY scanning stepping is either equidistant or adaptive, with a step size generally no greater than half the optical resolution. An additional overlap region is sampled at the X / Y boundary to ensure sufficient common features between adjacent subareas.

[0051] In one example, the source point cloud and the target point cloud are point clouds collected from adjacent areas on the surface of the same target object, and the source point cloud and the target point cloud have an overlapping area greater than or equal to 20% in the XY plane. Generally, the high-precision XY motor motion linearity is 5μm / 1mm. When scanning 100mm with a 5X objective lens, the margin needs to be 5×100=500μm=0.5mm. Considering that the full range will not be used in actual use, a minimum margin of 20% should be left.

[0052] In step S2, voxel filtering is performed on the point cloud dataset in sequence to obtain a transition point cloud set, and radius filtering is performed on the transition point cloud set to obtain a standard point cloud set, wherein the standard point cloud set includes a source point cloud and a target point cloud.

[0053] This step also includes steps S21 to S28.

[0054] Step S21: Calculate the minimum and maximum values ​​of any point cloud in the point cloud dataset in the X, Y, and Z directions. Based on the minimum and maximum values ​​of the point cloud and the preset voxel side length, divide the grid along the X, Y, and Z directions to construct a three-dimensional voxel array.

[0055] In this step, the original point cloud can be expressed as P = {p1,p2,…,p n}, where p i =(x i ,y i ,z i ); the voxel side length represents leafSize=(l x ,l y ,l z ); The filtered point cloud can be expressed as P′={c1,c2,…,c m}, m<<n. Count the minimum and maximum values ​​of the original point cloud in the X / Y / Z directions and get the bounding box [X min ,X max ]×[Y min ,Y max ]×[Z min ,Z max ]. The number of voxel grids divided along each direction is expressed as as well as

[0056] In step S22 , the coordinates of each original sampling point in the point cloud are divided by the preset voxel side length in the corresponding direction to determine the voxel grid to which the original sampling point belongs in the three-dimensional voxel array.

[0057] In this step, for each point p i , calculate its voxel index (i,j,k),

[0058] Step S23 , for each non-empty voxel in the three-dimensional voxel array, the coordinates of all original sampling points in the non-empty voxel are sequentially accumulated and divided by the number of original sampling points in the non-empty voxel to obtain the centroid of the non-empty voxel.

[0059] In this embodiment, for each non-empty voxel V k , let it contain point set S k ={p k1 ,…,p kn}, calculate the center of mass c k Added output point cloud.

[0060] In step S24 , the centroids of all non-empty voxels are integrated into a transition point cloud set for output.

[0061] In one example, voxel filtering is performed on two point clouds. Assume that there are N points in a voxel in the point cloud, and the coordinates of each point are (x1, y1, z1), (x2, y2, z2), ... (x n ,y n ,z n ), each voxel is N is generally set to 10, with the aim of reducing the size of the point cloud while preserving the geometric characteristics of the point cloud as much as possible.

[0062] In this implementation, uniform resampling of the point cloud is achieved by dividing the point cloud into grids of equal size in the X / Y / Z directions and using the centroid within each non-empty voxel to represent the original point set. This significantly reduces the number of points while preserving the geometric shape and surface relief of each local region to the greatest extent possible, significantly reducing the computational burden of subsequent filtering, registration, and stitching algorithms. This simplifies the scale of neighborhood searches and matrix operations, accelerating the overall processing flow. Isolated or extremely offset points are often distributed in empty voxels or voxels with very few points. Calculating the centroid naturally eliminates these unrepresentative, outlier samples. Regular 3D voxel arrays are naturally amenable to parallel processing or GPU acceleration, further reducing filtering time. The resulting "transitional point cloud" after voxel filtering features uniform distribution, low noise, and manageable data volume, facilitating rapid convergence and improving registration accuracy for registration algorithms such as ICP. The standardized point cloud density ensures more balanced reconstruction quality across regions during subsequent depth map conversion and deconversion, resulting in a more complete, smoother stitching result with manageable geometric errors.

[0063] Step S25 : for each transition sampling point in the transition point cloud set, a sampling sphere is constructed with the transition sampling point as the sphere center and the preset length threshold as the radius.

[0064] In this step, the preset length threshold r is a user-adjustable parameter used to balance the noise removal effect and the degree of point cloud detail preservation, and satisfies r≥2δ, where δ is the average point distance of the point cloud.

[0065] Step S26 , searching for all transition sampling points that meet a preset condition in the sampling sphere to obtain a neighborhood point set corresponding to the transition sampling point, wherein the preset condition is that the Euclidean distance of the transition sampling point is less than or equal to a preset length threshold point.

[0066] In this step, a pre-built kd tree or octree is used to accelerate the retrieval of points within the sphere.

[0067] Step S27: If the number of all transition sampling points in the neighborhood point set is less than the preset minimum number of neighbor points, the transition sampling points are marked as outlier sampling points, and the outlier sampling points are removed from the transition point cloud set.

[0068] Step S28 , determining in sequence whether each transition sampling point in the transition point cloud set is an outlier sampling point, and outputting the transition point cloud set after removing all outlier sampling points as a standard point cloud set.

[0069] In this embodiment, for each point p in the point cloud, a radius r is defined, and the number of neighboring points n within the sphere with p as the center and radius r is calculated; if the number of neighboring points is less than the set threshold N min , then point p is considered an outlier and removed from the point cloud. Assume that a point p1 in the point cloud is (x1, y1, z1), and r is defined based on this point. One of the surrounding points p2 is (x2, y2, z2). The distance between p1 and p2 can be expressed as If d is greater than r, p2 is considered an outlier and removed; otherwise, it is not removed.

[0070] In this embodiment, by constructing a neighborhood sphere with a fixed radius around each sampling point and setting a minimum threshold for the number of neighbors, points with too sparse neighborhoods are directly identified as outliers and removed, effectively eliminating isolated points and abnormal points caused by noise, occlusion, or scanning artifacts, and improving the purity of the point cloud. Radius filtering is based on dynamic judgment of local spatial density, which can evenly retain valid sampling points in each area, avoid density unevenness caused by simple downsampling, and make the standard point cloud more evenly distributed in space, providing stable input for subsequent registration and fusion. After removing unrepresentative isolated points, the distance measurement of registration algorithms such as ICP is more reliable, the iterative process is easier to converge, and the number of mismatched points is significantly reduced during the final splicing, and the overall registration error and morphological distortion are further reduced. By eliminating a small number of outliers with great influence at an early stage, invalid data in neighborhood search and matrix operations is reduced, which helps to accelerate subsequent depth map conversion, image filtering, and inverse conversion processes, and improve overall processing efficiency. The uniformity and low noise characteristics of the standard point cloud result in fewer holes and better smoothing when generating depth maps; after inverse conversion back to a three-dimensional point cloud, the surface continuity and detail restoration are significantly improved, making it suitable for high-precision surface morphology analysis.

[0071] In step S3, ICP registration is performed on the source point cloud and the target point cloud to obtain a preliminary spliced ​​point cloud.

[0072] In this step, for each point in the source point cloud, the nearest point in the target point cloud is found. The method is to establish the correspondence between the two point clouds by using the minimization objective function based on the Euclidean distance of the corresponding points, and to delete the wrong correspondence by the distance threshold to obtain the correct corresponding point (p i ,q i ), the minimization objective function based on the Euclidean distance of corresponding points is:

[0073]

[0074] Among them, p i and q i Represent the coordinates of the corresponding points in the source point cloud and the target point cloud respectively, f(R,t) represents the mean square error of the target distance, i represents the ordinal number of the normal point, N represents the total number of normal points, R represents the rotation matrix, and T represents the translation matrix.

[0075] In this step, the target point cloud is used as a fixed reference coordinate system, the source point cloud is used as the object to be moved, and the first rigid body transformation matrix from the source point cloud to the target point cloud is calculated using the ICP registration algorithm.

[0076] Use the SVD method to solve the rigid body transformation matrix Ti(Ri,ti) that satisfies the following formula, update the source point cloud, and calculate the objective function error value. The calculation process is as follows:

[0077] Assume that the source point cloud P = {p i |i=1,2,...,N S} and target point cloud Q = {q i |i=1,2,...,N T}, after establishing the correspondence between the two point clouds, the centroid of the corresponding point sets is:

[0078]

[0079] Corresponding points establish the covariance matrix C:

[0080]

[0081] Perform singular value decomposition on the matrix C to obtain C=U∧V T , then the rotation matrix R = VU T .

[0082] Calculate the translation vector t, When the number of iterations is greater than the set value or the change in the distance error between two adjacent iterations is less than the set threshold, the iteration is stopped and the optimal rigid body change matrix is ​​output. Otherwise, continue to execute the above steps until the iteration termination condition is met.

[0083] Step S4: convert the preliminary spliced ​​point cloud into a depth map corresponding to the preliminary spliced ​​point cloud, and perform an inverse conversion operation on the depth map to obtain a final spliced ​​point cloud.

[0084] This step also includes steps S41 to S47.

[0085] In step S41 , based on the minimum and maximum values ​​of the preliminary spliced ​​point cloud in the X and Y directions, three-dimensional pixels of a preset pixel size are divided along the X and Y directions in combination with the preset pixel size.

[0086] In this step, the spatial range and pixel resolution of the depth map are determined, that is, the minimum value X in the X and Y directions of the preliminary spliced ​​point cloud is determined. min 、Y min and the maximum value X max 、Y max , combined with the preset pixel sizes ΔX and ΔY, M×N pixels are divided along the X and Y directions.

[0087] Step S42 : performing coordinate calculation on each three-dimensional pixel point in the preliminary spliced ​​point cloud based on the first pixel conversion function to obtain a two-dimensional pixel point corresponding to the three-dimensional pixel point.

[0088] In this step, for each point P in the preliminary spliced ​​point cloud i =(X i ,Y i ,Z i ), calculate the corresponding pixel coordinates (u i ,v i ),

[0089] Step S43 : Map the depth value of each three-dimensional pixel point to a depth candidate value of a two-dimensional pixel point, and mark a mapping set in which all three-dimensional pixels are mapped to the same two-dimensional pixel point as a depth candidate value set.

[0090] In this step, P i Depth value Z i Mapping to pixels (u i ,v i ), and record the set of all depth candidate values ​​mapped to the same pixel as D(u,v).

[0091] Step S44 , for each two-dimensional pixel point, if the depth candidate value set corresponding to the two-dimensional pixel point is a non-empty set, calculate the pixel depth value; if the depth candidate value set corresponding to the two-dimensional pixel point is an empty set, set the pixel depth value corresponding to the two-dimensional pixel point to a preset invalid depth value.

[0092] In this step, for each pixel (u, v), when its corresponding depth candidate value set D(u, v) is not empty, the pixel depth value is calculated as follows: When D(u,v) is empty, Set to the default invalid depth value Z invalid .

[0093] In step S45 , the pixel depth values ​​of all two-dimensional pixels are arranged in a preset row and column order to obtain a depth map corresponding to the preliminary spliced ​​point cloud.

[0094] In this embodiment, by dividing the two-dimensional pixel grid according to the preset pixel size in the X and Y directions, the point cloud is mapped into a regular row and column structure, which unifies the data representation and facilitates subsequent storage, indexing and processing. The three-dimensional reconstruction problem is converted into a two-dimensional depth map processing, and mature image processing algorithms can be directly called to achieve efficient acceleration on the GPU or SIMD parallel architecture. For the same pixel position, all mapped depth candidate values ​​are aggregated and then calculated, which can fuse redundant information in multiple perspectives / overlapping areas, reduce the impact of isolated points or deviations on the final depth value, and improve the robustness of the depth mapping. Pixel points with empty depth candidate sets are assigned preset invalid depth values ​​to clearly distinguish sampling blind areas or invisible areas, which is conducive to the directional implementation of subsequent hole filling or completion strategies. The one-to-one mapping relationship between the depth map and the point cloud ensures that the spatial coordinates can be restored during the inverse conversion. Whether it is smoothing or detail enhancement, it can be returned to the three-dimensional point cloud format losslessly after the two-dimensional domain is completed. Using depth maps for hole repair and filtering can not only significantly reduce the computational complexity of three-dimensional direct filtering, but also accelerate the overall stitching and fusion process while maintaining micron-level geometric accuracy, ultimately obtaining a high-quality three-dimensional point cloud with uniform density, low noise, and continuous surface.

[0095] Step S46 , reading the pixel depth value corresponding to each 2D pixel in the depth map, and if the pixel depth value is not equal to the preset invalid depth value, calculating the coordinates of the 2D pixel corresponding to the standard 3D pixel based on the second pixel conversion function.

[0096] Step S47 : mirror mapping is performed on each standard three-dimensional pixel point along the Y direction, and all the mapped standard three-dimensional pixel points are collected to obtain a final spliced ​​point cloud.

[0097] In one example, the preliminary stitching point cloud I0=(X0 i ,Y0 i ,Z0 i ) is converted into a depth map, that is, XY in the point cloud is used as the pixel point of the image, Z is used as the grayscale value of the image, and when there are multiple Z in the overlapping area, the average is taken, that is, Complete the normalization of the overlapping area; leave 20% of the overlapping area, so that when Z is used as the grayscale value of the image, when there are multiple Zs in the overlapping area, when taking the average, since the overlapping area is relatively large, there will be a smoother effect when the depth map is stitched. The overlapping area is large enough to convert it into a depth map fusion effect.

[0098] like Figure 2 As shown, the depth map I1=(X1 i ,Y1 i ,Z1 i ) is converted to point cloud I2=(X2 i ,Y2 i ,Z2 i), i∈(1,n) During the transformation, the y direction needs to be reversed, that is, Y10=Y2 n , Y11=Y2 (n-1) , Y12=Y2 (n-2) ,...,Y1 i =Y2 (n-i) ...., Y1 n =Y20, other X1 i =X2 i ,Z1=Z2 i Because the zero points of the point cloud and the image are at different positions, they are mirror images of each other. This allows point cloud stitching to be completed, achieving uniform point spacing and deduplication.

[0099] Furthermore, if there is an unassigned two-dimensional pixel in the preliminary spliced ​​point cloud or the pixel depth value corresponding to the two-dimensional pixel is a preset invalid depth value, the two-dimensional pixel is interpolated by the distance-weighted average of the valid two-dimensional pixels around the two-dimensional pixel, where the preset invalid depth value is 0 or the minimum grayscale value of the image.

[0100] In this embodiment, by scanning the regions using a confocal microscope, high-density point cloud data of different viewpoints and regions on the surface of the target object can be obtained, retaining tiny structural features and detail information to the maximum extent. Voxel filtering evenly downsamples the point cloud without significantly losing geometric information, significantly reducing the subsequent computational burden. Radius filtering further removes isolated points with too few surrounding neighbors, effectively suppresses noise and outliers, and improves the quality of the point cloud. At the same time, iterative IPC registration is performed on the source point cloud and the target point cloud, and sub-pixel precise alignment is achieved by minimizing the point-to-point / point-to-surface distance. The preliminary spliced ​​point cloud is converted into a corresponding depth map, and then the depth map is inversely converted back into a three-dimensional point cloud. This not only repairs holes and lost areas, but also further evens out the point cloud distribution and improves surface continuity, thereby improving the integrity and geometric accuracy of the final spliced ​​point cloud.

[0101] Based on the above method, the embodiment of the present application discloses a point cloud fast stitching system for surface confocal topography, referring to Figure 3 The point cloud rapid stitching system 1 includes a data acquisition module 11, a point cloud registration module 12 and a point cloud stitching module 13, wherein:

[0102] The data acquisition module 11 is used to scan the target object in different areas through a confocal microscope to obtain multiple point cloud data sets;

[0103] The point cloud registration module 12 is used to perform voxel filtering on the point cloud dataset in sequence to obtain a transition point cloud set, and perform radius filtering on the transition point cloud set to obtain a standard point cloud set, wherein the standard point cloud set includes a source point cloud and a target point cloud, and perform ICP registration on the source point cloud and the target point cloud to obtain a preliminary spliced ​​point cloud;

[0104] The point cloud splicing module 13 is configured to convert the preliminary spliced point cloud into a depth map corresponding to the preliminary spliced point cloud, and perform an inverse conversion operation on the depth map to obtain the final spliced point cloud.

[0105] In one example, the point cloud registration module 12 is configured to calculate minimum and maximum values of any point cloud in the point cloud dataset in X, Y and Z directions, divide grids in the X, Y and Z directions according to the minimum value of the point cloud, the maximum value of the point cloud and a preset voxel side length, and construct a three-dimensional voxel array; divide the coordinates of each original sampling point in the point cloud by the preset voxel side length in the corresponding direction to determine the voxel grid to which the original sampling point belongs in the three-dimensional voxel array; for each non-empty voxel grid in the three-dimensional voxel array, sequentially accumulate the coordinates of all original sampling points in the non-empty voxel grid and divide the accumulated coordinates by the number of original sampling points in the non-empty voxel grid to obtain a voxel centroid of the non-empty voxel grid; and integrate the voxel centroids of all non-empty voxel grids into a transition point cloud set for output.

[0106] In one example, the point cloud registration module 12 is configured to, for each transition sampling point in the transition point cloud set, construct a sampling sphere domain with the transition sampling point as the sphere center and a preset length threshold as the radius; search for all transition sampling points in the sampling sphere domain that satisfy a preset condition to obtain a neighborhood point set corresponding to the transition sampling point, wherein the preset condition is that the Euclidean distance of the transition sampling point is less than or equal to the preset length threshold; if the number of all transition sampling points in the neighborhood point set is less than a preset minimum neighbor point number, mark the transition sampling point as an outlier sampling point and remove the outlier sampling point from the transition point cloud set; sequentially determine whether each transition sampling point in the transition point cloud set is an outlier sampling point, and output the transition point cloud set after removing all outlier sampling points as a standard point cloud set.

[0107] In one example, the point cloud splicing module 13 is configured to divide three-dimensional pixel points of a corresponding preset pixel size in the X and Y directions according to the minimum and maximum values of the preliminary spliced point cloud in the X and Y directions in combination with a preset pixel size; perform coordinate calculation on each three-dimensional pixel point in the preliminary spliced point cloud based on a first pixel conversion function to obtain a two-dimensional pixel point corresponding to the three-dimensional pixel point; map the depth value of each three-dimensional pixel point to a depth candidate value of the two-dimensional pixel point, and mark a mapping set of all three-dimensional pixel points mapped to the same two-dimensional pixel point as a depth candidate value set; for each two-dimensional pixel point, if the depth candidate value set corresponding to the two-dimensional pixel point is a non-empty set, calculate a pixel depth value, and if the depth candidate value set corresponding to the two-dimensional pixel point is an empty set, set the pixel depth value corresponding to the two-dimensional pixel point as a preset invalid depth value; and arrange the pixel depth values of all two-dimensional pixel points in a preset row-column order to obtain a depth map corresponding to the preliminary spliced point cloud.

[0108] In one example, the point cloud stitching module 13 is used to read the pixel depth value corresponding to each two-dimensional pixel point in the depth map. If the pixel depth value is not equal to the preset invalid depth value, the coordinates of the standard three-dimensional pixel point corresponding to the two-dimensional pixel point are calculated based on the second pixel conversion function; each standard three-dimensional pixel point is mirrored along the Y direction, and all the mapped standard three-dimensional pixel points are collected to obtain the final stitched point cloud.

[0109] In one example, the point cloud stitching module 13 is used to interpolate the two-dimensional pixel by the weighted average of the distances of the valid two-dimensional pixels around the two-dimensional pixel point if there is an unassigned two-dimensional pixel point in the preliminary stitching point cloud or the pixel depth value corresponding to the two-dimensional pixel point is a preset invalid depth value, wherein the preset invalid depth value is 0 or the minimum grayscale value of the image.

[0110] In one example, the source point cloud and the target point cloud are point clouds collected from adjacent areas on the surface of the same target object, and the source point cloud and the target point cloud have an overlapping area greater than or equal to 20% in the target plane.

[0111] See Figure 4 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 2 may include: at least one processor 21 , at least one network interface 24 , a user interface 23 , a memory 25 , and at least one communication bus 22 .

[0112] The communication bus 22 is used to realize the connection and communication between these components.

[0113] The user interface 23 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 23 may also include a standard wired interface and a wireless interface.

[0114] The network interface 24 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0115] The processor 21 may include one or more processing cores. The processor 21 utilizes various interfaces and lines to connect various parts of the entire server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 25, and calling data stored in the memory 25, the processor 21 performs various server functions and processes data. Optionally, the processor 21 may be implemented in at least one hardware form selected from digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 21 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display; and the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 21 and may be implemented separately on a single chip.

[0116] Among them, the memory 25 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 25 includes a non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 25 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 25 may also be optionally at least one storage device located away from the aforementioned processor 21. As Figure 4 As shown, the memory 25 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a method for rapid point cloud stitching of surface confocal topography.

[0117] exist Figure 4In the electronic device 2 shown, the user interface 23 is mainly used to provide an interface for the user to input, and obtain data input by the user; and the processor 21 can be used to invoke an application program of a point cloud fast stitching method of a surface topography stored in the memory 25, which, when executed by one or more processors, causes the electronic device to perform one or more methods as described in the above embodiments.

[0118] A computer-readable storage medium stores instructions that, when executed by one or more processors, cause a computer to perform one or more methods as described in the above embodiments.

[0119] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other orders or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0120] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0121] In several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of units is only a logical function division. In actual implementation, another division mode can be adopted, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some service interface, device or unit, and can be electrical or other forms.

[0122] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0123] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware, or in the form of a software functional unit.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0125] The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not described in the present disclosure.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for rapid point cloud stitching of confocal surface topography, characterized in that: The method comprises: Scan the target object in different areas using an area confocal microscope to obtain multiple point cloud data sets; performing voxel filtering on the point cloud dataset in sequence to obtain a transition point cloud set, and performing radius filtering on the transition point cloud set to obtain a standard point cloud set, wherein the standard point cloud set includes a source point cloud and a target point cloud; Performing ICP registration on the source point cloud and the target point cloud to obtain a preliminary spliced ​​point cloud; The preliminary spliced ​​point cloud is converted into a depth map corresponding to the preliminary spliced ​​point cloud, and an inverse conversion operation is performed on the depth map to obtain a final spliced ​​point cloud.

2. The method for rapid point cloud stitching of surface confocal topography according to claim 1, characterized in that: The voxel filtering of the point cloud data set specifically includes: Calculate the minimum and maximum values ​​of any point cloud in the point cloud dataset in the X, Y, and Z directions, and divide the grids along the X, Y, and Z directions according to the minimum value of the point cloud, the maximum value of the point cloud, and a preset voxel side length to construct a three-dimensional voxel array; Dividing the coordinates of each original sampling point in the point cloud by the preset voxel side length in the corresponding direction to determine the voxel grid to which the original sampling point belongs in the three-dimensional voxel array; For each non-empty voxel grid in the three-dimensional voxel array, sequentially accumulating the coordinates of all original sampling points in the non-empty voxel grid and dividing the sum by the number of original sampling points in the non-empty voxel grid to obtain the centroid of the non-empty voxel grid; The centroids of all non-empty cells are integrated into a transition point cloud set for output.

3. The method for rapid point cloud stitching of surface confocal topography according to claim 2, characterized in that: The performing radius filtering on the transition point cloud set specifically includes: For each transition sampling point in the transition point cloud set, construct a sampling sphere with the transition sampling point as the sphere center and a preset length threshold as the radius; Searching for all transition sampling points that meet a preset condition within the sampling sphere to obtain a neighborhood point set corresponding to the transition sampling point, wherein the preset condition is that the Euclidean distance of the transition sampling point is less than or equal to the preset length threshold point; If the number of all transition sampling points in the neighborhood point set is less than a preset minimum number of neighbor points, the transition sampling point is marked as an outlier sampling point, and the outlier sampling point is removed from the transition point cloud set; It is determined in turn whether each transition sampling point in the transition point cloud set is an outlier sampling point, and the transition point cloud set after removing all outlier sampling points is output as a standard point cloud set.

4. The method for rapid point cloud stitching of surface confocal topography according to claim 1, wherein: The converting the preliminary spliced ​​point cloud into a depth map corresponding to the preliminary spliced ​​point cloud specifically includes: Dividing the initial spliced ​​point cloud into three-dimensional pixels corresponding to the preset pixel size along the X and Y directions according to the minimum and maximum values ​​in the X and Y directions in combination with the preset pixel size; performing coordinate calculation on each three-dimensional pixel point in the preliminary spliced ​​point cloud based on a first pixel conversion function to obtain a two-dimensional pixel point corresponding to the three-dimensional pixel point; Mapping the depth value of each three-dimensional pixel to a depth candidate value of a two-dimensional pixel, and marking a mapping set in which all three-dimensional pixels are mapped to the same two-dimensional pixel as a depth candidate value set; For each two-dimensional pixel point, if the depth candidate value set corresponding to the two-dimensional pixel point is a non-empty set, the pixel depth value is calculated; if the depth candidate value set corresponding to the two-dimensional pixel point is an empty set, the pixel depth value corresponding to the two-dimensional pixel point is set to a preset invalid depth value; Arrange the pixel depth values ​​of all two-dimensional pixel points in a preset row and column order to obtain a depth map corresponding to the preliminary spliced ​​point cloud.

5. The method for rapid point cloud stitching of surface confocal topography according to claim 4, characterized in that: The performing of an inverse conversion operation on the depth map to obtain a final spliced ​​point cloud specifically includes: Reading a pixel depth value corresponding to each two-dimensional pixel point in the depth map, and if the pixel depth value is not equal to the preset invalid depth value, calculating the coordinates of the standard three-dimensional pixel point corresponding to the two-dimensional pixel point based on a second pixel conversion function; Each standard 3D pixel point is mirrored along the Y direction, and all the mapped standard 3D pixel points are collected to obtain the final spliced ​​point cloud.

6. The method for rapid point cloud stitching of confocal surface topography according to claim 4, characterized in that: Before arranging the pixel depth values ​​of all two-dimensional pixels in a preset row and column order, the following steps are also included: If there is an unassigned two-dimensional pixel in the preliminary spliced ​​point cloud or the pixel depth value corresponding to the two-dimensional pixel is a preset invalid depth value, the two-dimensional pixel is interpolated by using the distance-weighted average of the valid two-dimensional pixels around the two-dimensional pixel, wherein the preset invalid depth value is 0 or the minimum grayscale value of the image.

7. The method for rapid point cloud stitching of surface confocal topography according to claim 1, wherein: The source point cloud and the target point cloud are point clouds collected from adjacent areas on the surface of the same target object, and the source point cloud and the target point cloud have an overlapping area greater than or equal to 20% in the target plane.

8. A point cloud rapid stitching system for confocal surface topography, characterized in that: The point cloud rapid splicing system (1) comprises a data acquisition module (11), a point cloud registration module (12) and a point cloud splicing module (13), wherein: The data acquisition module (11) is used to scan the target object in different regions through a confocal microscope to obtain multiple point cloud data sets; The point cloud registration module (12) is used to perform voxel filtering on the point cloud data set in sequence to obtain a transition point cloud set, and perform radius filtering on the transition point cloud set to obtain a standard point cloud set, wherein the standard point cloud set includes a source point cloud and a target point cloud, and perform ICP registration on the source point cloud and the target point cloud to obtain a preliminary spliced ​​point cloud; The point cloud stitching module (13) is used to convert the preliminary stitching point cloud into a depth map corresponding to the preliminary stitching point cloud, and perform an inverse conversion operation on the depth map to obtain a final stitching point cloud.

9. An electronic device, characterized in that: The electronic device (2) comprises a processor (21), a memory (25), a user interface (23) and a network interface (24), wherein the memory (25) is used to store instructions, the user interface (23) and the network interface (24) are used to communicate with other devices, and the processor (21) is used to execute the instructions stored in the memory (25) so that the electronic device (2) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images