Real-time reverse scanning guiding method

Abstract

The invention discloses a real-time reverse scanning guiding method, which belongs to the technical field of three-dimensional point cloud processing, and comprises the following steps: collecting point cloud, establishing a three-dimensional main model and a W-model, aligning the two models, calculating an alignment error, sending rendered data to VR (Virtual Reality), and formulating a complementary scanning plan. The technical problems of time sequence synchronization, space precise registration and error compensation and efficient block cache rendering of multi-period dynamic point clouds during scanning modeling are solved, precise space alignment and time weighted error compensation of the multi-period point clouds are achieved, the problems of attitude drift and time asynchronization are effectively solved, and the accuracy of the dynamic point clouds is improved. According to the method, spatial block caching and multi-layer fusion rendering technologies are adopted, efficient management and VR real-time visualization of large-scale dynamic point clouds are achieved, and scanning missing area automatic identification and path planning based on the point cloud coverage rate greatly improve the scanning supplementing efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional point cloud processing, and particularly relates to a real-time reverse scanning guidance method. Background Art

[0002] With the rapid development of three-dimensional laser scanning technology and virtual reality (VR) technology, three-dimensional modeling and dynamic visualization based on point clouds have become an important research direction in the reverse development field of large sheet metal parts such as car bodies.

[0003] Currently, traditional technologies rely on hand-held lasers or structured light scanners to collect point clouds, and then through offline point cloud stitching, noise filtering, and CAD modeling. It is difficult for operators to visually check the scanning progress and quality on-site, and it is easy to have missed scans, repeated scans, and pose errors, resulting in low efficiency in subsequent surface fitting and assembly verification. Summary of the Invention

[0004] The purpose of the present invention is to provide a real-time reverse scanning guidance method, which solves the technical problems of time series synchronization, spatial precise registration and error compensation, and efficient block caching rendering of multi-period dynamic point clouds during scanning and modeling.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions: A real-time reverse scanning guidance method includes the following steps: Step 1: Use a hand-held three-dimensional scanner to scan the surface of the workpiece, and collect the spatial coordinate information and pose information of the point cloud in real time; Each point cloud data records its corresponding acquisition time, and an additional time tag is attached to each point according to the acquisition time to construct an original point cloud data set; Step 2: After obtaining the original point cloud data set and performing preprocessing, process the point cloud in the original point cloud data set through a voxel grid filtering algorithm + triangular mesh reconstruction algorithm to generate a three-dimensional main model with a continuous surface and output it; Step 3: By fusing the real-time six-degree-of-freedom pose of the hand-held three-dimensional scanner and the original point cloud data, use a homogeneous transformation matrix to unify each frame of point cloud to the world coordinate system; use a preset time interval to construct a set of standard time tags, align the time stamp of each point to the nearest neighbor standard tag to generate an enhanced point cloud data set; perform time block structure processing on the enhanced point cloud data set, divide the subsets in the block structure according to time order, and generate a W-model model with time tags; Step 4: Use the geometric center alignment method and the ICP algorithm to perform spatial alignment on the three-dimensional main model and the W-model model, calculate the alignment error by constructing an error compensation function, convert the alignment error into a visual color to obtain a color code, output the aligned three-dimensional main model and W-model model, and at the same time output the color code; Step 5: Perform block caching processing on the aligned W-model; Render the W-model and the three-dimensional main model, and push the rendered layer results to the VR device; Step 6: Compare the W-model with the CAD model of the preset workpiece, identify the missed scanning areas according to the coverage rate of the point cloud on the surface mesh of the CAD model, generate path planning using the cost function according to the positions of the missed scanning areas, render the lines of the path planning into the three-dimensional main model according to the rendering method in Step 5, and send it to the VR device.

[0006] Preferably, when performing Step 1, it specifically includes the following steps: Step 1-1: Use a laser scanner to scan the point cloud data on the surface of the workpiece, record the spatial coordinates and acquisition time of each point, and generate a point cloud set for each frame according to the spatial coordinates of each point , and attach a time tag to each point cloud according to the acquisition time ; Step 1-2: Generate the original point cloud data set based on the point cloud set and the time tags attached to each sampling point : ; where M is the total number of frames, represents the three-dimensional coordinates of the j-th point in the i-th frame, , , and N represents the number of point clouds in the i-th frame.

[0007] Preferably, when performing Step 2, it specifically includes the following steps: Step 2-1: Preprocess the original point cloud data set to remove invalid points, isolated points and abnormal noises; Step 2-2: Sparsify the point cloud through the voxel grid filtering algorithm; Step 2-3: Perform structured modeling on the sparse point cloud through the triangular mesh reconstruction algorithm to generate a three-dimensional main model with a continuous surface ; Step 2-4: Output the three-dimensional main model .

[0008] Preferably, when performing Step 3, it specifically includes the following steps: Step 3-1: Obtain the original point cloud data set , and at the same time obtain the attitude information; Step 3-2: Obtain the six-degree-of-freedom pose of each frame collected by the handheld 3D scanner through the spatial tracking system, generate a homogeneous transformation matrix, and convert each point in the homogeneous transformation matrix to the unified world coordinate system; Step 3-3: Construct a standard time tag set, and define a standard time tag every fixed time in it; Step 3-4: Calculate the standard time tag adjacent to the time tag of each point to generate a new enhanced point cloud data set; Step 3-5: Construct the enhanced point cloud data set into a time-block structure composed of multiple subsets, sort each subset in chronological order to obtain a structured point cloud model with time tags, that is, the W-model model.

[0009] Preferably, when performing Step 4, it specifically includes the following steps: Step 4-1: Obtain the 3D main model, W-model model, homogeneous transformation matrix, and standard time tag set; Step 4-2: Calculate the geometric center of all vertices of the 3D main model and the center of all points of the W-model model ; By calculating and the difference, obtain the translation compensation vector , translate all points of the W-model model onto the 3D main model for center alignment; Step 4-3: Adopt the weighted ICP algorithm to solve the optimal matching result based on the processing result of Step 4-2; Step 4-4: Construct an error compensation function to calculate the alignment error of each point, and convert the error value into a visual color through the color mapping function ColorMap to generate a color code; Step 4-5: Output the aligned W-model model and 3D main model, and at the same time output the alignment error of each point and its corresponding color code.

[0010] Preferably, when performing Step 5, it specifically includes the following steps: Step 5-1: Perform block caching processing on the aligned W-model model; when the W-model model is block-cached, its structure is as follows: [Meta-information]+[Time tag index]+[Spatial data block set]+[Operation log]; Among them, the [Spatial data block set] includes multiple [Spatial blocks], and the structure of each [Spatial block] is as follows: [Block unique identifier]+[Point cloud data]+[Time tag]+[Version tracking identifier]+[Alignment error]; Step 5-2: Adopt a multi-layer management method to render the W-model and the 3D main model respectively. Through transparency superposition and depth sorting, the superposition and fusion display of the two models are realized, and according to the color coding, the point cloud deviation is displayed in the layer to generate the rendered layer result; Step 5-3: Push the rendered layer result to the VR device.

[0011] When performing Step 6, it specifically includes the following steps: Step 6-1: Statistically analyze the coverage rate of the point cloud on the surface grid of the CAD model to obtain the coverage rate of each grid cell, and count the number of point clouds actually falling into each grid cell. When the coverage rate of a certain grid is less than the preset threshold, it is determined that the area where the grid is located is a missed scanning area; Step 6-2: Superimpose a highlighted color mask on the grid cells in the missed scanning area; Step 6-3: According to the position of the missed scanning area, plan the next scanning path to generate a path plan; when planning the next scanning path, first extract the geometric centers of all missed scanning grid cells to form a region set; Combined with the pose of the current handheld 3D scanner , taking the current pose position as the starting point, use the cost function to calculate the sequence of path points. The cost function is as follows: ; ; Among them, the cost function represents the optimal path; represents the sequence of path points, that is, starting from the current scanner position, passing through several path points in sequence can finally cover the missed scanning area; n represents the total number of path points, represents two consecutive points; Step 6-4: Generate path lines according to the path plan, render the path lines onto the 3D main model according to the rendering method in Step 5, and push them to the VR device.

[0012] A real-time reverse scanning guidance method described in the present invention solves the technical problems of time sequence synchronization, spatial precise registration and error compensation, and efficient block caching and rendering of multi-period dynamic point clouds during scanning and modeling. The present invention introduces a dynamic point cloud model with time sequence marks (W-model), combines the pose of the scanner to achieve precise spatial alignment and time-weighted error compensation of multi-period point clouds, effectively solves the problems of pose drift and time asynchrony, adopts spatial block caching and multi-layer layer fusion rendering technology, realizes the efficient management and VR real-time visualization of large-scale dynamic point clouds, and automatically identifies and plans the path of the missed scanning area based on the point cloud coverage rate, greatly improving the supplementary scanning efficiency. Description of the Drawings

[0013] Figure 1 is the main flow chart of the present invention; Figure 2 is the flow chart of Step 1 of the present invention; Figure 3 is the flow chart of Step 2 of the present invention; Figure 4 is the flow chart of Step 3 of the present invention; Figure 5 is the flow chart of Step 4 of the present invention; Figure 6 is the flow chart of Step 5 of the present invention; Figure 7 is the flow chart of Step 6 of the present invention. Detailed implementation manners

[0014] A real-time reverse scanning guiding method shown in FIGS. 1-7 includes the following steps: Step 1: Use a handheld 3D scanner to scan the surface of the workpiece, and collect the spatial coordinate information and attitude information of the point cloud in real time; Each point cloud data records its corresponding collection time, and attach a time tag to each point according to the collection time to construct an original point cloud data set; When performing Step 1, it specifically includes the following steps: Step 1-1: Use a laser scanner to scan the point cloud data on the surface of the workpiece, record the spatial coordinates and collection time of each point, and generate a point cloud set for each frame according to the spatial coordinates of each point , and attach a time tag to each point cloud according to the collection time ; The handheld 3D scanner performs continuous scanning sampling at a fixed frequency , and the number of laser points emitted by the handheld 3D scanner per second can be calculated according to the fixed frequency ; ; The point cloud set collected in the i-th frame is expressed as: ; Attach a time tag to each sampling point (i.e., point cloud) : ; wherein, represents the starting sampling time of the i-th frame, represents the delay of the sampling time of the j-th point relative to the starting sampling time, .

[0015] Step 1-2: According to the point cloud set Attach a time tag to each sampling point , generating an original point cloud data set : ; where M is the total number of frames, represents the three-dimensional coordinates of the j-th point in the i-th frame, , , and N represents the number of point clouds in the i-th frame.

[0016] In this embodiment, the original point cloud data set contains the spatial coordinates and precise time information of the point cloud, which can provide a data source for the subsequent modeling of the three-dimensional main model and the W-model.

[0017] Step 2: After obtaining the original point cloud data set and performing preprocessing, process the point cloud in the original point cloud data set through a voxel grid filtering algorithm + triangular mesh reconstruction algorithm to generate a three-dimensional main model with a continuous surface; When performing Step 2, it specifically includes the following steps: Step 2-1: Perform preprocessing on the original point cloud data set to remove invalid points, isolated points, and abnormal noises; In this embodiment, the statistical outlier filtering method is used to perform preprocessing on the original point cloud data set , including for each point, calculating the average distance μ and standard deviation σ of all points within its k neighborhoods. If the average distance of a certain point exceeds the range, it is identified as an abnormal outlier and removed; k represents the number of points within the neighborhood radius, taking values from 10 to 50, represents the outlier determination tolerance coefficient, taking values from 1 to 2.

[0018] The statistical outlier filtering method is an existing technology, so it will not be described in detail.

[0019] Step 2-2: Sparsify the point cloud through a voxel grid filtering algorithm; Specifically, divide the space into three-dimensional voxel grids with side length r, retain the centroid of all points within each voxel, and use it as the representative point. The calculation formula is as follows: ; where, represents the coordinate of the representative point of voxel v; represents the number of points falling into voxel v, represents the j-th point in the voxel.

[0020] The result after sparsification is represented as a sparse point cloud : ; r represents the voxel side length.

[0021] Step 2-3: Perform structured modeling on the sparse point cloud through the triangular mesh reconstruction algorithm to generate a three-dimensional main model of a continuous surface ; In this embodiment, the Poisson reconstruction method is specifically used to generate a closed mesh surface, and then a three-dimensional main model of a continuous surface is generated. The Poisson reconstruction method is an existing technology, so it will not be described in detail.

[0022] Step 2-4: Output the three-dimensional main model .

[0023] In this embodiment, the three-dimensional main model is represented as: ; where , V represents the vertex set, and the vertices are generated from the representative points of the voxels v; , F represents the face set, represents a triangular face, and both nn and mm represent the total number.

[0024] Step 3: By fusing the real-time six-degree-of-freedom pose of the handheld 3D scanner with the original point cloud data, use the homogeneous transformation matrix to unify each frame of point cloud to the world coordinate system; use a preset time interval to construct a set of standard time tags, align the timestamp of each point to the nearest neighbor standard tag, and generate an enhanced point cloud data set; perform time-block structure processing on the enhanced point cloud data set, divide the subsets in the block structure in chronological order, and generate a W-model model with time tags; When performing Step 3, it specifically includes the following steps: Step 3-1: Obtain the original point cloud data set , and at the same time obtain the pose information; Step 3-2: Obtain the six-degree-of-freedom pose of each frame collected by the handheld 3D scanner through the spatial tracking system, generate a homogeneous transformation matrix, and convert each point in the homogeneous transformation matrix to the unified world coordinate system; In this embodiment, after the handheld 3D scanner scans the surface of the workpiece, the current pose of the workpiece in the global coordinate system can be obtained in real time : ; where represents the rotation matrix at the i-th frame acquisition, represents the translation vector at the i-th frame acquisition.

[0025] Current current pose It is a six-degree-of-freedom pose.

[0026] The clock of the handheld 3D scanner is synchronized with that of the spatial tracker.

[0027] The current pose of each frame of the handheld 3D scanner is obtained through the spatial tracking system , and then is represented as a homogeneous transformation matrix : ; where represents the transformation from the coordinate system of the handheld 3D scanner to the global world coordinate system; is the special Euclidean group in three dimensions, representing all possible rotation + translation combinations in three-dimensional space, ensuring that the shape and size remain unchanged.

[0028] Each point in the original point cloud data set is transformed into the unified world coordinate system, and the specific formula is as follows: ; represents the point after the unified world coordinate system.

[0029] Step 3-3: Construct a standard set of time tags, and define a standard time tag every fixed time in it; In this embodiment, in order to unify the timing structure, a method of introducing a standard set of time tags is adopted to normalize the time tags. The standard set of time tags is as follows: ; where represents the standard time tag, and k takes values from 0 to nn. In this embodiment, the standard time tag is a discrete time slice, specifically defined as a standard time tag every 50 ms.

[0030] Step 3-4: Calculate the standard time tag closest to the time tag of each point to generate a new enhanced point cloud data set; In this embodiment, through the following formula, calculate the time tag of each point closest to the standard time tag ; Summarize the calculated results to form a new enhanced point cloud data set : ; where is the world coordinate of the point, is the adjacent standard time tag obtained after normalization processing, is the homogeneous transformation matrix, representing the real-time pose.

[0031] Step 3-5: Construct the enhanced point cloud dataset into a time-block structure composed of multiple subsets, sort each subset in chronological order, and obtain a structured point cloud model with time tags, namely the W-model; In this embodiment, all points are sorted according to the standard time tag for clustering to form a time-block structure : ; Among them, is a subset in the enhanced point cloud dataset , k represents the subset number, and nn represents the total number of subsets.

[0032] ; After sorting each subset in chronological order, the W-model is obtained.

[0033] In this embodiment, the purpose of constructing the structured point cloud model W-model with time tags is to retain the temporal evolution characteristics of the point cloud during the scanning process.

[0034] The W-model not only retains the spatial position of each point but also synchronously records its acquisition time and the corresponding scanner pose information, supporting subsequent operations such as time correction, error compensation, and path tracing.

[0035] Step 4: Use the geometric center alignment method and the ICP algorithm to perform spatial alignment on the three-dimensional main model and the W-model, calculate the alignment error by constructing an error compensation function, convert the alignment error into a visual color to obtain a color code, and output the aligned three-dimensional main model and W-model, and at the same time output the color code; When performing Step 4, it specifically includes the following steps: Step 4-1: Obtain the three-dimensional main model, the W-model, the homogeneous transformation matrix, and the standard set of time tags; specifically, the three-dimensional main model , the W-model, the homogeneous transformation matrix and the standard set of time tags ; Step 4-2: Calculate the geometric center of all vertices of the three-dimensional main model and the center of all points of the W-model respectively; by calculating the difference between and , the translation compensation vector is obtained Translate all points of the W-model to the 3D main model for center alignment; In this embodiment, the 3D main model The geometric center of all vertices Is calculated by the following formula: ; Where N represents the total number of vertices, Represents the vertices in the 3D main model .

[0036] The center of all points of the W-model Is calculated by the following formula: ; Where Represents the total number of subsets of the W-model, nk represents the number of point clouds in the k-th subset, Represents the coordinates of the j-th point in the k-th subset.

[0037] Translation compensation vector The calculation formula is as follows: ; Step 4-3: Use the weighted ICP algorithm to solve the optimal matching result based on the processing result of Step 4-2 , where Represents rotation, Represents translation; In this embodiment, the formula of the weighted ICP algorithm is as follows: ; Where Is the i-th point in the W-model; The 3D main model And The closest corresponding point; Is the weight: ; Where Is the time tag of point , Is the current reference time, Is the time tolerance window.

[0038] Step 4-4: Construct an error compensation function to correct the alignment error, and convert the error value into a visual color through the color mapping function ColorMap; The specific formula of the error compensation function is as follows: ; Among them, is the residual error of the i-th point, representing the spatial deviation between the W-model and the corresponding point of the three-dimensional main model; represents the three-dimensional coordinates of the i-th point in the W-model; represents the corresponding three-dimensional main model of the point.

[0039] Step 4-5: Output the aligned W-model and the three-dimensional main model, and at the same time output the error value of each point and its corresponding color coding.

[0040] Step 5: Perform block caching processing on the aligned W-model; Render the W-model and the three-dimensional main model, and push the rendered layer result to the VR device; When performing Step 5, it specifically includes the following steps: Step 5-1: Perform block caching processing on the aligned W-model; In this embodiment, when the W-model is block-cached, its structure is as follows: [Meta-information] + [Time tag index] + [Spatial data block set] + [Operation log]; Among them, the [Spatial data block set] includes multiple [Spatial blocks], and the structure of each [Spatial block] is as follows: [Block unique identifier] + [Point cloud data] + [Time tag] + [Version tracking identifier] + [Alignment error]; Among them, [Meta-information] is the basic information of the block, which is the name or number of the subset; [Time tag index] is an index established through time tags. In this embodiment, it is possible to quickly locate the corresponding spatial block according to the time range, which is convenient for historical point cloud backtracking, time-series animation, and dynamic error compensation; [Operation log] is a record of the user operation history of each spatial block, such as user ID, operation time, operation block ID, operation type (measurement, annotation, etc.).

[0041] [Block unique identifier], which is the ID of the spatial block; [Point cloud data] is a subset in the enhanced point cloud dataset ; [Version tracking identifier] is a version tracking identifier for multi-user collaboration.

[0042] Step 5-2: Adopt a multi-layer management method to render the W-model and the 3D main model respectively. Through transparency superposition and depth sorting, the superposition and fusion display of the two models are realized, and according to the color coding, the point cloud deviation is displayed in the layer to generate the rendered layer result; Step 5-3: Push the rendered layer result to the VR device.

[0043] Step 6: Compare the W-model with the CAD model of the preset workpiece. According to the coverage rate of the point cloud on the surface mesh of the CAD model, identify the missed scanning area. According to the position of the missed scanning area, generate a path planning. Render the lines of the path planning into the 3D main model according to the rendering method in Step 5 and send it to the VR device; When executing Step 6, it specifically includes the following steps: Step 6-1: Count the coverage rate of the point cloud on the surface mesh of the CAD model to obtain the coverage rate of each grid unit, and count the number of point clouds actually falling into each grid unit. When the coverage rate of a certain grid is less than the preset threshold, it is determined that the area where the grid is located is the missed scanning area; Step 6-2: Superimpose a high-brightness color mask on the grid units of the missed scanning area; In this embodiment, in the rendering layer of the 3D main model, a high-brightness color mask is added to all the grid areas of the missed scanning area, and then the surface of the missed scanning area is covered with red or other eye-catching colors to complete the high-brightness display.

[0044] Step 6-3: According to the position of the missed scanning area, plan the next scanning path to generate a path planning; In this embodiment, when planning the next scanning path, first extract the geometric centers of all the missed scanning grid units to form a region set; Combined with the current posture of the handheld 3D scanner , taking the current posture position as the starting point, use the cost function to calculate the sequence of path points. The cost function is as follows: ; ; Among them, the cost function represents the optimal path; represents the sequence of path points, that is, starting from the current scanner position, passing through several path points in sequence can finally cover the missed scanning area; n represents the total number of path points, represents two consecutive points; Step 6-4: Generate path lines according to the path planning, render the path lines onto the 3D main model according to the rendering method in Step 5, and push them to the VR device.

[0045] In this embodiment, a system architecture of master server + VR + spatial tracking system + handheld scanner can be adopted. The master server is responsible for core computing tasks such as processing and storing large-scale point cloud data, constructing W-model, error registration, and path planning to ensure data consistency and processing efficiency; the VR device is mainly responsible for real-time rendering and interactive display of 3D models, and the handheld scanner is responsible for data acquisition and uploading the original point cloud and pose information to the master server; the spatial tracking system Tracker is used to obtain the six-degree-of-freedom pose information of the handheld 3D scanner in space in real time, and data is transmitted between Tracker and the master server in real time through WLAN.

[0046] Devices can communicate with each other through a local area network. The master server centrally processes and coordinates the work processes of each module, and pushes the processing results (such as rendering layers, path guidance) to the VR device.

[0047] A real-time reverse scanning guidance method described in the present invention solves the technical problems of time sequence synchronization of multi-period dynamic point clouds, spatial precise registration and error compensation, and efficient block caching rendering during scanning and modeling. The present invention introduces a dynamic point cloud model with time sequence markers (W-model), combines the pose of the scanner to achieve precise spatial alignment and time-weighted error compensation of multi-period point clouds, effectively solves the problems of pose drift and time asynchrony, adopts spatial block caching and multi-layer layer fusion rendering technology to achieve efficient management and VR real-time visualization of large-scale dynamic point clouds, and automatically identifies and plans paths for missed scanning areas based on point cloud coverage, greatly improving the efficiency of supplementary scanning.

Claims

1. A real-time reverse scanning guidance method, characterized in that: It includes the following steps: Step 1: Use a handheld 3D scanner to scan the surface of the workpiece, and collect the spatial coordinate information and pose information of the point cloud in real time; each point cloud data records its corresponding acquisition time, and an additional time tag is attached to each point according to the acquisition time to construct an original point cloud data set; Step 2: After obtaining the original point cloud data set and preprocessing it, process the point cloud in the original point cloud data set through a voxel grid filtering algorithm + triangular mesh reconstruction algorithm to generate a 3D main model with a continuous surface and output it; Step 3: By fusing the real-time six-degree-of-freedom pose of the handheld 3D scanner and the original point cloud data, use the homogeneous transformation matrix to unify each frame of point cloud to the world coordinate system; Use a preset time interval to construct a set of standard time tags, align the time stamp of each point to the nearest neighbor standard tag, and generate an enhanced point cloud data set; Perform time block structure processing on the enhanced point cloud data set, divide the subsets in the block structure in chronological order, and generate a time-tagged W-model model; Step 4: Use the geometric center alignment method and the ICP algorithm to perform spatial alignment on the 3D main model and the W-model model, calculate the alignment error by constructing an error compensation function, convert the alignment error into a visual color to obtain a color code, output the aligned 3D main model and W-model model, and at the same time output the color code; Step 5: Perform block caching processing on the aligned W-model model; Render the W-model model and the 3D main model, and push the rendered layer result to the VR device; Step 6: Compare the W-model model with the CAD model of the preset workpiece, identify the missed scanning area according to the coverage rate of the point cloud on the surface mesh of the CAD model, generate a path plan using the cost function according to the position of the missed scanning area, and render the lines of the path plan into the 3D main model according to the rendering method in Step 5 and send it to the VR device.

2. The real-time reverse scanning guidance method according to claim 1, characterized in that: When performing Step 1, it specifically includes the following steps: Step 1-1: Use a laser scanner to scan the point cloud data of the workpiece surface, record the spatial coordinates and acquisition time of each point, and generate a point cloud set for each frame according to the spatial coordinates of each point , and attach a time tag to each point cloud according to the acquisition time ; Step 1-2: According to the point cloud set and attach a time tag to each sampling point , generate the original point cloud data set : ； where M is the total number of frames, represents the three-dimensional coordinates of the j-th point in the i-th frame, , , and N represents the number of point clouds in the i-th frame.

3. A real-time reverse scanning guidance method according to claim 1, characterized in that: When performing Step 2, it specifically includes the following steps: Step 2-1: Preprocess the original point cloud dataset to remove invalid points, isolated points, and abnormal noise; Step 2-2: Sparsify the point cloud through a voxel grid filtering algorithm; Step 2-3: Perform structured modeling on the sparse point cloud through a triangular mesh reconstruction algorithm to generate a three-dimensional main model of a continuous surface ; Step 2-4: Output the 3D main model .

4. A real-time reverse scanning guidance method according to claim 1, characterized in that: When performing Step 3, it specifically includes the following steps: Step 3-1: Obtain the original point cloud data set , and obtain the pose information at the same time; Step 3-2: Obtain the six-degree-of-freedom pose of each frame collected by the handheld 3D scanner through a spatial tracking system, generate a homogeneous transformation matrix, and convert each point in the homogeneous transformation matrix to the unified world coordinate system; Step 3-3: Construct a set of standard time tags, and define a standard time tag every fixed time in it; Step 3-4: Calculate the standard time tag closest to the time tag of each point to generate a new enhanced point cloud data set; Step 3-5: Construct the enhanced point cloud data set into a time block structure composed of multiple subsets, sort each subset in chronological order to obtain a time-tagged structured point cloud model, that is, the W-model model.

5. The real-time reverse scanning guidance method according to claim 1, characterized in that: When performing Step 4, it specifically includes the following steps: Step 4-1: Obtain the 3D main model, the W-model model, the homogeneous transformation matrix, and the set of standard time tags; Step 4-2: Calculate the geometric centers of all vertices of the 3D main model and the center of all points of the W-model ; By calculating and the difference, the translation compensation vector is obtained, and all points of the W-model are translated onto the 3D main model for center alignment; Step 4-3: Using the weighted ICP algorithm, solve the optimal matching result based on the processing result of Step 4-2; Step 4-4: Construct an error compensation function to calculate the alignment error of each point, convert the error value into a visual color through the color mapping function ColorMap, and generate a color code; Step 4-5: Output the aligned W-model and the three-dimensional main model, and at the same time output the alignment error of each point and its corresponding color code.

6. The real-time reverse scanning guidance method according to claim 1, characterized in that: When performing Step 5, it specifically includes the following steps: Step 5-1: Perform block caching processing on the aligned W-model; when the W-model is block-cached, its structure is as follows: [Meta-information] + [Time label index] + [Spatial data block set] + [Operation log]; Among them, the [Spatial data block set] includes multiple [Spatial blocks], and the structure of each [Spatial block] is as follows: [Block unique identifier] + [Point cloud data] + [Time label] + [Version tracking identifier] + [Alignment error]; Step 5-2: Adopt a multi-layer management method to render the W-model and the three-dimensional main model respectively, and through transparency superposition and depth sorting, realize the superposition and fusion display of the two models, and display the point cloud deviation in the layer according to the color code to generate a rendered layer result; Step 5-3: Push the rendered layer result to the VR device.

7. The real-time reverse scanning guidance method according to claim 1, characterized in that: When performing Step 6, it specifically includes the following steps: Step 6-1: Statistically calculate the coverage rate of the point cloud on the surface mesh of the CAD model to obtain the coverage rate of each grid unit, and count the number of point clouds actually falling into each grid unit. When the coverage rate of a certain grid is less than the preset threshold, it is determined that the area where the grid is located is a missed scanning area; Step 6-2: Superimpose a highlighted color mask on the grid units in the missed scanning area; Step 6-3: According to the position of the missed scanning area, plan the next scanning path to generate a path plan; when planning the next scanning path, first extract the geometric centers of all missed scanning grid units to form a region set; Combined with the current pose of the handheld 3D scanner , using the current pose position as the starting point, a cost function is used to calculate a sequence of path points. The cost function is as follows: ； ； Among them, the cost function represents the optimal path; represents the sequence of path points, that is, starting from the current scanner position, passing through several path points in sequence can finally cover the missed scanning area; n represents the total number of path points, represents two consecutive points; Step 6-4: Generate path lines according to the path plan, render the path lines onto the three-dimensional main model according to the rendering method in Step 5, and push them to the VR device.

Images