Scanning path planning and missing scanning compensation method based on space tracking

Abstract

The invention discloses a scanning path planning and missing scanning compensation method based on space tracking. The method comprises the following steps: acquiring pose data of a space tracker fixedly connected to a handheld three-dimensional scanner in real time, and synchronously acquiring three-dimensional point cloud data; performing coordinate transformation on the data, calculating to obtain a comprehensive quality weight, endowing each three-dimensional point with the quality weight, constructing a three-dimensional coverage map in real time, and identifying a real scanning missing area through a spatial data structure and a clustering algorithm; constructing a mixed cost function of the operation human engineering cost and the scanning task efficiency cost for the identified scanning missing area, and dynamically planning an optimal scanning supplementing path; the optimal path is visualized by a three-dimensional virtual path band and a holographic scanner model through virtual reality equipment, and an operator is guided to efficiently and accurately complete supplementary scanning in an immersive manner; according to the method, through objective missing scanning identification, optimal path planning and immersive operation guidance, the overall operation efficiency of scanning data by an operator is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional measurement and virtual reality technology, and in particular to a scanning path planning and missed scanning compensation method based on space tracking. Background Art

[0002] As a crucial bridge between the physical and digital worlds, 3D digitization technology plays a vital role in fields such as reverse engineering, industrial design, quality inspection, cultural heritage preservation, and virtual reality. Handheld 3D scanning systems, exemplified by laser triangulation or structured light technology, have become the mainstream tool for acquiring 3D morphological information about complex curved objects due to their high precision, efficiency, and portability. To overcome the measurement range limitations of traditional fixed scanners and enable flexible scanning of certain workpieces (such as automotive bodies-in-white, aircraft engine blades, and wind turbine blades), the industry is widely adopting a solution that combines handheld scanners with external, large-scale, high-precision spatial tracking systems. This solution uses real-time tracking of optical markers fixed to the scanner to accurately determine the scanner's position in a global coordinate system. This allows for seamless and automatic registration of local point cloud data collected from various viewpoints to a unified coordinate system, eliminating tedious post-processing and stitching steps, significantly improving data acquisition flexibility and overall efficiency.

[0003] However, while these technologies have enhanced the flexibility and automation of scanning operations, inherent flaws in existing scanning processes are becoming increasingly apparent in practical applications, particularly when dealing with large workpieces with complex structures and numerous features. First, current mainstream scanning processes rely heavily on the operator's personal experience and subjective judgment. Operators must constantly divide their attention between observing the physical workpiece, maintaining the scanner's proper posture, and reviewing the point cloud model generated on the computer screen. This disconnected "human-machine-object" interaction model results in a high cognitive load. Missed scans often result from visual fatigue, distraction, or blind spots caused by geometric obstructions in the workpiece. The identification and compensation process lacks objective standards and systematic guidance. Second, existing technologies lack path planning for compensating for missed scans. When operators discover a missing area, they often choose a random and suboptimal path to re-scan. They intuitively move the scanner to the missing area without considering the time cost, operational smoothness, or whether the scanning posture upon arrival at the target point is optimal for observing the features in that area. This blindness not only reduces re-scanning efficiency but also can lead to low-quality data due to improper scanning angles or distances, compromising the accuracy of the final model. Furthermore, while existing software systems can provide preliminary visualization of point cloud coverage (e.g., pseudo-color rendering), this analysis is superficial at the topological level and cannot intelligently distinguish between true missed scans due to complex internal structures (e.g., deep holes and grooves) and external boundaries of the model that have not been covered by the scanning process. This topological ambiguity often leads to ineffective or incorrect guidance for operators, disrupting the normal scanning process.

[0004] CN109813219B discloses a method and system for collecting and processing information on the inspection, identification and reinforcement of existing structures. This solution mainly uses real-time rendering of point clouds and overlays them with VR scenes to reduce multiple on-site supplementary measurements and surveys. At the same time, it reduces model reconstruction work, optimizes reinforcement design work, makes construction technology and organization more reasonable, and thus reduces rework. However, it does not consider the smoothness of the scanning path and the optimality of the scanning posture.

[0005] CN104137030A discloses a method for measuring a three-dimensional sample using a measuring device including a laser scanning microscope, and the measuring device. This solution primarily utilizes a three-dimensional virtual reality device to generate a three-dimensional virtual space of the measurement space, allowing operations to be selected in the virtual space and providing a real-time one-way or two-way connection between the measurement space and the virtual space, so that operations selected in the virtual space are performed in the measurement space, and data measured in the measurement space is displayed in the virtual space. Although the combination of three-dimensional scanning and virtual reality technology is mentioned, it is mainly used to achieve virtual reality interaction during microscopic measurement and does not involve scanning path guidance operations for large objects. Summary of the Invention

[0006] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0007] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a scanning path planning and missed scan compensation method based on space tracking to solve the problems mentioned in the background technology.

[0008] To solve the above technical problems, the present invention provides the following technical solution: a scanning path planning and missed scan compensation method based on spatial tracking, comprising: Acquire the position data of the spatial tracker attached to the handheld 3D scanner in real time, and simultaneously collect the 3D point cloud data output by the 3D scanner; Performing coordinate transformation on the three-dimensional point cloud data and the pose data, and calculating a comprehensive quality weight for each three-dimensional point in the transformed point cloud data set according to a preset quality assessment model; Based on the point cloud data set with the comprehensive quality weight, a three-dimensional coverage map is constructed in real time to identify missed scan areas; For the identified missed scanning area, combined with the current position of the scanner, a hybrid cost function is constructed to dynamically plan the optimal scanning path; The optimal re-scanning path is used through a virtual reality device to guide the operator to perform re-scanning in a visual manner.

[0009] As a preferred solution of the scanning path planning and missed scan compensation method based on spatial tracking of the present invention, the preset quality assessment model at least includes a combination of one or more of the following weights: Angle weight determined by the angle between the scanner's line of sight and the local normal of the target surface; a distance weight determined by the distance between the scanner and the target surface; a tracker confidence weight determined by the confidence of the pose data output by the spatial tracker; The local density weight is determined by the local point cloud density of the 3D point in its original point cloud frame.

[0010] As a preferred solution of the scanning path planning and missed scanning compensation method based on spatial tracking of the present invention, constructing a three-dimensional coverage map to identify missed scanning areas includes: The three-dimensional coverage map is constructed using a hierarchical spatial data structure to distinguish between areas inside the workpiece model and areas outside the workpiece model, and missed scan areas are identified using a density-based spatial clustering algorithm.

[0011] As a preferred solution of the scanning path planning and missed scanning compensation method based on spatial tracking described in the present invention, the dynamically planned optimal scanning path includes adopting a path planning algorithm based on random sampling and performing priority sampling in the missed scanning area.

[0012] As a preferred solution of the scanning path planning and missed scan compensation method based on spatial tracking described in the present invention, the hybrid cost function is obtained by the operation ergonomic cost and the scanning task efficiency cost.

[0013] As a preferred solution of the scanning path planning and missed scan compensation method based on spatial tracking described in the present invention, the operation ergonomic cost is obtained by a second-order finite difference method.

[0014] As a preferred solution of the scanning path planning and missed scan compensation method based on spatial tracking described in the present invention, the scanning task performance cost is obtained by comprehensively considering the visibility cost and the field of view center cost.

[0015] As a preferred solution of the scanning path planning and missed scan compensation method based on spatial tracking of the present invention, wherein: guiding the operator to perform missed scans in a visual manner includes: The optimal patching path is rendered as a three-dimensional virtual path belt.

[0016] As a preferred solution of the scanning path planning and missed scan compensation method based on space tracking of the present invention, it also includes: On the optimal re-scanning path, a holographic scanner model is rendered at a predicted position in front of the operator's current position to indicate the target posture.

[0017] As a preferred solution of the scanning path planning and missed scan compensation method based on spatial tracking of the present invention, the method further includes an initialization calibration step before execution, which includes: Calculating a fixed transformation relationship between the three-dimensional scanner coordinate system and the spatial tracker coordinate system through a hand-eye calibration method; The fixed transformation relationship between the virtual reality device coordinate system and the tracking marker point coordinate system fixed thereto is calculated through the head display device calibration method.

[0018] Compared with the prior art, the invention has the following beneficial effects: 1. By constructing a 3D coverage map in real time, it can instantly and objectively identify areas that were missed due to geometric occlusion or improper operation, avoiding secondary scans and fundamentally resolving the issue of delayed information feedback, thereby improving the success rate and data integrity of single scans. 2. By constructing a hybrid cost function combining scanning task efficiency and operator ergonomics, the optimal re-scanning path is dynamically planned. This ensures not only the shortest path and smooth, effortless operation, but also the optimal scanning posture upon reaching the target location. This guarantees the quality of the re-scanning data and reduces reliance on operator skills. 3. Using virtual reality equipment, the planned three-dimensional path and other guidance information are directly superimposed on the operator's field of view, realizing immersive guidance where what you see is what you get. The operator no longer needs to frequently switch his or her gaze between the physical workpiece and the external display screen, which greatly reduces the operator's cognitive load, further shortens the operation time, and improves the fluency and efficiency of the overall scanning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them: Figure 1 This is an overall flow chart of a scanning path planning and missed scan compensation method based on spatial tracking according to an embodiment of the present invention.

[0020] Figure 2 A schematic diagram of missed scan identification under a three-dimensional coverage map of a scanning path planning and missed scan compensation method based on spatial tracking according to an embodiment of the present invention; Figure 3 This is a gesture prompt diagram of a scan path for a scan path planning and missed scan compensation method based on spatial tracking according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0022] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0023] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0024] The present invention is described in detail with reference to schematic diagrams. For ease of illustration, cross-sectional views of device structures may be partially enlarged and not to scale when describing embodiments of the present invention. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0025] In the description of the present invention, it should be noted that the terms "upper, lower, inner, and outer" and other references to orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0026] In this disclosure, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be interpreted broadly. For example, they may refer to fixed, removable, or integral connections. They may also refer to mechanical, electrical, or direct connections, indirect connections through an intermediary, or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this disclosure.

[0027] Example 1 Reference Figures 1 to 3 , which is the first embodiment of the present invention, provides a scanning path planning and missed scan compensation method based on space tracking, comprising: S1. Acquire the position data of the spatial tracker attached to the handheld 3D scanner in real time, and simultaneously collect the 3D point cloud data output by the 3D scanner; It should be noted that before the scanning operation begins, the rigid body transformation calibration between the coordinate systems needs to be completed to ensure the accuracy of data fusion; Furthermore, a fixed transformation relationship between the 3D scanner coordinate system and the spatial tracker coordinate system is calculated through a hand-eye calibration method; Specifically, a calibration plate for the hand-eye calibration method is created. The calibration plate uses a 10×10 checkerboard calibration plate (where each grid has a side length of 20 mm, and is made of an aluminum alloy substrate with a high-contrast coating). It is fixed in the world coordinate system W, and the pose is expressed as ; It should be noted that the pose in the solution of the present invention includes a translation part and a rotation part, that is, it is composed of a three-dimensional coordinate vector and a unit quaternion respectively; Specifically, the operator scans the calibration plate from 12 different angles (position interval > 0.5m, rotation angle > 30°) using a handheld scanner. Each scan lasts 1 to 2 seconds. During the scanning process, the position of the calibration plate in the scanner coordinate system S is recorded. and the pose of the spatial tracker in the world coordinate system W , where the pose You can detect the corner positions and poses of the chessboard by calling the findChessboardCorners function in OpenCV It is obtained in real time by the space tracking system, which also includes a space tracker coordinate system T; Furthermore, with the above conditions, let the position of the calibration plate in the world coordinate system W be (fixed), from two different scanning processes i and j, the following transformation chain is obtained: in, That is, the assumed fixed change relationship, by combining the above two formulas, by eliminating ,get: make , then the hand-eye calibration equation is obtained ; Specifically, collect at least three sets of non-coplanar pose pairs (i.e. and ), solve the optimal X of the hand-eye calibration equation by quaternion method and store it. The optimal X is ; Furthermore, the fixed transformation relationship between the virtual reality device coordinate system and the tracking marker coordinate system fixed to it is calculated through the head display device calibration method; Specifically, the same calibration plate is used as in the hand-eye calibration method above. The head-mounted display device uses a built-in camera to identify the checkerboard on the calibration plate. During the scanning process, the pose of the tracking marker coordinate system and the pose of the head-mounted display device coordinate system are simultaneously recorded. By establishing a transformation chain, the head-mounted display device calibration equation is obtained, and X is solved and stored. The fixed transformation relationship between the virtual reality device coordinate system and the tracking marker coordinate system fixed to it is obtained. The specific processing steps are similar to those of the hand-eye calibration method above and will not be repeated here. In addition, the Network Time Protocol (NTP) or the more precise time protocol (PTP) is used to synchronize the time between the host in the space tracking system and the control host in the scanner, ensuring that the error of the timestamps of the two is controlled within milliseconds. Specifically, after obtaining time synchronization, the spatial tracking system outputs the position and posture of the spatial tracker at a high frequency (120Hz) to obtain the position and posture data , the scanner outputs point cloud frames at a low frequency (15Hz) , about 10 per frame 5 points, of which Represented as pose timestamp, Represented as point cloud timestamp, It is represented as a point cloud set in the scanner coordinate system S; S2. performing coordinate transformation on the 3D point cloud data and the pose data, and calculating a comprehensive quality weight for each 3D point in the transformed point cloud data set according to a preset quality assessment model; It should be noted that for each output point cloud frame, its timestamp Usually not associated with any pose timestamp Completely overlap, Therefore, it is necessary to find the timestamp of the point cloud in the pose data. The two closest poses and ; Furthermore, for coordinate transformation, spherical linear interpolation (SLERP) and linear interpolation (Lerp) are used to interpolate the rotation part (i.e., the rotation relationship of the object's own coordinate system relative to the world coordinate system) and the translation part (the specific coordinates (x, y, z) of the object in space) respectively; Specifically, spherical linear interpolation is expressed as follows: in, Expressed as The rotational part of the time-stamped spatial tracker pose is a quaternion represented by (satisfy ), similarly, Expressed as The rotational part of the time-stamped spatial tracker pose; Expressed as an interpolation coefficient in the range [0,1], used for timestamp In the timestamp interval The relative position in reflects the interpolation ratio of the point cloud timestamp to the pose timestamp; The result of spherical linear interpolation, indicating the timestamp The rotation quaternion of ; Specifically, linear interpolation is expressed as follows: in, and With the above and Similarly, respectively, Timestamp and The translation part of the spatial tracker pose is a translation vector, which is expressed as , through the pose The translation component of is directly extracted, and The same is true for the translation vector; Is the linear interpolation result, indicating the timestamp The translation vector of Specifically, the above Convert to a rotation matrix and compare Combined, the updated pose is ,in Represented as a rotation matrix of the transformation; Furthermore, Transformed to the world coordinate system, we get: in, Expressed as The expression in the world coordinate system; It should be explained that different interpolation methods are used because the translation part is usually linear and Euclidean, while the rotation part is usually nonlinear and spherical. If linear interpolation is directly used for the rotation part, the unit length will be lost because the unit quaternion is usually represented as a pure rotation. If linear interpolation is performed on two unit quaternions, the intermediate result will not only contain the unit quaternion but also the scale. This will cause the rendered holographic scanner model to be enlarged or reduced during the operator guidance process, making the guidance process look very unnatural and easily tiring the operator. Furthermore, in order to distinguish the quality of scanning (for example, the point cloud accuracy of grazing angle is lower), for each transformed 3D point cloud A quality weight is added, and a comprehensive quality weight is calculated using a preset quality assessment model. The model is preferably a weighted combination of the following four weights: (1) Considering the angle weight, which is obtained by the angle between the scanner's line of sight and the local normal direction of the target surface; Specifically, through The principal component analysis (PCA) of the surface and its neighboring points is performed to estimate the local normal vector of the surface in real time, and then the line from the optical center of the scanner to In addition, the angle weight is defined as the cosine function of the angle between the local normal vector and the view vector, resulting in: in, Expressed as angle weight, Expressed as the angle of the sight vector, is represented as the local normal vector, Expressed as the sight vector, , , Indicates the position of the scanner's optical center in the world coordinate system; Specifically, real-time estimation is achieved by creating a NormalEstimation object in the PLC library and calling computeFromNormalVectorField; (2) Consider the distance weight, which is obtained from the distance between the scanner and the target surface; Specifically, the scanning distance is based on the line of sight vector and takes into account the effective scanning distance of the target surface, and is obtained as follows: in, Expressed as distance weight, Expressed as the scanning distance, ; (3) Considering the confidence weight of the spatial tracker, the confidence of the pose data output by the spatial tracker is obtained; Specifically, since the spatial tracking system usually adds a confidence level when outputting the spatial tracker pose data to reflect the reliability of the current pose data, the confidence level can be directly expressed as the spatial tracker confidence weight. ,in Denoted as the spatial tracker confidence weight, Expressed as the confidence of the pose data of the spatial tracker; (4) Considering the local density weight, it is obtained by the local point cloud density of the 3D point in its original point cloud frame; Specifically, within the original point cloud frame, the point cloud density in its surrounding neighborhood is considered. The higher the point cloud density, the more reliable the 3D reconstruction quality of the point cloud frame. get: in, Expressed as the local density weight, Expressed as 's nearest neighbors; Represented as a set of neighboring points; Expressed as the number of neighboring points; Furthermore, the comprehensive quality weight is the combination of the weights in (1) to (4), which is output through the preset quality assessment model; It should be noted that the traditional method of determining whether an area has been scanned is usually to check whether there is point cloud data collected in the three-dimensional space of the corresponding area. The fatal flaw of this method is that it cannot distinguish between effective coverage and false coverage. Effective coverage means that the operator scans the area at the correct angle, at the optimal distance, and with a stable posture, and obtains a large number of accurate, low-noise points; while false coverage means that the edge of the scanner's line of sight accidentally "rubs" the area during the operator's movement, or scans at a large grazing angle (close to parallel to the surface). Although several points are obtained, the position errors of these points are extremely large and the noise level is very high, which is very difficult to use for inverse analysis. The so-called garbage data is useless for modeling or dimensional analysis. Traditional methods will mark both effective coverage and false coverage as scanned, thus assuming that the task is complete. However, in fact, the false coverage area needs to be re-scanned. The solution of the present invention considers the comprehensive quality weight and assigns a quality score to each collected 3D point cloud. This score represents the key factors affecting data quality. In this way, the accumulated data is not just the number of 3D point clouds, but the total score of the point cloud data quality. Therefore, for an area covered by garbage data, its accumulated data quality score will be very low, and it can naturally be accurately identified as a missed scan area that needs to be compensated. S3, based on the point cloud data set with comprehensive quality weights, builds a 3D coverage map in real time to identify missed areas; Furthermore, based on the weighted point cloud set, a hierarchical spatial data structure is used to construct a three-dimensional coverage map, from which the missed scanning areas that need to be compensated are accurately identified; Figure 2 , where the gray cells represent the detected missed scan areas and the grid lines represent the division of the 3D coverage map; Specifically, the hierarchical spatial data structure can use an octree as the data structure for spatial partitioning, because it can efficiently represent sparse three-dimensional environments and can significantly save memory compared to fixed-size voxel grids. Each leaf node (voxel) in the octree stores a cumulative coverage quality score, and uses a pre-loaded workpiece CAD model (STL format) to distinguish between areas where scan points are located inside the workpiece and areas on the workpiece's external boundary. Specifically, the cumulative coverage quality score is obtained by weighted summation of the number of weighted point clouds; It should be noted that for the multiple missed scan areas that need to be compensated, the coverage quality score setting is mainly used to decide which area to scan first and how to plan the scan path; Specifically, to determine missed scans, a coverage quality threshold needs to be set. This threshold is objectively set based on metrological calibration (i.e., prepare a metrological standard, fix the scanner on a bracket, scan the standard at the optimal working distance and a vertical angle close to 90 degrees as calibrated in the scanner specification, and extract the peak value of the scanning process to obtain the theoretical maximum coverage quality threshold). By traversing all leaf nodes (Voxels) in the 3D coverage map, if the coverage quality score of a leaf node is lower than the set coverage quality threshold, the leaf node is marked as having poor quality, and its center coordinates are added to a temporary set of candidate missed scan points. For each point in the candidate missed scan point set, the following judgment logic is executed: Shoot virtual rays from the point along multiple random directions (for example, 12 directions uniformly sampled on a unit sphere) and count the number of intersections between each virtual ray and the pre-loaded workpiece CAD model. If the ray intersects the model at least once in most directions, the point is considered to be "surrounded" by the model, i.e., located inside it. Otherwise, the point is considered to be outside it. In addition, for the coverage quality score, when a new weighted point cloud set arrives, each point cloud and its weight are traversed, and then the point cloud set is inserted into the octree, and the leaf node to which it belongs is located, and the coverage quality score of the leaf node is updated: Among them, CQS represents the coverage quality score, V represents the leaf node, Represented as the weight of the point cloud set; Specifically, candidate missed scan points that are determined to be located inside are treated as true missed scans and added to a new set of true missed scan points; Furthermore, the density-based spatial clustering algorithm DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used to identify discrete, spatially adjacent, true missed scan points and aggregate them into meaningful and continuous missed scan areas. The advantage of using DBSCAN is that it does not require a pre-specified number of clusters, can effectively identify clusters of any shape, and can identify sparse discrete points as noise, which is very suitable for the irregular morphology of missed scan areas. Specifically, the neighborhood radius of DBSCAN defines the distance between adjacent points. Its value needs to be set according to the size of the octet leaf node. For example, it can be set to 1.5 times the length of the leaf node diagonal. As for the minimum number of points of DBSCAN, it defines the minimum number of points required to form a region. It can be set to a smaller value (such as between 3 and 5) to ensure that even small independent missed scan areas can be successfully identified as a cluster. S4. For the identified missed scanning area, combined with the current position of the scanner, a hybrid cost function is constructed to dynamically plan the optimal scanning path; Specifically, after identifying one or more missed scan areas, the urgency of each missed scan area is calculated, that is, the ratio of the number of missed scan points in the missed scan area to the Euclidean distance from the current position of the scanner to the cluster center of the missed scan area. The urgency of each missed scan area is then sorted in descending order (from high to low), with priority given to repairing large and close missed scan areas. It should be explained that after determining the priority missed scanning areas, considering that there may be multiple missed scanning areas with the same priority, it is necessary to find the optimal re-scanning path. Traditional path planning usually only considers the shortest path or the fastest path. However, this path planning is often not applicable to handheld scanning tasks in human-machine collaboration and requires high operator skills, which not only increases operator fatigue but also reduces re-scanning efficiency. The solution of the present invention constructs a hybrid cost function to find a re-scanning path that can reduce operator fatigue while maintaining scanning efficiency. Specifically, the hybrid cost function is obtained by the operation ergonomics cost and the scanning task effectiveness cost, and the dynamic planning of the path is based on the sorted missed scanning area, using the RRT* algorithm to generate a sequence of discrete postures, also known as a waypoint set, where the first waypoint in the sequence is the current posture of the scanner, and the last waypoint in the sequence is the target posture; each waypoint contains a position vector and a posture quaternion; assuming that the time interval between waypoints is a constant, which represents the desired speed of guiding the operator's movement, then this constant can eventually be absorbed into the weight coefficient and can therefore be omitted in the operation ergonomics cost, which aims to evaluate the smoothness of the new path segment formed when connecting a new waypoint to an existing waypoint in the tree, and in order to calculate the rate of change (such as acceleration or jerk), the parent node of the waypoint is also required; Furthermore, we use the second-order finite difference method to approximate the acceleration vector of the path at the waypoint. The square of the vector modulus can be considered an excellent cost metric. A large acceleration modulus indicates a dramatic velocity change. However, for a smooth path, the velocity vector and angular velocity change slowly. Therefore, when a new waypoint is connected to an existing waypoint in the tree, only the "turn" and "attitude twist" costs generated at the existing waypoint in the tree need to be calculated. Specifically, the cost of turning is expressed as follows: in, Indicates a new waypoint. Indicates that there is a waypoint in the tree. Represents the parent node of the waypoint, Represented as a position vector; Expressed as the square of the acceleration vector modulus; Expressed as a linear motion smoothness cost function; It should be noted that if the three points (the new waypoint, the existing waypoint in the tree, and the parent node of the waypoint) are on a straight line and are equidistant, the "turn" cost value is 0, indicating the lowest cost; Specifically, the cost of "posture twist" is expressed as follows: in, Represented as attitude quaternion; It should be noted that if the attitude changes at a constant angular velocity (e.g., uniform rotation around the same axis), the “attitude twist” cost value is close to 0, indicating the lowest cost; Specifically, the above-mentioned "turning" cost and "posture twisting" cost are added together to obtain the operation ergonomic cost; In addition, if a certain posture on the path cannot see the target missed scanning area at all due to the occlusion of the workpiece itself (that is, the path collides with the obstacle (workpiece model)), then this posture is invalid for the re-scanning task. Therefore, the visibility cost of the scan needs to be considered. The visibility cost is similar to the calculation method of the angle weight and distance weight in the previous solution. It aims to ensure that the scanner and the target surface are kept at the optimal working distance, and the optical axis of the scanner is as perpendicular to the target surface as possible. On this basis, we encourage the target point to be scanned (missed scanning point) to be placed in the center area of the scanner's field of view to form the field of view center cost, which is calculated as follows: Project the target point onto the imaging plane of the scanner and calculate the Euclidean distance between the target point and the center of the imaging plane. The closer the target point is to the center of the imaging plane, the smaller the distance will be, and the lower the final cost will be. Specifically, the scanning task effectiveness cost can be obtained by weighted summing the above visibility cost and field of view center cost; S5. The optimal re-scanning path is displayed through a virtual reality device to visually guide the operator to perform the re-scanning. Furthermore, the planned optimal re-scanning path is sent to the virtual reality device in real time. The virtual reality device uses the aforementioned head device calibration method, taking the fixed transformation relationship between the virtual reality device coordinate system and the coordinate system of the tracking markers fixed to it as input, and guides the operator to perform the re-scanning in a precise and visual manner. Specifically, during this process, the optimal fill scan path is rendered as an intuitive three-dimensional virtual path belt, which is suspended in the operator's field of view and extends from the current scanner position to the target area; Specifically, the visual properties of the rendered 3D virtual path belt will dynamically change as the operator's operation progresses. For example, when the operator's scanner moves precisely along the path belt, the path belt appears green. When deviation occurs, the color changes to yellow or red, providing the operator with instant correction feedback. In addition, the texture on the path belt can flow, and its flowing speed indicates the recommended scanning speed. In addition, on the path tape, a predictive position ahead of the operator's current position (for example, a position 0.5 seconds ahead in time) is indicated by a gesture-controlled arrow (refer to Figure 3 ), which can render a semi-transparent holographic scanner model. This model is designed to accurately indicate the target posture (including position and posture) that the operator is about to reach, suggesting what posture the operator should adopt after reaching the scanning area; It should be noted that through the rendering path and the holographic scanner model, the operator can be guided immersively to improve the smoothness and efficiency of the overall scanning process.

[0028] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application may be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0029] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0030] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0031] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0032] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0033] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A scanning path planning and missed scan compensation method based on spatial tracking, characterized in that: include: Acquire the position data of the spatial tracker attached to the handheld 3D scanner in real time, and simultaneously collect the 3D point cloud data output by the 3D scanner; Performing coordinate transformation on the three-dimensional point cloud data and the pose data, and calculating a comprehensive quality weight for each three-dimensional point in the transformed point cloud data set according to a preset quality assessment model; Based on the point cloud data set with the comprehensive quality weight, a three-dimensional coverage map is constructed in real time to identify missed scan areas; For the identified missed scanning area, a hybrid cost function is constructed to dynamically plan the optimal scanning path in combination with the current position of the scanner; The optimal re-scanning path is used through a virtual reality device to guide the operator to perform re-scanning in a visual manner.

2. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 1, characterized in that: The preset quality assessment model includes at least one or more weight combinations of the following: Angle weight determined by the angle between the scanner's line of sight and the local normal of the target surface; a distance weight determined by the distance between the scanner and the target surface; a tracker confidence weight determined by the confidence of the pose data output by the spatial tracker; The local density weight is determined by the local point cloud density of the 3D point in its original point cloud frame.

3. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 1, wherein: Build a 3D coverage map to identify missed areas, including: The three-dimensional coverage map is constructed using a hierarchical spatial data structure to distinguish between areas inside the workpiece model and areas outside the workpiece model, and missed scan areas are identified using a density-based spatial clustering algorithm.

4. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 1, wherein: The dynamic planning of the optimal re-scanning path includes adopting a path planning algorithm based on random sampling and performing priority sampling in the missed-scan area.

5. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 1, wherein: The hybrid cost function is derived from the operation ergonomics cost and the scanning task performance cost.

6. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 5, characterized in that: The operational ergonomic cost is obtained by a second-order finite difference method.

7. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 5, characterized in that: The scanning task performance cost is obtained by comprehensively considering the visibility cost and the field of view center cost.

8. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 1, wherein: Visually guides the operator through the re-scan process, including: The optimal patch sweep path is rendered as a three-dimensional virtual path belt.

9. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 8, characterized in that: Also includes: On the optimal re-scanning path, a holographic scanner model is rendered at a predicted position in front of the operator's current position to indicate the target posture.

10. The scanning path planning and missed scan compensation method based on spatial tracking according to claim 1, wherein: The method also includes an initialization calibration step before execution, which includes: The fixed transformation relationship between the three-dimensional scanner coordinate system and the spatial tracker coordinate system is calculated through the hand-eye calibration method; the fixed transformation relationship between the virtual reality device coordinate system and the tracking marker point coordinate system fixed thereto is calculated through the head display device calibration method.

Images