A robotic path planning system and method based on scanner visual guidance

Abstract

The application discloses a kind of robot path planning system and method based on scanner visual guidance, method includes: three-dimensional scanner, path planning module, visual guidance module and robot;Path planning module is used to determine the optimal motion trajectory of robot based on preset scanning viewpoint;Visual guidance module is used to determine target scanning viewpoint in the optimal motion trajectory, real-time visual display is shown between the pose of the three-dimensional scanner and the pose deviation of the target scanning viewpoint and guide the three-dimensional scanner to reach specified pose;The robot has multiple degrees of freedom, for adjusting the pose of the three-dimensional scanner according to the pose deviation until coincides with the target viewpoint.The application improves the efficiency of robot path planning demonstration, improves the quality of the planned path, and does not depend on robot off-line simulation software, and does not limit specific robot type, low cost, strong universality.

Description

Technical Field

[0001] This invention relates to the field of automatic measurement technology for scanners, and more specifically to a robot path planning system and method based on scanner vision guidance. Background Technology

[0002] Automated 3D scanning measurement involves a multi-DOF robot (typically 6DOF) carrying a laser 3D scanner, moving along a planned path to scan and measure objects from multiple viewpoints, thereby obtaining complete 3D data. During the measurement process, the viewpoint directly determines the scanner's position and orientation, significantly influencing the obtained 3D data. Furthermore, the path itself, driven by the robot, determines the feasibility and efficiency of the measurement task. Many objects have complex structures, requiring numerous measurement viewpoints and complex paths due to visual occlusion, increasing measurement time. Currently, this relies on experienced humans who teach the robot based on their experience to plan the measurement viewpoint paths. This process is extremely time-consuming because humans lack reference methods and guidelines for teaching, making it difficult to accurately select the most suitable points and obtain optimal measurement viewpoints and paths. In reality, after a manual planning phase, trial runs often reveal unsatisfactory scanning results, requiring repeated adjustments to the points. This increases the number of unnecessary viewpoints and measurement time, failing to maximize the efficiency of automated measurement equipment. High-volume, high-speed online measurement is the future direction of digital factory inspection. Flow production lines have strict requirements for the debugging and inspection time of each product. Relying solely on human experience for teaching is inefficient and cannot meet the requirements of factory manufacturing.

[0003] Existing technologies aim to achieve mass automation by studying automatic viewpoint measurement and path planning. These technologies analyze and quantify the interrelationships between the workpiece reference model, measuring equipment, and measurement task requirements to establish constraints. These constraints then automatically select or create the optimal viewpoint and plan the measurement path, hoping to achieve waypoint planning in a fully automated manner. However, most of these methods remain in the research stage and are still some distance from practical application. Furthermore, automatic planning technology requires consideration of numerous factors and conditions, making research difficult. The process still requires manual intervention and demands high operator skills. Moreover, the viewpoints generated by these methods require offline simulation software for robotic arms, which is very expensive. From an energy efficiency perspective, these methods are not well-suited to the current realities of the 3D measurement industry. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned technical shortcomings and address the problem that existing path planning technologies for automated 3D scanner measurement rely too heavily on the teaching experience of professionals. This invention proposes a robot path planning system and method based on scanner vision guidance to solve the problems of inconsistent scanning quality and low efficiency of the teaching process caused by human experience. Compared with fully automated path planning methods that are still in the research stage, this method is more suitable for the needs of current industrial applications and has better practicality.

[0005] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a robot path planning system based on scanner vision guidance.

[0007] Includes: 3D scanner, path planning module, visual guidance module, and robot;

[0008] The 3D scanner is fixedly connected to the end of the robot and is used to create 3D data of the surface of the workpiece to be inspected.

[0009] The path planning module is communicatively connected to the 3D scanner and is used to determine the robot's optimal motion trajectory based on a preset scanning viewpoint.

[0010] The visual guidance module is used to determine the target scanning viewpoint in the optimal motion trajectory, visualize the pose deviation between the pose of the 3D scanner and the target scanning viewpoint in real time, and guide the 3D scanner to the designated position.

[0011] The robot has multiple degrees of freedom and is used to adjust the pose of the 3D scanner according to the pose deviation until it coincides with the target viewpoint.

[0012] In some embodiments, the path planning module includes a coordinate system transformation unit, a scanning viewpoint determination unit, and an optimal path planning unit;

[0013] The coordinate system transformation unit is used to define a reference coordinate system with the workpiece to be inspected as the reference object, transform the pose of the 3D scanner to the reference coordinate system, and determine the coordinate unification signal;

[0014] The scanning viewpoint determination unit is used to determine multiple scanning viewpoint signals based on the reference coordinate system and the point position information at the measurement feature position of the workpiece to be inspected.

[0015] The optimal path planning unit is communicatively connected to the scanning viewpoint determination unit, and is used to receive the scanning viewpoint signal and perform optimal sorting of the multiple scanning viewpoints according to a preset path planning algorithm to obtain the optimal path order for the robot.

[0016] In some embodiments, the visual guidance module includes an import unit, a display unit, a pose deviation generation unit, a guided interaction module, and a dynamic prompting module;

[0017] The import unit is communicatively connected to the display unit and is used to import the workpiece model to be inspected, the multiple scanning viewpoints, and the reference coordinate system.

[0018] The display unit is used to visualize the pose relationship between the 3D scanner and the target scanning viewpoint among the multiple scanning viewpoints in real time, based on the imported workpiece model to be inspected, the multiple scanning viewpoints and the reference coordinate system, and to generate pose relationship instructions.

[0019] The motion path generation unit is communicatively connected to the display unit and is used to receive the pose relationship instruction and, based on the pose relationship instruction, determine the pose deviation of the 3D scanner relative to the target scanning viewpoint and the movement direction of the current target scanning viewpoint relative to the next scanning viewpoint in the optimal motion trajectory.

[0020] The guided interaction module is used to provide an intuitive measurement of the deviation between the 3D scanner and the target viewpoint based on the pose deviation.

[0021] The dynamic prompting module is used to provide corresponding prompts based on the pose relationship between the 3D scanner and the target viewpoint.

[0022] Secondly, the present invention also provides a robot path planning method based on scanner vision guidance, applied to the robot path planning system based on scanner vision guidance described in any of the above claims, comprising:

[0023] Acquire multiple scanning viewpoints;

[0024] A preset path planning algorithm is used to optimize the sorting of the multiple scanning viewpoints to obtain the robot's optimal motion trajectory;

[0025] The optimal motion trajectory and the scanning pose of the 3D scanner are displayed in real time. Based on the pose deviation between the scanning pose of the 3D scanner and the target scanning viewpoint in the optimal motion trajectory, the robot adjusts the pose of the 3D scanner until it coincides with the target scanning viewpoint.

[0026] In some embodiments, before acquiring the scanning viewpoint, a preset coordinate transformation method is used to transform the coordinates of the 3D scanner to the reference coordinate system, using the coordinate system of the workpiece to be inspected as the reference coordinate system. This includes:

[0027] Calibrate features on multiple workpieces;

[0028] Obtain the coordinates of the feature in the scanner coordinate system and the coordinates of the key points of the feature in the workpiece coordinate system, respectively;

[0029] Based on the coordinates of the features in the scanner coordinate system and the key point coordinates of the features in the workpiece coordinate system, the coordinates of the 3D scanner are transformed to the reference coordinate system according to a preset coordinate system transformation relationship.

[0030] In some embodiments, acquiring multiple scanning viewpoints includes:

[0031] Based on the reference coordinate system, multiple scanning viewpoints are determined according to the point position information at the measurement feature pose of the workpiece to be inspected.

[0032] In some embodiments, the scanning viewpoint includes the pose information and angle information of the workpiece to be inspected.

[0033] In some embodiments, adjusting the pose of the 3D scanner by the robot until it coincides with the target scanning viewpoint includes:

[0034] Obtain the real-time pose information of the 3D scanner after reaching the current scanning viewpoint;

[0035] Obtain the pose deviation between the real-time pose information and the current scanning viewpoint;

[0036] If the pose deviation is less than a preset deviation threshold, then the current scanning viewpoint planning is determined to be successful.

[0037] In some embodiments, after the pose deviation is less than a preset deviation threshold and the current scanning viewpoint is successfully planned, the method further includes:

[0038] Based on the current scanning viewpoint's movement direction relative to the next scanning viewpoint in the optimal path, proceed to the point planning for the next scanning viewpoint.

[0039] In some embodiments, the visual guidance module provides a prompt for the next planning step before proceeding to the next scanning viewpoint.

[0040] Compared with existing technologies, the robot path planning system and method based on scanner vision guidance provided by this invention first pre-determines the scanning viewpoint of the 3D scanner based on the feature information of the workpiece to be inspected. Then, the path planning module determines the optimal motion trajectory of the robot based on the preset scanning viewpoint. Furthermore, the vision guidance module displays the optimal motion trajectory, the pose of the 3D scanner, and the pose relationship between the target scanning viewpoint in real time. Based on the real-time displayed pose relationship, the robot adjusts the pose of the 3D scanner until it coincides with the target scanning viewpoint, and then performs the scanning. This invention provides visual reference for manual path planning, quickly completes the teaching process, improves on-site debugging efficiency, and enhances the quality of the planned path, overcoming the problem of dependence on specific robot models and offline simulation software. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of an embodiment of the robot path planning system based on scanner vision guidance provided by the present invention;

[0042] Figure 2 This is a schematic diagram of an embodiment of the path planning module in the robot path planning system based on scanner vision guidance provided by the present invention;

[0043] Figure 3 This is a schematic diagram of an embodiment of the visual guidance module in the robot path planning system based on scanner vision guidance provided by the present invention;

[0044] Figure 4 This is a schematic diagram of an embodiment of the robot path planning system based on scanner vision guidance provided by the present invention, in which the vision guidance module displays the real-time display of an embodiment.

[0045] Figure 5 This is a flowchart of an embodiment of the robot path planning method based on scanner vision guidance provided by the present invention;

[0046] Figure 6 This is a schematic diagram illustrating the implementation of cluster analysis in the robot path planning method based on scanner vision guidance provided by the present invention.

[0047] Figure 7 This is a schematic diagram of an embodiment of step S503 in the robot path planning method based on scanner vision guidance provided by the present invention;

[0048] Figure 8 This is a flowchart illustrating another embodiment of the scanner-guided robot path planning method provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0050] This invention provides a robot path planning system 1 based on scanner vision guidance. Please refer to [link to relevant documentation]. Figures 1-3 It includes: a 3D scanner 11, a path planning module 12, a visual guidance module 13, and a robot 14;

[0051] The 3D scanner is fixedly connected to the end of the robot and is used to create 3D data of the surface of the workpiece to be inspected.

[0052] The path planning module is communicatively connected to the 3D scanner and is used to determine the robot's optimal motion trajectory based on a preset scanning viewpoint.

[0053] The visual guidance module is used to determine the target scanning viewpoint in the optimal motion trajectory, visualize the pose deviation between the pose of the 3D scanner and the target scanning viewpoint in real time, and guide the 3D scanner to reach the specified pose.

[0054] The robot has multiple degrees of freedom and is used to adjust the pose of the 3D scanner according to the pose deviation until it coincides with the target viewpoint.

[0055] In this embodiment, the scanning viewpoint of the 3D scanner is first predetermined based on the feature information of the workpiece to be inspected. Then, the path planning module determines the optimal motion trajectory of the robot based on the preset scanning viewpoint. The visual guidance module displays the optimal motion trajectory, the pose of the 3D scanner, and the pose relationship between the target scanning viewpoint in real time. Based on the real-time displayed pose relationship, the robot adjusts the pose of the 3D scanner until it coincides with the target scanning viewpoint, and then performs the scanning. This invention provides visual reference for manual path planning, quickly completes the teaching process, improves on-site debugging efficiency, and enhances the quality of the planned path. It also overcomes the problem of dependence on specific robot models and offline simulation software.

[0056] It should be noted that 3D scanners can include handheld laser scanners, tracking 3D scanners, photogrammetric area scanners, and other types of 3D scanners (i.e., scanners with binocular stereo vision capabilities).

[0057] It should be noted that the scanning viewpoints are planned using a preset method and exist in a discrete form.

[0058] Furthermore, the optimal motion path is planned based on the set scanning viewpoint, including minimizing the path the scanner travels along the path formed by the scanning viewpoint. It is understandable that different starting points correspond to different optimal motion paths.

[0059] In some embodiments, please refer to Figure 2 The path planning module 12 includes a coordinate system transformation unit 121, a scanning viewpoint determination unit 122, and an optimal path planning unit 123;

[0060] The coordinate system transformation unit is used to define a reference coordinate system with the workpiece to be inspected as the reference object, transform the pose of the 3D scanner to the reference coordinate system, and determine the coordinate unification signal;

[0061] The scanning viewpoint determination unit is used to receive the coordinate unification signal and, based on the reference coordinate system, determine multiple scanning viewpoint signals according to the point information at the measurement feature pose of the workpiece to be inspected.

[0062] The optimal path planning unit is communicatively connected to the scanning viewpoint determination unit, and is used to receive the scanning viewpoint signal and perform optimal sorting of the multiple scanning viewpoints according to a preset path planning algorithm to obtain the optimal path order for the robot.

[0063] In this embodiment, before visual guidance, scanning is performed according to the initially planned path to obtain a preliminary model of the workpiece to be inspected. By analyzing and calculating the preliminary model, the deficiencies in the preliminary model are obtained. The deficiencies are used to optimize the viewpoint of the 3D scanner, so that the data of the workpiece to be inspected obtained by the 3D scanner is more complete. That is, the scanning viewpoint and scanning path are adjusted based on the initially planned scanning path, thereby overcoming the problem of repeatedly modifying and debugging the path by relying solely on human experience, and improving the path debugging efficiency of the 3D scanner.

[0064] In one specific embodiment, firstly, a preset coordinate transformation method is used to transform the coordinates of the 3D scanner to the reference coordinate system, with the coordinate system of the workpiece to be inspected as the reference coordinate system. Then, based on the reference coordinate system, multiple scanning viewpoints are determined according to the point information of the measurement feature positions of the workpiece to be inspected. Subsequently, a preset path planning algorithm is used to optimize and sort the multiple scanning viewpoints to obtain the optimal motion path of the robot. Finally, based on the optimal motion path of the robot, and according to the pose relationship between the pose of the 3D scanner and the scanning viewpoint to be reached, the 3D scanner is operated to reach the scanning viewpoint.

[0065] It should be noted that, based on the reference coordinate system, hand-eye calibration is performed on the robot and the 3D scanner to obtain the coordinate transformation relationship between the robot end effector and the 3D scanner relative to the reference coordinate system; multiple sets of static poses of the robot in the coordinate system of the 3D scanner are obtained; and based on the coordinate transformation relationship and the multiple sets of static poses of the robot in the coordinate system of the 3D scanner, the multiple sets of static poses of the robot are transformed to the reference coordinate system. Furthermore, since the scanner is fixed to the robot end effector, the hand-eye calibration process can be a solution to the coordinate transformation relationship between the robot end effector and the scanner. Specifically, the coordinates of both the robot and the scanner are transformed to the coordinate system of the workpiece to be inspected and represented in the form of a data matrix. The rigid transformation matrix between the robot end effector and the scanner is determined to achieve hand-eye calibration between the robot end effector and the scanner, thus obtaining the transformation relationship between them. Therefore, when the scanner moves along the optimal path of the scanning viewpoint, the robot follows. It should be noted that this invention does not limit the specific robot; therefore, the hand-eye calibration results will be inconsistent for different robots.

[0066] In some embodiments, please refer to Figure 3 The visual guidance module 13 includes an import unit 131, a display unit 132, a pose deviation generation unit 133, a guided interaction module 134, and a dynamic prompt module 135.

[0067] The import unit is communicatively connected to the display unit and is used to import the workpiece model to be inspected, the multiple scanning viewpoints, and the reference coordinate system.

[0068] The display unit is used to visualize the pose relationship between the 3D scanner and the target scanning viewpoint among the multiple scanning viewpoints in real time, based on the imported workpiece model to be inspected, the multiple scanning viewpoints and the reference coordinate system, and to generate pose relationship instructions.

[0069] The pose deviation generation unit is communicatively connected to the display unit and is used to receive the pose relationship instruction and, based on the pose relationship instruction, determine the pose deviation of the 3D scanner relative to the target scanning viewpoint and the movement direction of the current target scanning viewpoint relative to the next scanning viewpoint in the optimal motion trajectory.

[0070] The guided interaction module is used to provide an intuitive measurement of the deviation between the 3D scanner and the target viewpoint based on the pose deviation.

[0071] The dynamic prompting module is used to provide corresponding prompts based on the pose relationship between the 3D scanner and the target viewpoint.

[0072] In this embodiment, the visual guidance module displays the optimal motion path in real time, as well as the current pose of the 3D scanner and its distance, direction, and trajectory from the next target scanning viewpoint. This allows the operator to manipulate the robot to move the 3D scanner to the target scanning viewpoint along the path and direction provided by the visual guidance module. By guiding the movement of the 3D scanner through the visual guidance module, the problem of low scanning efficiency caused by blind movement is avoided.

[0073] For further details, please refer to Figure 4 , Figure 4 The diagram illustrates a visual interface that provides real-time, multi-dimensional display of the pose relationship and deviation between the 3D scanner and the target viewpoint. This allows for more intuitive guidance for operators to adjust the 3D scanner's pose.

[0074] Based on the aforementioned scanner-guided robot path planning system, this invention also provides a scanner-guided robot path planning method. Please refer to [link to relevant documentation]. Figure 5 ,include:

[0075] S501, Acquire multiple scanning viewpoints;

[0076] S502. Using a preset path planning algorithm, the multiple scanning viewpoints are optimally sorted to obtain the robot's optimal motion trajectory.

[0077] S503. The optimal motion trajectory and the scanning pose of the 3D scanner are displayed in real time. Based on the pose deviation between the scanning pose of the 3D scanner and the target scanning viewpoint in the optimal motion trajectory, the robot adjusts the pose of the 3D scanner until it coincides with the target scanning viewpoint.

[0078] In this embodiment, the scanning viewpoint is acquired, the optimal motion path is then determined based on the scanning viewpoint, and finally, the 3D scanner is operated to reach the target location by displaying the optimal path, the motion direction and distance of the 3D scanner in real time.

[0079] Furthermore, in this embodiment, by visually guiding a series of scanning viewpoints in the order of the optimal path, the visualization module interface of the guidance software sequentially displays the angle information, position information, and movement direction of the current scanning viewpoint relative to the next scanning viewpoint in the optimal path. By calculating the pose deviation between the scanner and the current scanning viewpoint and displaying the pose deviation on the visualization interface through a visual graphic, the pose of the scanner can be adjusted according to the pose deviation until it coincides with the current scanning viewpoint.

[0080] Specifically, the pose deviation is displayed as a visual image on the visualization interface. In one specific embodiment, the deviation between the 3D scanner and the reference pose is typically displayed through the visualization interface, combined with indicator bars on the top, left, and right sides. Only when the laser corresponding to the scanner aligns with the pose of the three indicator bars is it considered that the scanner has reached the designated pose. In this way, the scanner's movement has a clear target and direction, which can more conveniently guide the scanner's positioning.

[0081] In step S502, the preset path planning algorithm includes one of the following: enumeration method, backtracking method, and greedy algorithm.

[0082] In this embodiment, considering both the algorithm's approach and time complexity, a greedy algorithm is employed to find the shortest scanning path for the surface scanning robot system. The greedy algorithm is based on a divide-and-conquer strategy, breaking down a large problem into several smaller problems and then merging the solutions to obtain the final result. When solving this problem, the greedy algorithm selects a scanning viewpoint as the starting point. When searching for the next scanning viewpoint, it only considers the viewpoint closest to the current viewpoint as the next point, continuing until all scanning viewpoints have been visited. Since each selection involves the shortest distance, the path traveled when all scanning viewpoints have been visited and the robot returns to the starting position is the shortest path for the scanning robot.

[0083] The specific algorithm is as follows: First, select the robot's starting position as the initial scanning viewpoint for the scanning path. Find the scanning viewpoint closest to the starting position in the scanning viewpoint array and mark it as the second point on the scanning path. Then, find the unmarked point closest to the second scanning viewpoint in the viewpoint array and mark it as the third point on the scanning path, and so on, until all viewpoints in the viewpoint array are marked. At this point, all points in the viewpoint array are reordered, and the points at the robot's initial position are connected sequentially with the reordered viewpoints, thus generating the shortest scanning path for the area scanning measurement robot.

[0084] In some embodiments, before acquiring the scanning viewpoint, a preset coordinate transformation method is used to transform the coordinates of the 3D scanner to the reference coordinate system, using the coordinate system of the workpiece to be inspected as the reference coordinate system. This includes:

[0085] Calibrate features on multiple workpieces;

[0086] Obtain the coordinates of the feature in the scanner coordinate system and the coordinates of the key points of the feature in the workpiece coordinate system, respectively;

[0087] Based on the coordinates of the features in the scanner coordinate system and the key point coordinates of the features in the workpiece coordinate system, the coordinates of the 3D scanner are transformed to the reference coordinate system according to a preset coordinate system transformation relationship.

[0088] In this embodiment, at least four features are first specified at any position on the workpiece. On the one hand, the 3D scanner collects the position data of these features and performs scanning feature extraction to obtain the coordinates of the key points of the features in the scanner coordinate system. On the other hand, digital model feature extraction is performed on the at least four features, and the key point coordinates of these features in the CAD model are extracted based on the workpiece CAD model. Finally, the transformation relationship from the scanner coordinate system to the workpiece coordinate system is calculated based on the corresponding points of the key point coordinates of the features in the scanner coordinate system and the key point coordinates in the CAD model.

[0089] In some embodiments, acquiring multiple scanning viewpoints includes:

[0090] Based on the reference coordinate system, multiple scanning viewpoints are determined according to the point information at the measurement feature positions of the workpiece to be inspected.

[0091] In this embodiment, a preset clustering analysis algorithm is first used to segment the triangular mesh of the workpiece to be inspected to obtain multiple triangular mesh sub-regions; then, based on the multiple triangular mesh sub-regions, a preset minimum bounding box method is used to determine multiple scanning viewpoints of the workpiece to be inspected.

[0092] It should be noted that triangular mesh is a data structure used in computer graphics to describe various irregular objects. In order to adapt to the workpiece to be inspected with complex curved surfaces, this embodiment first performs triangular mesh segmentation to obtain multiple triangular mesh sub-regions.

[0093] Specifically, based on the characteristics of the triangular mesh model, the Gaussian mapping of the model is first calculated. The mapping process is as follows: the starting point of the normal vector at each point on the object's surface is translated to the center of a Gaussian sphere. Each normal vector will intersect the surface of the Gaussian sphere at a point. The set of all these intersection points is the Gaussian mapping of the surface. Please refer to [link / reference]. Figure 6 , Figure 6 This is a diagram illustrating the principle of Gaussian mapping, where point P is a point on surface S, and vector... Let P be the normal vector of point P. If we translate the starting point to the center O of the sphere, we obtain the intersection point K between it and the sphere. Following this method, we move the normal vectors of all points on the surface to the center O of the unit sphere, and the set of intersection points of the normal vectors with the sphere is the Gaussian mapping of the surface S.

[0094] Subsequently, based on the results of the Gaussian mapping, a pre-defined clustering analysis algorithm was used to roughly segment the triangular mesh. Please refer to [link / reference]. Figure 7 , Figure 7 This diagram illustrates cluster analysis. Points A and B are two points on a sphere. The angle between vectors OA and OB is α. The angle between the normals of the two triangular faces corresponding to points A and B is also α. The magnitude of α corresponds to the distance between points A and B, which is the chord AB. Therefore, the larger the angle between the normals of the two triangular faces, the longer the chord AB, and the greater the distance between points A and B. Thus, grouping triangular faces with similar normal angles into one class is essentially grouping closely spaced mapping points on the sphere into one class. The problem of clustering based on the angle between the normals of triangular faces is transformed into a process of clustering data points based on the shortest distance.

[0095] Finally, based on the approximate categories obtained from cluster analysis, and according to whether the triangular facets in this category are connected to each other, this large category is divided into several independent regions that are not connected. Then, these regions with similar and adjacent average normal angles are merged to form new, larger regions. This process is repeated until all independent regions and their adjacent regions can no longer be merged, thus determining multiple triangular mesh sub-regions.

[0096] Furthermore, firstly, based on the scanner's parameter characteristics, a scanning frustum conforming to the scanner's features is established to make the region within the frustum visible. Then, the obtained triangular mesh sub-regions are traversed, and one sub-region is selected in spatial order and projected onto a two-dimensional plane perpendicular to the average normal of that region, obtaining the two-dimensional projection surface of that spatial region. Based on the scanner's frustum size constraints, the optimal scanning distance of the scanner is selected to fit the two-dimensional projection surface, and the two-dimensional projection surface is fully subdivided to obtain a series of two-dimensional scanning viewpoints. For these two-dimensional scanning viewpoints, the average coordinates of the three-dimensional points within the triangular mesh region mapped by the projection surface are used as the three-dimensional position of the scanning viewpoint, and the average normal vector of the mapped three-dimensional points is used as the angular orientation of the scanning viewpoint. This determines the sequence of scanning viewpoints for a sub-region. By traversing the remaining sub-regions sequentially, the scanning viewpoints for all regions can be calculated.

[0097] In some embodiments, the scanning viewpoint includes position information and angle information.

[0098] In this embodiment, the scanning viewpoint P(x,y,z,a,b,c) is used, where abc are the angles between the viewpoint and the three directions. The scanning viewpoint in space can be accurately located using the position coordinate information and angle information.

[0099] In some embodiments, adjusting the pose of the 3D scanner by the robot until it coincides with the target scanning viewpoint includes:

[0100] Obtain the real-time pose information of the 3D scanner after reaching the current scanning viewpoint;

[0101] Obtain the pose deviation between the real-time pose information and the current scanning viewpoint;

[0102] If the pose deviation is less than a preset deviation threshold, then the current scanning viewpoint planning is determined to be successful.

[0103] In this embodiment, in order to improve the efficiency and fault tolerance of path planning, a deviation threshold is set to determine whether the scanner has reached the specified pose and whether it meets the positioning standard by the relationship between the pose deviation and the deviation threshold.

[0104] In some embodiments, if the pose deviation is less than a preset deviation threshold, it indicates that the current scanning viewpoint planning is successful, including:

[0105] Based on the current scanning viewpoint's movement direction relative to the next scanning viewpoint in the optimal path, the planning for the next scanning viewpoint location is initiated.

[0106] In this embodiment, the scanner is guided to move along the scanning viewpoints in the order of the optimal path until all scanning viewpoints in the optimal path have been scanned.

[0107] Furthermore, after a planning phase is completed, if a scan test reveals that certain poses require modification or adjustment, the scan viewpoint data can be adjusted. After adjustment, the planning guidance module is accessed again. If only a few adjustments are needed, the subsequent guidance can choose to plan the modified points or skip already confirmed points during the planning process. Robot waypoints can also be inserted autonomously during planning as needed, or redundant points can be deleted based on data acquisition results to achieve the best planning effect. The embodiments provided in this invention are merely one example of the situations covered by this invention, and the method itself can encompass various embodiments, including user-defined modifications and point adjustments.

[0108] In a specific embodiment, please refer to Figure 8 First, the positional relationship between the scanner and the workpiece coordinate system is calibrated using feature calibration or marker point positioning. Then, the measurement viewpoint of the scanner in the workpiece coordinate system is determined based on one of the following methods: measurement CAD export, area array planning, or interactive editing generation. The optimal path is sorted for multiple measurement viewpoints. Finally, the scanner is guided to the specified pose through visualization or interactive visual guidance.

[0109] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A robot path planning system based on scanner visual guidance, characterized by, The application relates to a three-dimensional scanning system, which comprises a three-dimensional scanner, a path planning module, a visual guiding module and a robot. The three-dimensional scanner is fixedly connected to the end of the robot and is used for creating three-dimensional data of a workpiece surface to be detected. The path planning module is in communication connection with the three-dimensional scanner and is used for determining an optimal motion trajectory of the robot based on a preset scanning viewpoint. The visual guiding module is used for determining a target scanning viewpoint in the optimal motion trajectory, visually displaying a pose deviation between the three-dimensional scanner and the target scanning viewpoint in real time and guiding the three-dimensional scanner to reach a specified position. The robot has multiple degrees of freedom and is used for adjusting the pose of the three-dimensional scanner according to the pose deviation until the three-dimensional scanner coincides with the target scanning viewpoint. The path planning module comprises a coordinate system conversion unit, a scanning viewpoint determination unit and an optimal path planning unit. The coordinate system conversion unit is used for defining a reference coordinate system with the workpiece to be detected as a reference object, converting the pose of the three-dimensional scanner to the reference coordinate system, and determining a coordinate unification signal. The scanning viewpoint determination unit is used for receiving the coordinate unification signal, determining multiple scanning viewpoint signals according to point information at a measurement feature position of the workpiece to be detected based on the reference coordinate system. The optimal path planning unit is in communication connection with the scanning viewpoint determination unit and is used for receiving the scanning viewpoint signals and optimally sequencing the multiple scanning viewpoints according to a preset path planning algorithm to obtain an optimal path sequence of the robot. The visual guiding module comprises an importing unit, a display unit, a pose deviation generation unit, a guiding interactive unit and a dynamic prompting unit. The importing unit is in communication connection with the display unit and is used for importing the workpiece model to be detected, the multiple scanning viewpoints and the reference coordinate system. The display unit is used for visually displaying a pose relationship between the three-dimensional scanner and a target scanning viewpoint in the multiple scanning viewpoints in real time according to the imported workpiece model to be detected, the multiple scanning viewpoints and the reference coordinate system, and generating a pose relationship instruction. The pose deviation generation unit is in communication connection with the display unit and is used for receiving the pose relationship instruction, determining a pose deviation of the three-dimensional scanner relative to the target scanning viewpoint and a moving direction of the target scanning viewpoint relative to a next scanning viewpoint in the optimal motion trajectory according to the pose relationship instruction. The guiding interactive unit is used for providing an intuitive deviation measurement degree between the three-dimensional scanner and the target scanning viewpoint based on the pose deviation. The dynamic prompting unit is used for giving corresponding prompts according to the pose relationship between the three-dimensional scanner and the target scanning viewpoint. The application further relates to a three-dimensional scanning method.

2. A method of robot path planning based on scanner visual guidance, applied to the robot path planning system based on scanner visual guidance according to claim 1, characterized in that, A plurality of scanning viewpoints are acquired. An optimal motion trajectory of a robot is obtained by optimally sequencing the multiple scanning viewpoints according to a preset path planning algorithm. ​ The optimal motion trajectory and the scanning pose of the three-dimensional scanner are displayed in real time, and the pose of the three-dimensional scanner is adjusted by the robot according to the pose deviation between the scanning pose of the three-dimensional scanner and a target scanning viewpoint in the optimal motion trajectory until the target scanning viewpoint is reached.

3. The method of claim 2, wherein, Before acquiring the scanning viewpoints, a preset coordinate conversion method is adopted to convert the coordinates of the three-dimensional scanner to a reference coordinate system taking the coordinate system of the workpiece to be detected as the reference coordinate system, which includes: Calibrating features on a plurality of workpieces; Respectively acquiring coordinates of the features in the scanner coordinate system and key point coordinates of the features in the workpiece coordinate system; Based on the coordinates of the features in the scanner coordinate system and the key point coordinates of the features in the workpiece coordinate system, the coordinates of the three-dimensional scanner are converted to the reference coordinate system according to a preset coordinate system conversion relationship.

4. The method of claim 3, wherein, The acquisition of the plurality of scanning viewpoints includes: Based on the reference coordinate system, a plurality of scanning viewpoints are determined according to point position information at the measurement feature positions of the workpiece to be detected.

5. The method of claim 4, wherein, The scanning viewpoints include position information and angle information of the workpiece to be detected.

6. The robot path planning method based on scanner visual guidance according to claim 2, wherein, The adjustment of the pose of the three-dimensional scanner by the robot until the target scanning viewpoint is reached includes: Acquiring real-time pose information of the three-dimensional scanner after reaching the current scanning viewpoint; Acquiring the pose deviation between the real-time pose information and the current scanning viewpoint; If the pose deviation is less than a preset deviation threshold, it is determined that the current scanning viewpoint planning is successful.

7. The method of claim 6, wherein, When the pose deviation is less than the preset deviation threshold and the current scanning viewpoint planning is successful, it further includes: According to the movement direction of the current scanning viewpoint relative to the next scanning viewpoint in the optimal path, the point position planning of the next scanning viewpoint is entered.

8. The method of claim 6, wherein, Before entering the point position planning of the next scanning viewpoint, the visual guidance module is prompted for the next planning.

Images