Multi-view measurement path planning and registration method based on measured reachable domain

Abstract

The application discloses a multi-view measurement path planning and registration method based on a measurement reachable domain, wherein a suitable measurement angle is sequentially screened out from a discrete point set by taking the measurement reachable domain with the largest measurement reachable domain as a screening condition, and a measurement track is formed, the method optimizes the scanning efficiency of a detector, minimizes the number of measurement angles, significantly reduces the number of measurement angles required for obtaining complete point cloud data, and helps to reduce the cumulative error caused by multi-piece point cloud pairwise registration.

Description

Multi-view measurement path planning and registration method based on measurement reachability domain Technical Field

[0001] This invention relates to the field of machining inspection technology, and in particular to a multi-view measurement path planning and registration method based on the measurement reachability domain. Background Technology

[0002] Turbine blades, screws, and other complex peripheral components are often manufactured robotically, allowing for high precision and accuracy in their three-dimensional form, including dimensions and angles. Therefore, precise three-dimensional topographical surveying during machining plays a crucial role in improving the accuracy of finished parts.

[0003] Currently, contact probe-based measurement methods are widely used in the design, manufacturing, and quality inspection of cast parts due to their high accuracy. However, these methods are inefficient and offer limited feedback. On the other hand, optical surface measurement methods are attracting attention due to their simplicity, flexibility, and semi-automation. Line laser measurement is renowned for its high precision and efficiency, but it has limitations in acquiring complete point cloud data in a single scan due to occlusion reflections and field-of-view constraints. Therefore, it is necessary to obtain a complete and highly accurate model by measuring from multiple perspectives and aligning multiple point clouds.

[0004] The goal of point cloud registration is to align point cloud data from multiple coordinate systems into a unified coordinate system. Point cloud registration can be divided into two categories: two-view registration and multi-view registration. The former operates by aligning two sets of point clouds with overlapping areas. First, the source and target point clouds need to be identified, and their features extracted. Then, the extracted features are used to find the corresponding points. Finally, the coordinate system of the source point cloud is converted to that of the target point cloud, and the rotation and translation parameters are estimated.

[0005] Compared to two-view point cloud registration, multi-view point cloud registration is more complex and critical because it requires addressing more parameters and accumulated errors. In practical engineering applications, a common strategy is to first perform coarse registration on point clouds from multiple views, pre-aligning the point clouds across all views, and then perform fine registration to eliminate accumulated errors. The accuracy of coarse registration depends on the accuracy of the calibration method used to obtain the initial positional relationships of the point clouds. If the initial positional relationships cannot be obtained, the two point clouds must be aligned to obtain an initial transformation. However, multi-view coarse registration algorithms based on generating mesh maps or shapes are essentially still registrations of two point clouds, which introduces accumulated errors. Multi-view point cloud registration involves aligning multiple point clouds from the same scene from different viewpoints. Coarse registration is the first step in this process, involving obtaining initial values ​​for a set of point cloud transformation matrices as input to the fine matching stage. The goal of fine matching is to eliminate all transformations and accumulated errors to achieve high accuracy in multi-view point cloud registration.

[0006] However, existing multi-view point cloud measurements mostly rely on manual teaching to obtain the data, followed by registration to acquire complete workpiece point cloud data. In complex workpieces, numerous occlusions exist, necessitating the addition of measurement points. However, manual teaching heavily depends on experience, often resulting in repeated measurements at many locations to ensure completeness, leading to a large number of measurement angles and impacting efficiency. Furthermore, the selection of measurement angles is not optimized, often resulting in two point clouds lacking significant features, leading to decreased point cloud registration accuracy. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a multi-view measurement path planning and registration method based on the measurement reachability domain, aiming to obtain complete measurement data using the fewest possible measurement angles.

[0008] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0009] A multi-view measurement path planning and registration method based on the measurement reachability domain includes the following steps:

[0010] Step 1: Input the CAD model of the component to be measured, and take uniform discrete points on the surface of the CAD model to form a set of discrete points, and divide the workpiece into multiple sections;

[0011] Step 2: Calculate the measurement reachable region of all discrete points in a single cross-section;

[0012] Step 3: Select the discrete point with the largest reachable measurement domain as the starting measurement point, and determine the starting measurement angle of the starting measurement point. Search for the discrete points that can be measured from the starting measurement angle and mark them as measured points.

[0013] Step 4: Remove the measured points from the set of discrete points, and reselect the measurement angle corresponding to the detector with the most measurement points from the remaining discrete points in the set of discrete points as the next measurement point. Determine the measurement angle of the measurement point. Repeat step 3 until all discrete points in the set of discrete points are marked as measured points.

[0014] Step 5: Repeat steps 2-4 until all discrete points contained in all cross sections are marked. Then connect the measurement points of all cross sections on the component to be measured to obtain the measurement trajectory. Smooth the measurement trajectory to obtain the final measurement trajectory.

[0015] Step 6: Perform measurements based on the planned measurement points and the final measurement trajectory to obtain multiple multi-view point clouds. Use a point cloud registration neural network based on graph optimization method to register the multi-view point clouds and output the registration results.

[0016] Step 7: Obtain the coordinate transformation relationship between the multi-view point clouds based on the registration result, merge the multiple point clouds in the same coordinate system through the coordinate transformation relationship, and output the final measurement model of the complex component.

[0017] In some implementations, in step 1, any cross section of the CAD model is taken, and the positions of multiple discrete points are obtained from the contour of the cross section until the positions of all discrete points on the surface of the CAD model are obtained.

[0018] In some implementations, in step 2, multiple detectors are simulated around the component to be measured in the plane where the cross section is located. Each detector corresponds to a measurement angle. For a detector and a discrete point, if the current detector can detect the discrete point, the measurement angle of the current detector is defined as a measurement angle of the discrete point. The different detectors are iterated to calculate whether they can observe the discrete point, forming a measurement reachable region for a certain discrete point.

[0019] In some implementations, the method for obtaining the initial measurement angle in step 3 includes:

[0020] Step 301: Count the number of discrete points that the detectors at different positions can acquire, and define it as the number of measurement points. Select the measurement angle corresponding to the detector with the most measurement points as the starting measurement angle.

[0021] In some implementations, step 3 further includes planning the scanning angle of the cross section:

[0022] Step 302: Remove the measurement points that can be measured at the starting measurement angle from the set of discrete points;

[0023] Step 303: Count the number of remaining measurement points in the discrete point set, and reselect the measurement angle corresponding to the detector with the most measurement points as the measurement angle.

[0024] Step 304: Continue to remove measurement points that can be measured at the measurement angle from the set of discrete points;

[0025] Step 305: Repeat steps 303 and 304 until the set of discrete points is empty. Use the obtained initial measurement angle and the measurement angle to plan the scanning angle of each section.

[0026] Step 306: Perform steps 301 to 305 on the contour of each of the cross sections to obtain the starting measurement angle and measurement angle corresponding to the contour of each cross section.

[0027] The beneficial effects of this invention are as follows: by using the largest reachable measurement domain as the screening condition, suitable measurement angles are sequentially selected from the discrete point set and a measurement trajectory is formed. This method optimizes the scanning efficiency of the detector and minimizes the number of measurement angles, significantly reducing the number of measurement angles required to obtain complete point cloud data, which helps to alleviate the cumulative error caused by pair registration of multiple point clouds. Attached Figure Description

[0028] Figure 1 is a flowchart illustrating the multi-view measurement path planning and registration method based on the measurement reachability domain disclosed in an embodiment of the present invention.

[0029] Figure 2a is a schematic diagram of the structure of the component to be measured;

[0030] Figure 2b is a schematic diagram of the cross-sectional dividing lines of the component to be measured;

[0031] Figure 3 is a schematic diagram of the measurement relationship between the detector and discrete points in a certain cross section of the component to be measured;

[0032] Figure 4 is a schematic diagram showing the relationship between the measurement points and the measurement trajectory on the component to be measured;

[0033] Figure 5 is a schematic diagram of the multi-view registration network framework. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the content of this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to this invention are shown in the accompanying drawings, not all of them.

[0035] This embodiment proposes a multi-view measurement path planning and registration method based on the measurement reachability domain, as shown in Figure 1, including the following steps:

[0036] Step 1: Input the CAD model of the component to be measured, and take uniform discrete points on the surface of the CAD model to form a set of discrete points. Divide the workpiece into multiple cross sections, and the multiple cross sections form a set of measurement cross sections.

[0037] In step 1, any cross-section of the CAD model is selected, and the positions of multiple discrete points are obtained from the contour of the cross-section until the positions of all discrete points on the surface of the CAD model are obtained. In this embodiment, the CAD 3D model of the component to be measured shown in Figure 2a is used as an example. After multiple layers of scribing, the surface of the model obtains the cross-sectional dividing lines shown in Figure 2b. By carefully planning the viewpoint of each cross-sectional curve, the number of scanning angles required to obtain complete point cloud data can be significantly reduced, which helps to mitigate the cumulative error caused by the pair registration of multiple point clouds.

[0038] Step 2: Calculate the reachable domain of all discrete points in a single cross section.

[0039] In step 2, multiple detectors are simulated around the component to be measured in the plane where the cross section is located. Each detector corresponds to a measurement angle. For a detector and a discrete point, if the current detector can detect the discrete point, the measurement angle of the current detector is defined as a measurement angle of the discrete point. The different detectors are traversed to calculate whether they can observe the discrete point, forming a measurement reachable region for a certain discrete point.

[0040] As shown in Figure 3, in order to perform scanning, under the multi-view measurement scheme, the component to be measured needs to be rotated, and multiple scanning laser cameras are placed within a 360-degree range around the component's axis. For the cross-sectional contour line C (section) of the component to be measured, the concept of a measurement point is first defined: taking point P1 (a discrete point) on the cross-sectional contour line C of the component to be measured as an example, if point P1 and the scanning line segment PL1 of the scanning laser camera L1 and the cross-sectional contour line C have no other intersection points besides point P1, then the angle of the scanning laser camera L1 is considered a measurement angle of point P1, and point P1 is a measurement point of the scanning laser camera L1. Still taking Figure 3 as an example, the measurement angles of point P1 that can be determined are only L1, L2, and L3, forming a measurement reachable region containing L1, L2, and L3.

[0041] Step 3: Select the discrete point with the largest reachable measurement area as the starting measurement point, determine the starting measurement angle of the starting measurement point, search for the discrete points that can be measured from the starting measurement angle, and mark them as measured points.

[0042] In this embodiment, a discrete point is selected from the set of discrete points as the starting measurement point, using the largest measurable region as the filtering criterion.

[0043] In step 3, the method for obtaining the initial measurement angle includes:

[0044] Step 301: Count the number of discrete points that the detectors at different positions can acquire, and define it as the number of measurement points. Select the measurement angle corresponding to the detector with the most measurement points as the starting measurement angle.

[0045] Step 3 also includes planning the scanning angle of the cross section:

[0046] Step 302: Remove the measurement points that can be measured at the initial measurement angle from the set of discrete points;

[0047] Step 303: Count the number of remaining measurement points in the discrete point set, and reselect the measurement angle corresponding to the detector with the most measurement points as the measurement angle.

[0048] Step 304: Continue to remove measurement points that can be measured at the measurement angle from the discrete point set;

[0049] Step 305: Repeat steps 303 and 304 until the set of discrete points is empty. Use the obtained initial measurement angle and measurement angle to plan the scanning angle of each cross section.

[0050] Step 306: Perform steps 301 to 305 on the contour of each cross section to obtain the starting measurement angle and measurement angle corresponding to the contour of each cross section.

[0051] By running steps 301-306, the optimal camera angle for each cross-section can be determined efficiently while minimizing the number of scanning angles required, ensuring full coverage of the component to be measured.

[0052] Step 4: Remove the measured points from the discrete point set, and reselect the measurement angle corresponding to the detector with the most measurement points from the remaining discrete points in the discrete point set as the next measurement point. Determine the measurement angle of the measurement point, and repeat Step 3 until all discrete points in the discrete point set are marked as measured points.

[0053] Step 5: Repeat steps 2-4 until all discrete points contained in all cross sections are marked. Then connect the measurement points of all cross sections on the component to be measured to obtain the measurement trajectory. Smooth the measurement trajectory to obtain the final measurement trajectory, as shown in Figure 4.

[0054] Step 6: Perform measurements based on the planned measured points and the final measurement trajectory to obtain multiple multi-view point clouds. Use a point cloud registration neural network based on graph optimization to register the multi-view point clouds and output the registration results.

[0055] In step 6 of this embodiment, a multi-scale geometric feature extraction network is proposed for feature extraction, while transforming the registration problem of multi-view point clouds into a graph optimization problem. Here, vertices are defined as the initial position and pose of each segment of the point cloud in the coarse registration output of the multi-view point cloud. The edges connecting the vertices are used as edges to construct the graph. A graph optimization algorithm is used to perform nonlinear optimization on the estimated positions to perform multi-view registration and mitigate the cumulative error caused by pairwise registration of multi-view point clouds.

[0056] It is important to know that the constructed point cloud registration neural network is not unique, but it includes at least the modules shown in Figure 5: point cloud input, feature extraction, feature matching, graph optimization algorithm, registration result evaluation and output.

[0057] The feature extraction network module initially uses a multidimensional coding structure to focus on semantically rich regions at different levels. Then, it uses a geometric coding module (GE) to consider geometric features, and finally decodes and outputs features at three scales through a decoder.

[0058] For feature matching, an improved Hungarian algorithm could be considered.

[0059] The graph optimization algorithm module is used for feature matching to fine-tune the matching results from multiple perspectives.

[0060] Methods to improve the accuracy of multi-view registration based on graph optimization algorithms:

[0061] The point cloud registration problem can be described as follows: Given a source point cloud X = {xi∈R3}i = 1, 2, ..., N and a target point cloud Y = {yj∈R3}j = 1, 2, ..., M, where N and M represent the number of points in X and Y, respectively. The goal of point cloud registration is to determine a transformation T in SE(3) space to achieve optimal registration between X and Y, which can be expressed as:

[0062]

[0063] Traditional ICP algorithms are used to find the correct point correspondences between point clouds. After the initial coarse registration transformation, the algorithm calculates the distance from each point in the source point cloud to the target point cloud, and then determines the point pair with the minimum distance as the corresponding point pair between the two point clouds. This ensures that there is a correspondence between the points in the source point cloud and the points in the target point cloud, and at the same time constructs the residual sum of squares objective function. Using the least squares method, the algorithm minimizes the error function and performs iterative steps until the mean square error drops below a specified threshold.

[0064] It utilizes the ICP algorithm and a feature extraction network to match corresponding features extracted from the point cloud. Furthermore, as shown in Figure 5, the model transforms the registration problem into a graph optimization problem, employing the Hungarian algorithm for feature matching. A graph is constructed where workpiece positions are vertices and connecting lines are edges. Then, the Graph Optimization (G2O) algorithm is used for nonlinear optimization of the workpiece positions. This method effectively solves the problem of accumulated error caused by pairwise registration of multiple point cloud blocks.

[0065] Step 7: Obtain the coordinate transformation relationship between the multi-view point clouds based on the registration results. Merge the multiple point clouds in the same coordinate system through the coordinate transformation relationship and output the final measurement model of the complex component.

[0066] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A multi-view measurement path planning and registration method based on the measurement reachability domain, characterized in that, Includes the following steps: Step 1: Input the CAD model of the component to be measured, and take uniform discrete points on the surface of the CAD model to form a set of discrete points, and divide the workpiece into multiple sections; In Step 1, take any section of the CAD model, and obtain the positions of multiple discrete points from the contour of the section until the positions of all discrete points on the surface of the CAD model are obtained; Step 2: Calculate the measurement reachable domain of all discrete points in a single section. In step 2, multiple detectors are simulated around the component to be measured in the plane where the cross section is located. Each detector corresponds to a measurement angle. For a detector and a discrete point, if the current detector can detect the discrete point, the measurement angle of the current detector is defined as a measurement angle of the discrete point. The different detectors are traversed to calculate whether they can observe the discrete point, forming a measurement reachable domain for a certain discrete point. Step 3: Select the discrete point with the largest reachable measurement area as the starting measurement point, and determine the starting measurement angle of the starting measurement point. Search for the measurable discrete points from the starting measurement angle and mark them as measured points. Step 4: Remove the measured points from the set of discrete points, and reselect the measurement angle corresponding to the detector with the most measurement points from the remaining discrete points in the set as the next measurement point. Determine the measurement angle of the measurement point, and repeat Step 3 until all discrete points in the set of discrete points are marked as measured points. Step 5: Repeat Steps 2-4 until all discrete points contained in all cross sections are marked. Then connect the measurement points of all cross sections on the component to be measured to obtain the measurement trajectory, and smooth the measurement trajectory to obtain the final measurement trajectory. Step 6: Perform measurements based on the planned measurement points and the final measurement trajectory to obtain multiple multi-view point clouds. Use a point cloud registration neural network based on graph optimization method to register the multi-view point clouds and output the registration results. Step 7: Obtain the coordinate transformation relationship between the multi-view point clouds based on the registration result, merge the multiple point clouds in the same coordinate system through the coordinate transformation relationship, and output the final measurement model of the complex component.

2. The multi-view measurement path planning and registration method based on the measurement reachability domain as described in claim 1, characterized in that, In step 3, the method for obtaining the starting measurement angle includes: step 301, counting the number of discrete points that the detectors at different positions can obtain, defining it as the number of measurement points, and selecting the measurement angle corresponding to the detector with the most measurement points as the starting measurement angle.

3. The multi-view measurement path planning and registration method based on the measurement reachability domain as described in claim 2, characterized in that, Step 3 also includes the planning of the scanning angle of the cross section: Step 302, remove the measurement points that can be measured at the starting measurement angle from the discrete point set; Step 303, count the number of remaining measurement points in the discrete point set, and reselect the measurement angle corresponding to the detector with the most measurement points as the measurement angle; Step 304, continue to remove the measurement points that can be measured at the measurement angle from the discrete point set; Step 305, repeat steps 303 and 304 until the discrete point set is empty, and use the obtained starting measurement angle and the measurement angle to plan the scanning angle of each cross section; Step 306, perform steps 301 to 305 on the contour of each cross section to obtain the starting measurement angle and the measurement angle corresponding to the contour of each cross section.

Images