A catheter detection method and system based on scanning point cloud

Abstract

This invention belongs to the field of industrial inspection and reverse engineering technology, specifically disclosing a catheter inspection method based on scanning point clouds. The method includes: a dynamic self-calibrating optical system to acquire an initial point cloud of the measuring tube; segmenting the initial point cloud to form a complete tube shape; distinguishing straight segments and bending segments by the change in inclination angle of adjacent cylindrical axes; aligning bending points using the bending point ICP algorithm and obtaining matrix transformation relationships; correcting bending points based on these relationships and solving for process data; reconstructing catheter feature data; and finally aligning the reconstructed tube with the model tube using the bending point ICP algorithm to determine the measurement deviation. This invention solves problems such as low tracking accuracy caused by environmental interference, alignment affected by point cloud noise, difficulty in digitizing process data, and difficulty in feature reconstruction. It is suitable for high-precision inspection of complex catheters, providing quantitative basis for quality control and process optimization in industrial production.

Description

Technical Field

[0001] This invention belongs to the field of industrial inspection and reverse engineering technology, and specifically relates to a method and system for duct detection based on scanning point clouds. Background Technology

[0002] As a core transmission component in industrial and civilian fields, conduits play a vital role in various application scenarios. Whether it's the high-temperature and high-pressure environment in the aerospace field, the corrosion and fatigue challenges in the shipbuilding and automotive industries, or the large-scale corrosion monitoring needs in the energy and chemical industries, the inspection of conduits faces a series of challenges.

[0003] Current catheter inspection technologies mainly include photogrammetry, harpoon laser measurement, point cloud scanning, and traditional fixture methods. While each has its advantages, they also reveal several limitations in practical applications: Photogrammetry offers high speed and accuracy but is unsuitable for measuring complex catheters. Harpoon laser measurement provides real-time dynamic data feedback but suffers from low efficiency, difficulty in guaranteeing accuracy, and inability to effectively measure catheters with features or accessories, especially those lacking straight segments. Point cloud scanning is slow and complex in data processing, failing to effectively measure PRB process data, process deviations, process compensation, sheath deviations, and feature data of catheters. Traditional fixtures can only measure preset dimensions, cannot quantify specific inspection results, and only support measurement of the same type of catheter, resulting in excessively high measurement costs. When using point cloud scanning technology, environmental factors such as temperature fluctuations and system vibrations severely affect the accuracy of the optical system's tracking and positioning device, leading to inaccuracies in camera intrinsic and extrinsic parameters and a significant decrease in tracking accuracy. Furthermore, point cloud data often contains noise from non-measured targets, such as point clouds from conduit clamps, background walls, or workbenches. This noise severely interferes with the alignment of the measurement target point cloud, increasing the difficulty of subsequent data processing. Existing conduit inspection technologies struggle to directly extract conduit process data from point cloud data when dealing with complex pipe fittings, limiting their application value in actual production. Simultaneously, complex conduit features, such as end faces, flanges, nuts, and bayonets, lack effective point cloud processing methods, making it difficult to accurately reconstruct their feature data for subsequent inspection and analysis.

[0004] Therefore, given the current measurement technology, there is an urgent need to propose measurement methods for complex catheters, especially complex catheters applicable to different scenarios. Summary of the Invention

[0005] The purpose of this invention is to overcome the defects in the prior art and provide a duct detection method and system based on scanning point clouds.

[0006] A first aspect of the present invention provides a duct detection method based on scanned point clouds, comprising the following steps:

[0007] S1. Marking points are fixedly set on the scanner housing, and several feature points are evenly preset around the periphery of the measurement area. The camera system tracks the scanner moving along the scanning path and continuously captures the feature points. The feature point images captured at each scanning path point are used to calibrate the external parameter relationship of the camera system to complete the dynamic self-calibration of the camera system. The scanning point cloud corresponding to each scanning path point is converted to the global coordinate system where the feature points are located, and the initial point cloud of the measurement tube is obtained by stitching them together.

[0008] S2. Evaluate the quality of the initial point cloud. If the evaluation criteria are met, proceed to step S3. If the evaluation criteria are not met, downsample the initial point cloud and determine candidate points. Search for nearby point cloud sets with the candidate points as the center, fit a cylinder, and determine the seed point through the normal vector relationship and the point distance relationship. Based on the seed point, establish a seed cylinder and spread it along the trend of the initial point cloud to form a complete tube shape.

[0009] S3. Determine the straight segment and the bending segment based on the inclination angle change of adjacent cylindrical axes, fit the cylindrical axis of the straight segment and determine the bending point, apply the ICP algorithm to align the bending point of the measuring tube with the corresponding model tube, and obtain the matrix transformation relationship;

[0010] S4. Adjust the straight segment of the measuring tube based on the matrix transformation relationship, fit the cylindrical axis of the straight segment and solve the intersection of adjacent axes to obtain the calibrated bending point, and solve the process data of the measuring tube based on the bending point coordinates.

[0011] S5. Based on the matrix transformation relationship, the point cloud of the corresponding feature segment of the model tube is measured by applying the feature segment point cloud computing of the model tube, and the relevant data of the feature segment is restored.

[0012] S6. Apply the ICP algorithm to align the point clouds of the measuring tube and the model tube, and the resulting matrix transformation relationship is the measurement deviation.

[0013] A further solution is that, in S1, the dynamic self-calibration of the camera system includes:

[0014] At each path point, the camera system reconstructs the scanner shell marker points and feature points around the measurement plane; given the coordinates of the marker points in the marker point coordinate system and the coordinates of the feature points in the global coordinate system, solve for the matrix transformation relationship between the scanning coordinate system and the tracking coordinate system. And the matrix transformation relationship between the tracking coordinate system and the global coordinate system. ,in The transformation matrix between the pre-determined scanning coordinate system and the tracking coordinate system. The transformation matrix from the marker coordinate system to the tracking coordinate system is represented by the singular value decomposition method (SVD). The transformation matrix from the scan coordinate system to the marker point coordinate system.

[0015] A further solution is that the expression for point cloud stitching described in S1 is:

[0016] ;

[0017] in For measuring tube point cloud in scanning coordinate system Measurement tube point cloud in global coordinate system This represents the matrix transformation relationship between the scan coordinate system and the global coordinate system at the corresponding scan path point.

[0018] A further solution is that, in S2, the evaluation criterion is: the ratio of the initial point cloud area of ​​each feature region to the point cloud area at the corresponding position of the model tube is greater than or equal to a first preset threshold.

[0019] In S2, the process of determining the seed point is as follows:

[0020] The point cloud of all candidate points and their vicinity is fitted into a cylinder using the least squares method.

[0021] Based on principal component analysis (PCA), the axial direction of the corresponding cylinder is determined by applying the point cloud of candidate points and their vicinity. ;

[0022] If the points in the candidate point and its vicinity that exceed the second preset threshold satisfy the condition... The calculation ends, and the axis of the cylinder is output. ;in, , The first point in the point cloud of the candidate point and its vicinity The normal vector of each point The cylindrical axis direction vector fitted to the point cloud of the candidate points and their vicinity;

[0023] Find the center point of the candidate point and its nearby point cloud, combined with the existing cylindrical axis. Establish a projection plane and project the candidate points and their nearby point cloud onto the projection plane;

[0024] By applying the least squares method, a circle is fitted to all two-dimensional projection points to determine the center of the fitted circle. and radius , No. The distance between each projection point and the center of the circle is denoted as . , For the index of the projection point;

[0025] If the points in the candidate point and its vicinity that exceed the third preset threshold satisfy the condition... End the calculation and output the center of the fitted circle. The candidate point is designated as the center point of the cylinder corresponding to the candidate point, and the candidate point is determined as the seed point.

[0026] In S2, the process of forming a complete tubular shape is as follows:

[0027] Randomly select an existing seed point as the search starting point, and move the seed point and the normal vector of the corresponding cylinder along the initial point cloud by one search step, using the set search step size as the unit.

[0028] The seed point at the previous position and the normal vector of the corresponding cylinder are used as the initial values ​​for iteration. The point cloud near the position of the seed point after the move is applied to fit the cylinder and solve for the new cylinder center point and normal vector corresponding to the current position.

[0029] This process continues until no more point clouds participate in cylinder fitting at the search position. Then, the search direction of the seed point is reversed, and the cylinder center point and normal vector corresponding to the point cloud at each search position are solved until all initial point clouds have completed cylinder fitting, resulting in a complete cylindrical model of the measuring tube.

[0030] A further solution is that the process for determining the bending point in S3 is as follows:

[0031] Cylindrical axes with an inclination angle change less than or equal to an inclination angle threshold are classified as straight segments, while those with an inclination angle change greater than the inclination angle threshold are classified as bent segments. Based on the directions of all cylindrical axes of a straight segment, the axis of the straight segment is fitted using the least squares method, and the intersection of the axes of adjacent straight segments is determined as the bend point.

[0032] A further proposed solution is that the ICP algorithm includes:

[0033] Determine the point cloud set of the bending points of the measuring tube. Collection of point clouds of bending points in model tubes The correspondence is used to calculate the centroid. Center of mass and decentralized coordinates Decentralized coordinates Construct the covariance matrix and use SVD decomposition to obtain the rotation and translation matrices. Iterate and update until the alignment error is minimized to obtain the rotation matrix transformation relationship. Transformation relationship of translation matrix ;in, It is a collection of points on the bending of the measuring tube. Yes, point clouds. Model tube bending point cloud collection The point cloud, and yes Point cloud set of bending points in the model tube In the nearest neighbor, i represents the point cloud index of the bend point of the measuring tube, j represents the point cloud index of the bend point of the model tube, N represents the number of bend points of the measuring tube, and M represents the number of bend points of the model tube.

[0034] A further solution is that the bending point correction in S4 includes: utilizing the transformation relationship of the rotation matrix. Translation matrix transformation relationship Project the point cloud of the straight segments of the model tube onto the measurement tube area, and readjust the straight segments of the measurement tube; fit the axis of the straight segments of the measurement tube using the least squares method, and solve for the intersection of adjacent axes as the calibrated bending point.

[0035] A further solution is that the feature segment in S5 includes at least one of the following: end face, flange, nut, and bayonet; the feature segment data restoration includes: fitting the end face of the feature segment based on the least squares principle, and determining the end face normal vector by solving the eigenvector corresponding to the smallest eigenvalue of the covariance matrix after the centroid of the point cloud is translated.

[0036] A further solution involves solving the matrix relationships in S6, including:

[0037] Extract the point cloud set of the bending points and feature points of the measuring tube. and the corresponding point cloud set of the model tube. The ICP algorithm is applied to align two sets of point clouds, and the alignment error is solved. Minimal rotation matrix Translation matrix ,in, For the x-th measurement point, Let x be the x-th model point, and H be the total number of points where the measuring tube and the model tube are aligned.

[0038] A second aspect of the present invention provides a catheter detection system based on scanned point clouds, comprising:

[0039] A robotic arm is used to drive the scanner to move along a planned path;

[0040] A scanner, fixed to the end of a robotic arm, has marking points on its housing;

[0041] The camera system is used to track marker points and measure feature points on the periphery of the measurement area in real time;

[0042] The data processing unit, connected to the scanner and camera system respectively, is configured to perform the methods described above.

[0043] Compared with existing technologies, the beneficial effects of this invention are as follows: The guide tube reconstruction and detection method based on scanning point clouds provided by this invention achieves dynamic self-calibration of the optical system by setting feature points on the periphery of the measurement area. This can correct the camera extrinsic parameter offset caused by ambient temperature and system vibration in real time, solving the problem of tracking accuracy being affected by environmental interference in traditional optical measurement, ensuring the global consistency of multi-path point cloud stitching, and improving the reliability of the initial point cloud. For point clouds with different matching degrees, a scheme combining direct alignment by digital-analog matching and point cloud segmentation plus bending point alignment is adopted, which can effectively filter noisy point clouds such as fixtures and backgrounds, avoid alignment deviations caused by noise interference, and enhance the accuracy and stability of point cloud alignment. Through bending point correction and... Axis fitting can directly solve for process data such as the advance amount, bending angle, and rotation angle of the conduit, breaking through the bottleneck of traditional technology that cannot convert point cloud data into quantitative process parameters, and providing direct data support for conduit processing machine adjustment and process optimization. Based on the matrix transformation relationship projection model pipe feature point cloud, combined with least squares fitting, it restores feature data such as end face, flange, and nut, solving the problem that existing technology is difficult to restore complex conduit features, and improving the completeness and accuracy of feature data. It is suitable for complex conduits with continuous bends, welded accessories, and no straight segments, and takes into account the non-destructive testing requirements of non-destructive testing with non-stick point scanning. Compared with traditional inspection tools, it significantly improves inspection efficiency and supports universal inspection of different conduit models, reducing inspection costs and equipment investment. Attached Figure Description

[0044] The following figures are for illustrative purposes only and are not intended to limit the scope of the invention, wherein:

[0045] Figure 1 : Flowchart of the detection method of this invention;

[0046] Figure 2 : Schematic diagram of dynamic self-calibration of camera system;

[0047] Figure 3 The intersection of the axes of adjacent straight lines determines the bend point;

[0048] Figure 4 : Schematic diagram of point cloud alignment using the ICP algorithm;

[0049] Figure 5 : Schematic diagram of calibrating bending points and restoring characteristic flanges;

[0050] Figure 6 Schematic diagram of ICP algorithm aligning measurement tube and model tube;

[0051] In the diagram: 1. Robotic arm; 2. Scanner; 3. Camera system; 4. Feature point; 5. Measuring tube; 6. Marker point. Detailed Implementation

[0052] To make the objectives, technical solutions, design methods, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0053] like Figure 2 As shown, the present invention provides a duct detection system based on scanning point clouds, comprising:

[0054] Robotic arm 1 drives scanner 2 to move along the planned path;

[0055] Scanner 2 is fixed to the end of robotic arm 1, and its outer shell is marked with marking points 6;

[0056] Camera system 3 includes a camera fixed to a metal structure, which tracks marker point 6 and feature points 4 around the measurement area in real time;

[0057] The data processing unit is connected to the robotic arm 1, the scanner 2, and the camera system 3 respectively, and executes the duct detection method based on the scanned point cloud, including the following operations: dynamic self-calibrated matrix transformation calculation, seed point determination and duct type generation for point cloud segmentation, ICP algorithm for bending point alignment, duct digital reconstruction and deviation detection.

[0058] Specifically, such as Figure 1 As shown, the duct detection method based on scanned point clouds includes:

[0059] S1. Marker points 6 are fixedly set on the outer shell of scanner 2. Several feature points 4 are evenly preset around the periphery of the measurement area. Camera system 3 tracks scanner 2 moving along the scanning path and continuously captures the feature points 4. The external parameter relationship of camera system 3 is calibrated by using the feature points 4 captured at each scanning path point to complete the dynamic self-calibration of camera system 3. The scanning point cloud corresponding to each scanning path point is converted to the global coordinate system where the feature points 4 are located, and the initial point cloud of measuring tube 5 is obtained by stitching them together.

[0060] In this embodiment, the scanner 2 moves along the scanning path to scan the tube to be measured placed on the measurement plane, and the camera system 3 tracks the moving scanner 2 in real time. Due to the long-term tracking operation, the camera is affected by its own temperature and system vibration, and the transformation relationship between its tracking coordinate system and the global coordinate system will change. Therefore, the relative positional relationship between the camera and the scanner 2 needs to be dynamically self-calibrated. Specifically, at each scanning path point, the camera system 3 synchronously reconstructs the coordinates of the marker point 6 (rigidly connected to the scanner 2, whose coordinates in the marker point coordinate system are pre-calibrated) on the outer shell of the scanner 2 and the coordinates of the feature point 4 on the periphery of the measurement plane (whose coordinates in the global coordinate system are pre-defined) in the tracking coordinate system. First, the transformation matrix between the two coordinate systems is calculated by using the singular value decomposition method of the marker point 6 in the marker point coordinate system and the tracking coordinate system. Combined with the predetermined transformation relationship between the scanning coordinate system and the marker point coordinate system The transformation matrix between the scanning coordinate system and the tracking coordinate system is obtained:

[0061] ;

[0062] Then, the transformation matrix between the tracking coordinate system and the global coordinate system is calculated from the coordinates of feature point 4 in the global coordinate system and the tracking coordinate system. Ultimately, utilizing The chain transformation relationship converts the measurement tube point cloud in the scanning coordinate system of each path point. Transform to the global coordinate system to obtain

[0063] ;

[0064] in, For a given scan path point, the measurement tube point cloud in the scan coordinate system. Point cloud of the measurement tube in the global coordinate system; This represents the matrix transformation relationship between the scan coordinate system and the global coordinate system at the corresponding scan path point.

[0065] S2. Evaluate the quality of the initial point cloud. If the evaluation criteria are met, proceed to step S3. If the evaluation criteria are not met, downsample the initial point cloud and determine candidate points. Search for nearby point cloud sets with the candidate points as the center, fit a cylinder, and determine the seed point through the normal vector relationship and the point distance relationship. Based on the seed point, establish a seed cylinder and spread it along the trend of the initial point cloud to form a complete tube shape.

[0066] Specifically, in this embodiment, the quality of the initial point cloud is evaluated. The specific evaluation criteria are as follows: compared with the standard digital model (model tube), the initial point cloud contains sufficient features of the conduit, and the area of ​​the point cloud in each feature region is not less than a first preset threshold of the area of ​​the point cloud at the corresponding position in the standard digital model. For example, the area of ​​the point cloud in each feature region is not less than 50% of the area of ​​the point cloud at the corresponding position in the standard digital model. If the initial point cloud meets this evaluation criterion, this step can be ignored, and bending point alignment can be performed directly; otherwise, point cloud segmentation continues. Specifically, the initial point cloud is downsampled, and all initial point clouds are traversed. While preserving the characteristics of the initial point cloud, multiple candidate points are randomly selected. Using the candidate points as centers, a search radius is set in the initial point cloud to search and determine the set of nearby point clouds. A cylinder is fitted using all candidate points and their nearby point cloud sets, and the candidate points are determined as seed points based on normal vector relationships and point distance relationships. Finally, based on the existing seed points and their nearby point cloud sets, a seed cylinder is established with a set search step size, and the cylinder is diffused along the trend of the initial point cloud to form a complete tubular shape. The method for fitting the cylinder using candidate points and their nearby point cloud sets and determining candidate points as seed points based on normal vector relationships and point distance relationships is as follows: All candidate points and their nearby point clouds are fitted with a cylinder using the least squares method; then, based on PCA principal component analysis, the axial direction of the corresponding cylinder is determined using the candidate points and their nearby point clouds. The specific solution to the equation is as follows:

[0067] ;

[0068] in, , The first point in the point cloud of the candidate point and its vicinity The normal vector of each point The cylinder's axis direction vector is fitted to the point cloud of the candidate point and its vicinity. If the number of points in the point cloud exceeding the second preset threshold (e.g., exceeding 60% of the points) satisfies the above requirement, the calculation ends, and the cylinder's axis is output. Find the center points of the candidate points and their nearby point clouds, combining this with the existing cylindrical axis. A projection plane is established, and the candidate points and their nearby point clouds are projected onto this projection plane. The least squares method is applied to fit a circle to all two-dimensional projection points, and the center of the fitted circle is determined. and radius , No. The distance between each projection point and the center of the circle is denoted as . Let be the index of the projection point; then the equation is solved as follows:

[0069] ;

[0070] If, in the point cloud of the candidate point and its vicinity, the number of points exceeding the third preset threshold (e.g., exceeding 60%) satisfies the above requirement, the calculation ends, and the fitted circle center is output. The candidate point is designated as the center point of the cylinder corresponding to the candidate point, and the candidate point is determined as the seed point.

[0071] The method for establishing a seed cylinder from existing seed points, setting a search step size, and expanding along the trend of the initial point cloud to form a complete tube shape includes:

[0072] A seed point is randomly selected as the starting point for the search. Using a set search step size, the seed point and its corresponding cylinder normal vector are moved one search step to the left along the initial point cloud. The seed point and its corresponding cylinder normal vector at the previous position are used as the initial values ​​for iteration. The point cloud near the new seed point's position is then used for cylinder fitting to obtain the new cylinder center point and normal vector at the current position. This process continues until no more point clouds participate in cylinder fitting at the search position. Then, the search direction of the seed point is reversed, and the seed point moves to the right. The cylinder center point and normal vector corresponding to the point clouds at each search position are then determined. It is important to note that if another seed point is found during the search process, that seed point takes over the above operations until all initial point clouds have completed cylinder fitting, resulting in a complete cylindrical model of the measuring tube 5.

[0073] S3. Determine the straight segments and bending segments based on the inclination angle changes of adjacent cylindrical axes, fit the cylindrical axis of the straight segments and determine the bending points, and apply the ICP algorithm to align the bending points of the measuring tube 5 and the corresponding model tube to obtain the matrix transformation relationship. In this embodiment, based on the inclination angle changes of adjacent cylindrical axes of the reconstructed tube type, it is confirmed that the adjacent cylindrical axes with small or no inclination angle changes are the straight segments of the measuring tube 5, and the adjacent cylindrical axes with large inclination angle changes are the bending segments of the measuring tube 5. Then, the cylindrical axis of the straight segments is fitted and solved to determine the cylindrical axis of the adjacent straight segments as the bending points. Finally, the ICP algorithm is applied to align the bending points of the measuring tube 5 and the corresponding model tube, determine the matrix transformation relationship of the two sets of bending point point clouds, and complete the bending point alignment.

[0074] like Figure 3 and Figure 4 As shown, based on the change in inclination angle of adjacent cylindrical axes, the straight segment of measuring tube 5 is determined, and the intersection point of adjacent straight segments is calculated. Figure 3 and Figure 4The method for identifying bend points (BP1, BP2, BP3) includes: comparing the changes in the inclination angles of adjacent cylindrical axes, determining that cylinders with changing inclination angles are bend segments, and those with very small or nearly overlapping inclination angles are straight segments; based on the axial directions of all cylinders in the straight segments, applying the least squares method to fit the corresponding straight segment's axis; and determining the intersection of the axes of adjacent straight segments as bend points. The method for using the ICP algorithm to solve for the matrix transformation relationship between the bend point cloud of measuring tube 5 and the bend point cloud of the corresponding model includes:

[0075] The correspondence between the bending points of measuring tube 5 and the bending points of the model tube was determined using nearest-point matching. The point cloud set of the bending points of measuring tube 5 is as follows: The set of bend points of the model tube is , yes Point cloud at the bend of the model tube The nearest neighbor in the middle, , To determine the rotation and translation matrix relationship between the point cloud of the bending point of tube 5 and the point cloud of the bending point of the model tube, where, Let be the number of iterations, for each The following minimum distance equation is established for the point cloud at the bending point of the model tube. Find the closest one , To fix the search threshold, i represents the point cloud index of the bend point of the measuring tube, j represents the point cloud index of the bend point of the model tube, N represents the number of bend points of the measuring tube, and M represents the number of bend points of the model tube.

[0076] ;

[0077] The correspondence between the bending point cloud of measuring tube 5 and the bending point cloud of the model tube was determined. Using Singular Value Decomposition (SVD), the rotation and translation matrix transformation relationships between the bending point cloud of measuring tube 5 and the bending point cloud of the model tube were solved. Let the initial rotation and translation matrices be: , The specific calculation process is as follows:

[0078] Calculate the point cloud of the bending point in tube 5 and the centroid of the point cloud of the bending point in the model. Center of mass :

[0079] ;

[0080] The solution involves solving the matrix transformation relationship from the bending point of measuring tube 5 to the bending point of the model. Therefore, the number of bending points should be consistent with the number of bending points of measuring tube 5, which is N.

[0081] Calculate the decentralized coordinates of the point clouds of two sets of bend points. Decentralized coordinates :

[0082] ;

[0083] Calculate the covariance matrix :

[0084] ;

[0085] SVD decomposition for finding rotation matrices:

[0086] For covariance matrix Perform singular value decomposition:

[0087] ;

[0088] Optimal rotation matrix Translation matrix for:

[0089] ;

[0090] Combining the initial rotation and translation matrices, and simultaneously accumulating the transformations obtained from the SVD decomposition steps for obtaining the rotation matrix. , To total transformation , middle:

[0091] ;

[0092] Continue updating the point cloud coordinates of the bending point of measuring tube 5:

[0093] ;

[0094] The alignment error at the current stage is calculated using the following formula:

[0095] ;

[0096] Repeat the above calculation process until the alignment error is minimized, then terminate the calculation and output the matrix transformation relationship between the two sets of bend point cloud data, denoted as . , .

[0097] S4. Based on the matrix transformation relationship, adjust the straight segment of the measuring tube 5, fit the cylindrical axis of the straight segment, and solve for the intersection of adjacent axes to obtain the calibrated bending point. Based on the bending point coordinates, solve for the process data of the measuring tube 5. In this embodiment, the matrix transformation relationship is used. , The point cloud of the straight line segment of the model tube is projected onto the point cloud region of the measuring tube 5. Using the principle that the point with the smallest distance is the corresponding point, the straight line segment of the measuring tube 5 is readjusted. Based on the least squares principle, the point cloud of the straight line segment of the adjusted measuring tube 5 is fitted with the axis. The intersection point of the axes of each adjacent straight line segment is solved, which is the calibrated bending point.

[0098] The method for adjusting the straight segment of measuring tube 5, which utilizes the principle that the minimum distance equals the corresponding point, includes:

[0099] Using the matrix transformation relationship of the solved bending points , The point cloud of the straight line segment of measuring tube 5 is The point cloud of the straight line segment of the model tube is , yes Point cloud of the straight segment in measuring tube 5 The minimum distance between nearest neighbors in a given context is defined as:

[0100]

[0101] By minimizing the aforementioned distance difference, the point cloud of the straight line segment of the model tube is determined to correspond to the point cloud of the straight line segment of the measurement tube 5.

[0102] The methods for fitting the axis of the point cloud of the straight line segment of the adjusted measuring tube 5 include:

[0103] The point cloud of a certain straight line segment of the adjusted measuring tube 5 is known to be... There are n points ;

[0104] The target line is required to pass through the point cloud. center of mass ,Right now:

[0105] ;

[0106] Point Cloud The goal is to minimize the distance from all points to the target line, expressed as the sum of the squares of the distances from all points to the line:

[0107] ;

[0108] in, It passed through the center of mass. The unit direction vector of the target line.

[0109] Expanding the above expression, we get:

[0110] ;

[0111] Where C is the covariance matrix, expressed as:

[0112] ;

[0113] To minimize the SSD, we need to find the eigenvector corresponding to the smallest eigenvalue of the covariance matrix. Therefore, the fitted line is obtained as follows:

[0114] ;

[0115] in, Let be the eigenvector corresponding to the smallest eigenvalue of the covariance matrix, and a be a constant coefficient.

[0116] The method for calibrating the bending point is as follows: The axis of two adjacent straight segments is obtained by solving the problem.

[0117] ;

[0118] Solving the above equations simultaneously, we get:

[0119] ;

[0120] The matrix is ​​obtained by rearranging:

[0121] ;

[0122] Solve the above matrix equation to determine the intersection point of the two axes, which is the bend point.

[0123] S5. Based on the matrix transformation relationship, the point cloud of the corresponding feature segment of the model tube is measured using the feature segment point cloud computing method, and the relevant data of the feature segment is restored; such as Figure 5 As shown in the figure, the process of restoring the feature flange is illustrated; in this embodiment, matrix transformation relationships are utilized. , The point cloud of the feature segment of the model tube is projected onto the point cloud region of the measuring tube 5. Using the principle that the point cloud of the feature segment of the measuring tube 5 is the point of correspondence with the smallest distance, the point cloud of the feature segment of the measuring tube 5 is readjusted. Based on the principle of least squares, the point cloud of the feature segment of the measuring tube 5 is fitted to the end face to determine the data of each feature segment of the measuring tube 5. Since the bending point and feature point 4 of the measuring tube 5 are known, the digitization of the measuring tube 5 can be completed, and the complete tube shape of the measuring tube 5 can be output.

[0124] Among them, the method for end-face fitting of the point cloud of the feature segment of the adjusted measuring tube 5 based on the least squares principle is as follows:

[0125] Given the point cloud of a certain feature segment of measuring tube 5 The fitted end face can be represented in the form of a point-normal vector as follows:

[0126] ;

[0127] in, It is the centroid of the point cloud; It is the unit normal vector of the fitted end face; Let be any point on the plane; the objective is to minimize the sum of the squared perpendicular distances from all points to the fitted end face, i.e.:

[0128] st ;

[0129] Due to the center of mass It is the centroid of the point cloud:

[0130] ;

[0131] If the point cloud is translated to a point with its centroid as the origin, then:

[0132]

[0133] The simplified end face equation is:

[0134] ;

[0135] Construct a covariance matrix A that reflects the distribution characteristics of the point cloud, i.e.:

[0136] ;

[0137] Find the eigenvector corresponding to the smallest eigenvalue of matrix A, which is the end face normal vector n.

[0138] Based on the known bending points and feature point 4 data, the method for digitizing the measuring tube 5 is as follows: The equations of the planes at the beginning and end of the measuring tube 5 are known, and the centroid of the plane is also the bending point of the measuring tube 5, referred to as the endpoint. According to the definition of the digitization parameters of the guide tube, the cylindrical point in the point cloud region where the bending segment and the straight segment meet, where the forward and backward tilt angles suddenly increase or decrease, is the tangent point of the bending segment. Combining the tangent point position, the advance amount, bending angle, and rotation angle of the measuring tube 5 are determined. The advance amount is the axial distance between the bending point and the tangent point near the straight segment of the measuring tube 5; the bending angle is the angle between the axes of adjacent straight segments; and the rotation angle is the angle between the planes containing two adjacent straight segments of the measuring tube 5.

[0139] S6. Apply the ICP algorithm to align the point clouds of the measuring tube 5 and the model tube. The resulting matrix transformation relationship is the measurement deviation.

[0140] Among them, the bending points and feature point cloud set of the measuring tube 5 are extracted. and the corresponding point cloud set of the model tube. ,like Figure 6 As shown, the ICP algorithm is applied to align two sets of point clouds, and the alignment error is solved. Minimal rotation matrix Translation matrix ;

[0141] in, For the x-th measurement point, Let x be the x-th model point, and H be the total number of points where the measuring tube and the model tube are aligned.

[0142] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for duct detection based on scanned point clouds, characterized in that, Includes the following steps: S1. Marking points are fixedly set on the scanner housing, and several feature points are evenly preset around the periphery of the measurement area. The camera system tracks the scanner moving along the scanning path and continuously captures the feature points. The feature point images captured at each scanning path point are used to calibrate the external parameter relationship of the camera system to complete the dynamic self-calibration of the camera system. The scanning point cloud corresponding to each scanning path point is converted to the global coordinate system where the feature points are located, and the initial point cloud of the measurement tube is obtained by stitching them together. S2. Evaluate the quality of the initial point cloud. If the evaluation criteria are met, proceed to step S3. If the evaluation criteria are not met, downsample the initial point cloud and determine candidate points. Search for nearby point cloud sets with the candidate points as the center, fit a cylinder, and determine the seed point through the normal vector relationship and the point distance relationship. Based on the seed point, establish a seed cylinder and spread it along the trend of the initial point cloud to form a complete tube shape. S3. Determine the straight segment and the bending segment based on the inclination angle change of adjacent cylindrical axes, fit the cylindrical axis of the straight segment and determine the bending point, apply the ICP algorithm to align the bending point of the measuring tube with the corresponding model tube, and obtain the matrix transformation relationship; S4. Adjust the straight segment of the measuring tube based on the matrix transformation relationship, fit the cylindrical axis of the straight segment and solve the intersection of adjacent axes to obtain the calibrated bending point, and solve the process data of the measuring tube based on the bending point coordinates. S5. Based on the matrix transformation relationship, the point cloud of the corresponding feature segment of the model tube is measured by applying the feature segment point cloud computing of the model tube, and the relevant data of the feature segment is restored. S6. Apply the ICP algorithm to align the point clouds of the measuring tube and the model tube, and the resulting matrix transformation relationship is the measurement deviation. In S2, the evaluation criterion is: the ratio of the initial point cloud area of ​​each feature region to the point cloud area at the corresponding position of the model tube is greater than or equal to the first preset threshold.

2. The duct detection method based on scanning point clouds according to claim 1, characterized in that, In S1, the dynamic self-calibration of the camera system includes: At each path point, the camera system reconstructs the scanner shell marker points and feature points around the measurement plane; given the coordinates of the marker points in the marker point coordinate system and the coordinates of the feature points in the global coordinate system, solve for the matrix transformation relationship M between the scanning coordinate system and the tracking coordinate system. track_scan =M track_mark *M mark_scan And the matrix transformation relationship M between the tracking coordinate system and the global coordinate system. global_track M track_scan M is the transformation matrix between the pre-determined scanning coordinate system and the tracking coordinate system. track_mark The transformation matrix from the marker coordinate system to the tracking coordinate system is represented by M, which is solved using the Singular Value Decomposition (SVD) method. mark_scan The transformation matrix from the scan coordinate system to the marker point coordinate system.

3. The duct detection method based on scanning point clouds according to claim 2, characterized in that, The expression for point cloud stitching described in S1 is: P global_scan =M global_Scan *P scan =M global_track *M track_mark *M mark_scan *P scan Where P global_scan For the measurement tube point cloud P in the scanning coordinate system scan M, the measurement tube point cloud in the global coordinate system global_Scan This represents the matrix transformation relationship between the scan coordinate system and the global coordinate system at the corresponding scan path point.

4. The duct detection method based on scanning point clouds according to claim 3, characterized in that, In S2, the process of determining the seed point is as follows: The point cloud of all candidate points and their vicinity is fitted into a cylinder using the least squares method. Based on principal component analysis (PCA), the axial direction of the corresponding cylinder is determined by applying the point cloud of candidate points and their vicinity. If the points in the candidate point and its vicinity that exceed the second preset threshold satisfy the condition... End the calculation and output the axis of the cylinder. Where i = 1, 2, 3, ..., n, n i Let be the normal vector of the candidate point and the i-th point in the point cloud. The cylindrical axis direction vector fitted to the point cloud of the candidate points and their vicinity; Find the center point of the candidate point and its nearby point cloud, combined with the existing cylindrical axis. Establish a projection plane and project the candidate points and their nearby point cloud onto the projection plane; Using the least squares method, fit a circle to all two-dimensional projection points, determine the center o and radius R of the fitted circle, and denote the distance d between the j-th projection point and the center of the circle. j ; If, in the point cloud of the candidate point and its vicinity, the points exceeding the third preset threshold satisfy 0 ≤ |d j -R|≤0.001, end the calculation, output the fitted circle center o as the center point of the cylinder corresponding to the candidate point, and determine the candidate point as the seed point; In S2, the process of forming a complete tubular shape is as follows: Randomly select an existing seed point as the search starting point, and move the seed point and the normal vector of the corresponding cylinder along the initial point cloud by one search step, using the set search step size as the unit. The seed point at the previous position and the normal vector of the corresponding cylinder are used as the initial values ​​for iteration. The point cloud near the position of the seed point after the move is applied to fit the cylinder and solve for the new cylinder center point and normal vector corresponding to the current position. This process continues until no more point clouds participate in cylinder fitting at the search position. Then, the search direction of the seed point is reversed, and the cylinder center point and normal vector corresponding to the point cloud at each search position are solved until all initial point clouds have completed cylinder fitting, resulting in a complete cylindrical model of the measuring tube.

5. The duct detection method based on scanning point clouds according to claim 4, characterized in that, The process for determining the bending point in S3 is as follows: Cylindrical axes with an inclination angle change less than or equal to an inclination angle threshold are classified as straight segments, while those with an inclination angle change greater than the inclination angle threshold are classified as bent segments. Based on the directions of all cylindrical axes of a straight segment, the axis of the straight segment is fitted using the least squares method, and the intersection of the axes of adjacent straight segments is determined as the bend point.

6. The duct detection method based on scanning point clouds according to claim 5, characterized in that, The ICP algorithm includes: Determine the point cloud set of the bending points of the measuring tube. Collection of point clouds of bending points in model tubes The correspondence is used to calculate the centroid μ. p centroid μ q and decentralized coordinates p i ' Decentralized coordinates q j ' Construct the covariance matrix and use SVD decomposition to obtain the rotation and translation matrices. Iterate and update until the alignment error is minimized to obtain the rotation matrix transformation relation R. bend Transformation relation t of translation matrix bend ; where p i It is the point cloud of the point cloud set P of the measurement tube bending points, q j It is the point cloud of the set Q of the bend points of the model tube, and q j It is p i The nearest neighbor in the point cloud set Q of the bending point of the model tube.

7. The duct detection method based on scanning point clouds according to claim 6, characterized in that, The bending point correction in S4 includes: utilizing the rotation matrix transformation relationship R bend Translation matrix transformation relationship t bend Project the point cloud of the straight segments of the model tube onto the measurement tube area, and readjust the straight segments of the measurement tube; fit the axis of the straight segments of the measurement tube using the least squares method, and solve for the intersection of adjacent axes as the calibrated bending point.

8. The duct detection method based on scanning point clouds according to claim 7, characterized in that, The feature segment in S5 includes at least one of the following: end face, flange, nut, and bayonet. The process of restoring the feature segment includes: fitting the end face of the feature segment based on the least squares principle, and determining the normal vector of the end face by solving the eigenvector corresponding to the smallest eigenvalue of the covariance matrix after the centroid of the point cloud is translated.

9. The duct detection method based on scanning point clouds according to claim 8, characterized in that, Solving matrix relationships in S6 includes: Extract the point cloud set of the bending points and feature points of the measuring tube. and the corresponding point cloud set of the model tube The ICP algorithm is applied to align two sets of point clouds, and the alignment error is calculated. Minimal rotation matrix R tube Translation matrix t tube , where p measure_i For the i-th measurement point, q standard_i Let i be the i-th model point.

10. A duct detection system based on scanned point clouds, characterized in that, include: A robotic arm is used to drive the scanner to move along a planned path; A scanner, fixed to the end of a robotic arm, has marking points on its housing; The camera system is used to track marker points and measure feature points on the periphery of the measurement area in real time; The data processing unit, connected to the scanner and camera system respectively, is configured to perform the method described in any one of claims 1-9.

Images