Method for accurate measurement of large diameter conduits

Abstract

The present application relates to a large diameter conduit precise measurement method. The steps are: firstly, calibrate the internal parameters of each camera, globally calibrate the position relationship among the plurality of cameras, determine the position of each camera and the relative position relationship among the other cameras, then synchronously collect the conduit images, collect a plurality of images under different exposure states for image fusion, extract the edge, skeleton and end face center point of the fused image, obtain the rough spatial three-dimensional form of the rough pipe by means of the traditional three-dimensional reconstruction method, project the obtained three-dimensional spatial points, establish a nonlinear objective function, optimize the objective function, and finally obtain the accurate three-dimensional spatial posture of the large diameter conduit. The method overcomes the phenomenon that the three-dimensional reconstruction precision of the conduit is low due to the inaccurate extraction of the central axis in the traditional cylindrical detection, has high measurement precision and high efficiency, and provides a strong guarantee for the assembly of the rough conduit.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically a method for precise measurement of large-diameter ducts. Background Technology

[0002] Aerospace engines have thousands of ducts of various shapes and sizes, which are responsible for the transmission of media such as cool air and fuel. They play a vital role in the normal operation of the entire engine. If there are problems with the ducts themselves, or if there is stress between the ducts or improper installation during the assembly process, it will threaten the safety of the entire aerospace industry. Therefore, the safety of duct assembly directly affects the safety of the entire aerospace industry. How to ensure the safety of every process of engine duct manufacturing, processing and assembly is a core technology of aerospace.

[0003] Currently, the assembly of ducts on engines in my country requires the measurement of the assembled pipes. The measurement methods generally include templates, coordinate measuring machines, and articulated coordinate measuring machines. These methods suffer from difficulties in manufacturing comparison devices, low efficiency, excessive reliance on operator subjective judgment, and difficulty in guaranteeing measurement accuracy. Operators need to perform multiple test tubes and adjustments during assembly, which not only results in extremely low installation efficiency but also places high demands on operators, thus limiting the rapid development of the entire aviation and aerospace technology.

[0004] The assembly of small-diameter conduits can sometimes be altered due to human error. Large-diameter conduits have high rigidity and large volume after molding, which brings greater difficulties to measurement and assembly. Operators need to test tubes more often, and inaccurate measurements or improper installation can have extremely adverse effects.

[0005] The absolute positioning accuracy of the pipe end and the measurement of the direction of the last straight segment of the conduit are particularly important for the entire assembly. In the study on "Research on the Measurement Method of Pipe End Coordinates Based on Multi-view Vision", Sun Peng et al. used projection tilt to combine the fitting contour and centerline endpoint positioning methods to calculate the conduit endpoints at different angles. However, the calculated endpoint accuracy was poor, and the direction of the two ends of the conduit could not be accurately obtained. Summary of the Invention

[0006] To address the aforementioned shortcomings in existing technologies, the present invention aims to provide a high-precision measurement method for large-diameter conduits. This method employs a nonlinear optimization approach to accurately measure large-diameter conduits, precisely analyzes the conduit ends and routing, and is simple to operate with high measurement efficiency, reducing the time spent on pipe fittings and test tubes during assembly.

[0007] The technical solution adopted by the present invention to achieve the above objectives is: a method for accurate measurement of large-diameter catheters, comprising the following steps:

[0008] A method for precise measurement of large-diameter catheters includes the following steps:

[0009] Step 1: Perform multi-view vision calibration on multiple cameras to obtain the intrinsic and extrinsic parameters of all cameras;

[0010] Step 2: Simultaneously trigger multiple cameras to acquire multiple sets of original images of the duct on the backlight panel from multiple angles;

[0011] Step 3: Perform brightness fusion on the image sequences at the same angle to obtain a fused image for each angle; and perform steps 4-6 for each angle respectively;

[0012] Step 4: Use the Canny edge detection algorithm to obtain the sub-pixel edges of the large-diameter ducts in the fused image;

[0013] Step 5: Obtain the central axis of the entire catheter using the skeleton extraction method;

[0014] Step 6: Discretize the sub-pixel edges and calculate the tangent vectors corresponding to the discrete points; use the ellipse fitting method to obtain the projection point of the end center of the discrete points of the duct tip edge;

[0015] Step 7: Use epipolar constraints to determine the corresponding points of the central axis of multiple angle images and the corresponding points of the projection point of the end center; reconstruct the central axis of the entire thick pipe and the three-dimensional coordinate position of the end based on the positional relationship between multiple cameras and the forward intersection method; according to the radius of the thick pipe, sweep along the central axis with the end as the starting point to obtain the three-dimensional spatial model of the thick pipe.

[0016] Step 8: Reproject the straight segments and ends of the reconstructed three-dimensional spatial model of the thick pipe onto the image at the corresponding angle, and use a nonlinear optimization method to obtain an accurate three-dimensional spatial model of the thick pipe and the orientation of the two ends of the duct.

[0017] The intrinsic and extrinsic parameters are used to obtain the positional relationship between multiple cameras.

[0018] The multiple sets of image sequences have different exposure parameters for each exposure.

[0019] The brightness fusion is a weighted average of the brightness values ​​at each pixel position of multiple sets of image sequences.

[0020] The formula for the weighted average of the brightness values ​​is:

[0021]

[0022] in i, j, k represent the number of rows, columns, and image sequence, respectively; I and W represent the brightness and weight of the image; and F is the fused image.

[0023] The discrete points on the image edge include discrete points on the edge of the middle part of the duct region and discrete points on the end edge.

[0024] The accurate three-dimensional model of the thick pipe and the orientation of both ends of the duct were obtained using a nonlinear optimization method, and the following processing method was adopted:

[0025] i. The orientation of the duct tip includes the spatial position of the center point of the tip and the direction vector of the straight segment of the tip; the tip and straight segment region of one side of the three-dimensional spatial model of the thick tube are discretized into point clouds, and all the discretized point clouds are back-projected onto the image at the corresponding angle according to the transformation relationship between the camera coordinate system and the global coordinate system, and the projection points on the straight segment are obtained according to the maximum encirclement principle.

[0026] ii. Using the edge points and corresponding tangent vectors of each angle image as input values, and the spatial orientation (x0, y0, z0, l0, m0, n0) of the end center point as the parameters to be optimized, the objective function is established as follows:

[0027]

[0028] Where (x0, y0, z0) represents the initial three-dimensional spatial coordinates of the end center point, (l0, m0, n0) represents the initial direction vector of the end line segment, and f(x, y) = ax 2 +by 2 +cxy+dx+ey+m,T p Let f(x,y) be the tangent vector of point P on f(x,y), S be the projection point of the end face, g(x,y) = ax + by + c, (x,y) represent the image coordinate points, and a, b, c represent the linear parameters fitted to the straight line segment of the edge extraction duct.

[0029] iii. Nonlinear optimization minimizes the objective function, continuously updates the current optimization variables, causing the objective function to decrease, and obtains the orientation of both ends of the conduit.

[0030] The nonlinear optimization minimization objective function is as follows:

[0031] Given initial values ​​(x0, y0, z0, l0, m0, n0), the Gauss-Newton method is used to perform multiple iterations. For the k-th iteration, an increment (Δx, Δy, Δz, Δl, Δm, Δn) is found to minimize the objective function, and the iteration ends.

[0032] Finally, the precise single-end orientation of the catheter was obtained (x0+Δx,y0+Δy,z0+Δz,l0+Δl,m0+Δm,n0+Δn).

[0033] The obtained precise three-dimensional spatial model of the catheter and the orientation of both ends of the catheter are used for positioning during catheter assembly and operation.

[0034] A precision measurement system for large-diameter conduits includes: a backlight panel, multiple sets of industrial cameras, and a PC. The large-diameter conduit to be measured is placed on the backlight panel. The multiple sets of industrial cameras are used to capture multiple sets of raw images of the conduit and transmit them to the computer. The PC includes a processing unit and a storage unit. The storage unit stores a program, and the processing unit loads the program and executes the steps of the method as described in any one of claims 1-8 to realize the measurement of the three-dimensional spatial model of the large-diameter conduit and the orientation of both ends of the conduit.

[0035] The present invention has the following beneficial effects and advantages:

[0036] 1. The method for accurate measurement of large-diameter catheters described in this invention can effectively measure large-diameter catheters accurately, and is also suitable for thin catheters.

[0037] 2. The large-diameter catheter precision measurement method described in this invention has high measurement accuracy and can provide effective protection for stress-free catheter assembly.

[0038] 3. The large-diameter catheter precision measurement method described in this invention has high measurement efficiency, which is thousands of times faster than contact measurement methods, laying a solid foundation for the digital manufacturing of catheters. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the hardware device for the method of the present invention;

[0040] Figure 2 This is a flowchart of the method of the present invention;

[0041] Figure 3a , Figure 3b These are the two pipe end projection images coarsely reconstructed according to the present invention;

[0042] Figure 3c This is the final image of the accurate projection measurement result of this invention. Detailed Implementation

[0043] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0045] See attached document Figure 1 The hardware device of the method of the present invention includes: a backlight panel, a duct to be inspected, four industrial cameras with a large field of view, and a computer as a controller. The four industrial cameras perform image processing and 3D reconstruction on the duct to obtain the endpoints and orientation of the large-diameter duct. The computer is equipped with programs for each step of various algorithm detection steps. The computer, in conjunction with the other device, ultimately detects the endpoints and orientation of the large-diameter duct and outputs the results to a host computer. The host computer can then control an actuator to assemble the duct onto a pre-connected device based on the detected endpoints and orientation, thus achieving duct assembly.

[0046] See attached document Figure 2 The specific steps of the large-diameter catheter precision measurement method described in this invention are as follows:

[0047] Step 1: Perform multi-view vision calibration on the four industrial cameras to obtain the intrinsic parameters of all cameras and the transformation matrix from the camera coordinate system to the world coordinate system, which is the extrinsic parameter.

[0048] The intrinsic parameters represent the transformation matrix from the industrial camera coordinate system to the image coordinate system; the distortion degree of each lens is different, including radial distortion and eccentric distortion; the intrinsic parameters of each industrial camera are calibrated using a two-dimensional flexible target.

[0049] The extrinsic parameters represent the transformation matrix from the industrial camera coordinate system to the world coordinate system, implemented through the rotation matrix R and the translation matrix T. The world coordinate system is used for subsequent output of the actual control point coordinates.

[0050] Step 2: Simultaneously trigger four industrial cameras to acquire multiple sets of raw duct images placed statically on a backlight plate with different exposures;

[0051] Step 3: Take a weighted average of the brightness values ​​of each pixel in an image sequence (e.g., 10 consecutive frames) taken by the same camera under different exposures to obtain a fused image result at each angle. This can overcome the phenomenon of missing duct edge information in high-exposure areas due to material properties, and obtain higher contrast and richer detail information. Then perform steps 4-6 at each angle.

[0052] Image sequences under different exposures were processed in the following way:

[0053]

[0054] in i, j, k represent the number of rows, columns, and image sequence, respectively; I and W represent the brightness and weight of the image; and F is the fused image.

[0055] Step 4: Using the Canny edge detection algorithm, the edges of the fused image are extracted to obtain the sub-pixel edges of the large-diameter duct.

[0056] Step 5: Extract the skeleton from the sub-pixel edge information of the image to obtain the skeleton of the entire duct, which is the central axis;

[0057] Step 6: Discretize the sub-pixel edges obtained in Step 4 and calculate the discrete point tangent vectors to obtain discrete points on the image edges and their corresponding tangent vectors. The discrete points on the image edges include the discrete points of the middle part of the conduit region and the discrete points of the end edges. The projection points of the end center are obtained by using ellipse fitting for the discrete points of the end edges.

[0058] The method for obtaining the tangent vector is as follows:

[0059] Given a discrete point P, and a point Q on the edge adjacent to P, the direction vector of PQ as point Q approaches P along the edge is the tangent vector of point P. By iterating through all discrete points, we can obtain the tangent vectors of the corresponding points.

[0060] Step 7: Using epipolar constraints, determine the corresponding points of the central axis of the four images and the corresponding points of the projection point of the end center. With the help of the positional relationship between the four cameras obtained by global calibration and the forward intersection method in surveying, reconstruct the three-dimensional central axis of the thick pipe and the spatial coordinate position of the end. According to the pre-given radius of the thick pipe, sweep along the center line with the end as the starting point to obtain the three-dimensional spatial model of the thick pipe.

[0061] Step 8: To ensure stress-free assembly of the thicker tube during the assembly process, the reconstructed straight section and ends of the conduit are reverse-projected onto the image coordinate system of the four cameras. A nonlinear optimization method is then used to obtain an accurate 3D model of the thicker tube and the orientation of both ends. Specifically:

[0062] i. Taking one end of the conduit as an example, the direction of the conduit end includes the spatial position of the center point of the end and the direction vector of the straight segment of the end. Take the point cloud discretization of the one end of the conduit and the straight segment region of the end in the three-dimensional model of the thick tube obtained in step 7. Perform back projection on all the discretized point clouds according to the transformation relationship between the camera coordinate system and the global coordinate system, and project them onto the four angle image coordinate systems respectively. Obtain the projection points on the edge of the tube end and the straight segment of the tube end according to the maximum encirclement principle.

[0063] ii. Using the edge points of each image obtained in step 6, as shown by the edge contour pixels of the two ends in Figures 3(a) and 3(b), and the corresponding tangent vectors as input values, with the goal of infinitely approximating the original points on the four angle image coordinate systems by the projection points, as shown by the edge contour pixels of the entire thick pipe in Figure 3(c), and with the spatial orientation (x0, y0, z0, l0, m0, n0) of the end center point as the parameter to be optimized, establish the objective function as follows:

[0064]

[0065] Where (x0, y0, z0) represents the initial three-dimensional spatial coordinates of the end center point, (l0, m0, n0) represents the initial direction vector of the end line segment, and f(x, y) = ax 2 +by 2 +cxy+dx+ey+m, f(x,y) represents the parametric equation of the image ellipse, T p Let f(x,y) be the tangent vector of point P on f(x,y), S be the projection point of the end face, g(x,y) = ax + by + c, (x,y) represent the image coordinate points, and (a,b,c) represent the linear parameters of the fitting of the straight line segment of the duct for edge extraction.

[0066] iii. Nonlinear optimization minimizes the objective function, continuously updating the current optimization variables to reduce the objective function. The specific implementation method is as follows:

[0067] Given initial values ​​(x0, y0, z0, l0, m0, n0), the Gauss-Newton method is used for multiple iterations. For the k-th iteration, an increment (Δx, Δy, Δz, Δl, Δm, Δn) is found that minimizes the objective function, at which point the iteration terminates. The spatial 3D model is adjusted according to this increment to obtain the accurate orientation of the two ends of the duct.

[0068] The final precise orientation of the two ends of the catheter is (x0+Δx, y0+Δy, z0+Δz, l0+Δl, m0+Δm, n0+Δn).

[0069] Step 9: Using the obtained accurate three-dimensional model of the catheter, catheter assembly and other operations can be performed.

[0070] The above-described procedures are loaded onto the on-site computer. Four industrial cameras are set up on-site to capture images, which are then sent to the on-site computer and processed according to the above steps to obtain the three-dimensional spatial position of the large-diameter conduit. Finally, the precise end position S and orientation values ​​(x0+Δx, y0+Δy, z0+Δz, l0+Δl, m0+Δm, n0+Δn) are obtained and transmitted to the robot or other equipment for further operations.

[0071] The method was tested using VS2015 on a Win10 system. The repeatability can reach 0.1mm, the measurement time is about 10s, and the measurement accuracy is high, the efficiency is high, and the stability is strong.

[0072] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should be considered within the scope of protection of the present invention.

Claims

1. A method for precise measurement of large-diameter catheters, characterized in that, Includes the following steps: Step 1: Perform multi-view vision calibration on multiple cameras to obtain the intrinsic and extrinsic parameters of all cameras; Step 2: Simultaneously trigger multiple cameras to acquire multiple sets of original images of the duct on the backlight panel from multiple angles; the exposure parameters of each set of original images are different. Step 3: Perform brightness fusion on the image sequences at the same angle to obtain a fused image for each angle; and execute steps 4-6 for each angle; the brightness fusion is to perform a weighted average of the brightness values ​​at each pixel position of multiple sets of image sequences; Step 4: Use the Canny edge detection algorithm to obtain the sub-pixel edges of the large-diameter ducts in the fused image; Step 5: Obtain the central axis of the entire catheter using the skeleton extraction method; Step 6: Discretize the sub-pixel edges and calculate the tangent vectors corresponding to the discrete points; use the ellipse fitting method to obtain the projection point of the end center of the discrete points of the duct tip edge; Step 7: Use epipolar constraints to determine the corresponding points of the central axis of multiple angle images and the corresponding points of the projection point of the end center; reconstruct the central axis of the entire thick pipe and the three-dimensional coordinate position of the end based on the positional relationship between multiple cameras and the forward intersection method; according to the radius of the thick pipe, sweep along the central axis with the end as the starting point to obtain the three-dimensional spatial model of the thick pipe. Step 8: Reproject the straight segments and ends of the reconstructed three-dimensional spatial model of the thick pipe onto the image at the corresponding angle, and use a nonlinear optimization method to obtain an accurate three-dimensional spatial model of the thick pipe and the orientation of the two ends of the duct.

2. The method for precise measurement of large-diameter catheters according to claim 1, characterized in that, The intrinsic and extrinsic parameters are used to obtain the positional relationship between multiple cameras.

3. The method for precise measurement of large-diameter catheters according to claim 1, characterized in that, The formula for the weighted average of the brightness values ​​is: ； in , These represent the number of rows, columns, and the image sequence, respectively. The values ​​represent the brightness and weight of the image, and F is the image after fusion.

4. The method for precise measurement of large-diameter catheters according to claim 1, characterized in that, The discrete points on the image edge include discrete points on the edge of the middle part of the duct region and discrete points on the end edge.

5. The method for precise measurement of large-diameter catheters according to claim 1, characterized in that, The accurate three-dimensional model of the thick pipe and the orientation of both ends of the duct were obtained using a nonlinear optimization method, and the following processing method was adopted: i. The orientation of the duct tip includes the spatial position of the center point of the tip and the direction vector of the straight segment of the tip; the tip and straight segment region of one side of the three-dimensional spatial model of the thick tube are discretized into point clouds, and all the discretized point clouds are back-projected onto the image at the corresponding angle according to the transformation relationship between the camera coordinate system and the global coordinate system, and the projection points on the straight segment are obtained according to the maximum encirclement principle. ii. Using the edge points and corresponding tangent vectors of each angle image as input values, and the spatial orientation of the end center point... The objective function is established as follows for the parameters to be optimized: ； in, This represents the initial three-dimensional spatial coordinates of the end center point. This represents the initial direction vector of the end line segment. , for The tangent vector to point P, where S is the projection point of the end face. (x,y) represents the image coordinates, and a,b,c represent the linear parameters fitted to the straight line segment of the edge extraction duct. iii. Nonlinear optimization minimizes the objective function, continuously updates the current optimization variables, causing the objective function to decrease, and obtains the orientation of both ends of the conduit.

6. The method for precise measurement of large-diameter catheters according to claim 1, characterized in that, The nonlinear optimization minimization objective function is as follows: Given initial value Using the Gauss-Newton method, multiple iterations are performed. For the k-th iteration, an increment is found. This causes the objective function to reach its minimum value, at which point the iteration ends. Ultimately, a precise single-end orientation of the catheter is obtained. .

7. The method for precise measurement of large-diameter catheters according to claim 1, characterized in that, The obtained precise three-dimensional spatial model of the catheter and the orientation of both ends of the catheter are used for positioning during catheter assembly and operation.

8. A precision measurement system for large-diameter catheters, characterized in that, include: The system comprises a backlight panel, multiple sets of industrial cameras, and a PC. A large-diameter conduit to be measured is placed on the backlight panel. The multiple sets of industrial cameras are used to capture multiple sets of raw images of the conduit and transmit them to the computer. The PC includes a processing unit and a storage unit. The storage unit stores a program, and the processing unit loads the program and executes the steps of the method described in any one of claims 1-6 to realize the measurement of the three-dimensional spatial model of the large-diameter conduit and the orientation of both ends of the conduit.

Images