A method for precise measurement of catheter endpoints

Abstract

This invention discloses a method for accurately measuring the endpoint of a conduit. The steps are as follows: calibration using multi-view vision to acquire an image of the curved pipe; removal of image distortion and image correction based on the parameters of each camera; segmentation of the pipe region and extraction of the pipe edges; fitting the centerline of the conduit image; determining the spatial position of the centerline based on epipolar constraints; sampling n points on the spatial centerline; establishing a central plane using the camera center and two of the sampling points; calculating the intersection point of the central plane and the circumference of the curved pipe end face, and determining the specific position of the intersection point using the radius of the curved pipe and the sampling points; projecting this intersection point onto the spatial centerline, where the projected point is the conduit endpoint. This method does not rely on ellipse fitting, thus avoiding errors caused by ellipse fitting. Furthermore, this method does not first search for the projection point in the image and then determine the actual position of the endpoint, but directly determines the position of the endpoint in space, overcoming the drawback of the projection point's position differing in different images.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically a method for precise measurement of duct endpoints. Background Technology

[0002] Conduits are widely used in electromechanical equipment in fields such as aviation and aerospace, primarily responsible for transporting liquid or gaseous working media such as fuel and coolant. Therefore, any problem with a conduit in such equipment will affect the entire system. As products become increasingly sophisticated and their internal structures more compact, high precision is required for conduits, especially at the ports where they connect to other conduits or components. Therefore, to ensure the quality of conduit manufacturing and achieve stress-free assembly, post-processing conduit inspection becomes crucial.

[0003] Methods for detecting bent pipes are mainly divided into two categories: contact and non-contact. Contact methods use instruments such as coordinate measuring machines (CMMs) to measure the bent pipe in contact. These methods are highly accurate and technologically mature, but they also have many drawbacks. First, the detection speed is slow; a CMM can often take several hours to detect a single object, making it unsuitable for large-scale testing. Second, the detection range is limited; these instruments cannot measure bent pipes with complex structures. Furthermore, these high-precision instruments are usually expensive and have stringent environmental requirements. Non-contact methods are mainly based on computer vision, and can be further subdivided into photometric stereo, structured light measurement, and multi-view vision methods, depending on the technical principles. These methods achieve the measurement purpose through three-dimensional reconstruction of the object. They have advantages such as short measurement time and simple equipment. In recent years, with the improvement of computer capabilities and the reduction of camera costs, this technology has been widely applied and developed in many fields. However, due to the complex structure, mutual occlusion, lack of obvious texture features, and reflective properties of bent pipes, methods such as photometric stereo and structured light measurement, which use forward projection, cannot effectively measure bent pipes. Multi-view vision technology based on backlight projection can capture images from multiple angles, effectively avoiding the aforementioned problems, and is a relatively ideal method for measuring ducts.

[0004] The endpoint coordinates determine the starting position and total length of the catheter, and also affect the positioning of the catheter centerline. Therefore, accurately determining the endpoints is crucial but also a very challenging problem during catheter measurement. After perspective imaging, part of the catheter end face appears as an arc in the image, while another part overlaps with the catheter and becomes inseparable. Because the arc of the end face edge observed by cameras from different perspectives is different, and there is no direct correspondence between the midpoint of the arc and the endpoint, endpoint detection becomes extremely difficult.

[0005] The current mainstream idea is to fit an ellipse based on an arc and use the center of the ellipse as the projection point of the endpoint on the image. This type of method has the following problems: (1) If the elliptical arc extracted from the image is irregular or incomplete, it will have a great impact on the calculation of the endpoint position. (2) The center of the circle fitted from multiple images will have a certain error, so the endpoints of the reconstructed space using these images will be randomly distributed within a certain area. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention discloses a method for accurately measuring the endpoint of a catheter. This method avoids errors caused by fitting an ellipse and directly determines the endpoint's position in space, thus overcoming the drawback of the projection point's position varying across different images.

[0007] The technical solution adopted by the present invention to achieve the above objectives is: a method for accurate measurement of catheter endpoints, comprising the following steps:

[0008] S1. Perform internal parameter calibration, external parameter calibration, and multi-view vision calibration of cameras deployed at multiple viewpoints, with any camera coordinate system selected as the reference.

[0009] S2. Control multiple cameras from different angles to simultaneously capture images of the original bent tube placed on the backlight panel;

[0010] S3. Perform image preprocessing on the images from each angle in sequence, calculate the center line of the duct image, establish the center plane, find the point F on the circumference of the single end face of the curved tube in the reference camera coordinate system that has a unique projection relationship with the point f1 on the outer contour of the single end face of the duct on the image plane; calculate the endpoint L1 of one end face of the duct.

[0011] S4. In the reference camera coordinate system, the endpoint coordinates of one side face of multiple ducts are optimized by mean averaging.

[0012] The catheter endpoint is the center of the catheter end face.

[0013] The intrinsic parameter is the transformation matrix from the industrial camera coordinate system to the image coordinate system, and the extrinsic parameter is the transformation matrix from the camera coordinate system to the world coordinate system.

[0014] The steps of image preprocessing, calculating the center line of the duct image, establishing the center plane, and finding a point F on the circumference of the single-sided end face of the curved tube in the reference camera coordinate system that has a unique projection relationship with the point f1 on the outer contour of the single-sided end face of the duct on the image plane include:

[0015] Step a: Dedistort the image and perform image correction;

[0016] Step b: Segment the pipeline region from the image and extract the pipeline edges using sub-pixel edge extraction;

[0017] Step c: Fit the center line L of the catheter image;

[0018] Step d: Perform 3D reconstruction in the reference camera coordinate system to obtain the spatial straight line equation of the image centerline;

[0019] Step e: Sample n points on the spatial centerline L;

[0020] Step f: Establish a central plane using the camera center O and two sampling points. The central plane intersects the circumference of the single-sided end face of the curved pipe at two points F1 and G1. According to the discrimination principle, select the point that has a unique projection relationship with the outer contour point f1 of the single-sided end face of the conduit on the image plane from the two intersection points F1 and G1, and take it as F.

[0021] Step g: Calculate the coordinates of point F based on the intersection of centerline L and ray Of1 at point F1';

[0022] Step h: Project the point F onto the spatial centerline L to obtain the projection point L1, which is the endpoint of one side of the duct face.

[0023] The distortions removed in step 2 include radial distortion caused by the lens shape and tangential distortion caused by the camera assembly process.

[0024] The meaning of image correction in step 2 is to convert the image into an epipolar standard geometry.

[0025] The discrimination principle is as follows: if the angle β between ray Of1 and the spatial center line L is acute or right, select point F1 which is closer to the camera center point O; otherwise, select point G1 which is farther away from the camera center point O as F.

[0026] The discrimination principle is as follows: calculate J = |Of1×l|;

[0027] If J≥0, then the point F on the circumference of one end face of the bend and the point f1 on the outer contour of the single end face of the duct on the image plane that has a unique projection relationship is the intersection point F1 closer to the camera side; if J<0, point F is the intersection point G1 farther away from the camera.

[0028] The process of solving the F coordinate is as follows:

[0029] Calculate the intersection point F′1 of ray Of1 and the center line in space;

[0030] If F is a point closer to the camera center point O, then move F′1 along Of1 towards point O by R / sinβ; if F is a point farther from the camera center point O, then move F1′ along Of1 in the opposite direction towards point O by R / sinβ; thus obtaining the coordinates F(x) F ,y F ,zF ); where R is the radius.

[0031] The point F is projected onto the spatial centerline L to obtain the projection point L1, and the endpoint L1(x) L ,y L ,z L Based on the coordinates of sampling point X1(x1,y1,z1) and F(x F ,y F ,z F Calculation of coordinates of )

[0032]

[0033] Where X1(x1,y1,z1) are the coordinates of the sampling point X1, and F(x F ,y F ,z F Let f1 be the coordinate of point F1 on the circumference of the single-sided end face of the curved pipe in the reference camera coordinate system, which has a unique projection relationship with point f1 on the outer contour of the single-sided end face of the duct in the image plane. Let (m,n,p) be the direction vector of the spatial centerline, and (x L1 ,y L1 ,z L1 ) represents the endpoint to be determined. Attached Figure Description

[0034] Figure 1 A schematic diagram of the hardware device of the present invention.

[0035] Figure 2 The method flowchart of the present invention.

[0036] Figure 3 Front view of the measurement method model of the present invention.

[0037] Figure 4 Side view of the measurement method model of the present invention. Detailed Implementation

[0038] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0040] See attached document Figure 1 The hardware device of this invention includes: a white parallel backlight source, an industrial computer, a conduit under test, and 16 industrial cameras. The conduit under test is placed on the backlight source, and the 16 industrial cameras are evenly distributed in four directions to capture images of the conduit end from different angles.

[0041] See attached document Figure 2 Appendix Figure 3 Appendix Figure 4 In order to attach Figure 3 Taking endpoint L1 as an example, the specific steps of the catheter endpoint precision measurement method of the present invention are as follows:

[0042] Step 1: First, calibrate the 16 cameras and obtain the intrinsic and extrinsic parameters of each camera. Then, using the coordinate system of the first camera as a reference, calculate the transformation matrix from all camera coordinate systems to this first coordinate system to achieve multi-view vision calibration. Next, place the bent pipe to be tested into the camera's measurement area and acquire the image of the bent pipe. Since the pipes produced by the pipe bending machine have straight sections at both ends, for ease of explanation, the steps described in this method will all be illustrated using a straight pipe as an example. The intrinsic parameters are the transformation matrix from the industrial camera coordinate system to the image coordinate system, and the extrinsic parameters are the transformation matrix from the camera coordinate system to the world coordinate system.

[0043] For images from various angles, perform steps 2-8 sequentially to obtain the endpoint of one side of the catheter, i.e., the center of the end face L1.

[0044] Step 2: Based on the obtained current camera parameters, remove distortion from the acquired image and perform image correction. The removed distortion includes radial distortion caused by the lens shape and tangential distortion caused during camera assembly. Image correction means converting the image into an epipolar standard geometry. Step 3: For each image, segment the pipeline region from the image, and then use sub-pixel edge extraction to extract the edges of the pipeline.

[0045] Step 4: Fit the center line of the catheter image; The image center line, also known as the skeleton line, is defined as: the straight line formed by fitting the largest possible circle in a series of cross-sections perpendicular to the catheter.

[0046] Step 5: Perform 3D reconstruction to obtain the spatial line segment of the image center line and the spatial line L1L2 in which it is located;

[0047] Step 6: Sample along the spatial straight line segment to obtain n sampling points. The reasons for sampling the spatial straight line segment are: firstly, to calculate the central plane; and secondly, in subsequent steps, the central plane will intersect with one end face of the bend at two points, but only one point will be projected into the image. The sampled points can be used to determine which of the two points is projected into the image.

[0048] Since the image of the duct on the image plane is its outer contour, only one of the two intersection points F1 and G1 between the central plane and the circumference of one end face of the curved duct can be displayed on the outer contour of the image plane (i.e., only one of points F1 and G1 has a unique projection relationship with point f1 on the outer contour of the duct's one end face on the image plane). The point with a unique projection relationship with point f1 on the outer contour of the duct's one end face on the image plane is either the intersection point F1 closer to the camera or the intersection point G1 farther from the camera. The specific operation steps are as follows:

[0049] a. The central plane intersects one side of the duct image at f1. Take two sampling points (X1(x1,y1,z1), X2(x2,y2,z2)) to calculate the direction vector (m,n,p) of the spatial centerline;

[0050] With sampling point X1(x1,y1,z1) and the direction vector of ray Of1 as (x f1 s,y f1 s,z f1 s), where (parameter s>0). Then the intersection point F′1 of Of1 and the spatial centerline can be solved using the following system of equations:

[0051] i. Calculate the values ​​of parameters s and t when ray Of1 intersects the spatial centerline according to the following equation;

[0052]

[0053] ii. Based on the parameters s and t, the coordinates of the intersection point F1′(x′1,y′1,z′1) can be obtained.

[0054] b. If t is 0, then select a new point X3 from the n sampling points to replace one of the two sampling points in step a, and repeat steps a and b to calculate the coordinate F1′.

[0055] If t is not 0, then calculate the direction vector of the spatial centerline:

[0056] l=τ(t)*(x1-x′1, y1-y′1, z1-z′1)

[0057] in:

[0058] c. The following two criteria can be used for judgment:

[0059] i. Calculate J = |Of1×l|;

[0060] If J≥0, then the point F on the circumference of one end face of the bend and the point f1 on the outer contour of the single end face of the duct on the image plane that has a unique projection relationship is the intersection point F1 closer to the camera side; if J<0, it is the intersection point G1 farther away from the camera.

[0061] ii. Alternatively, the value of β can be obtained from J = |Of1×l| = |Of1|·|l|·sinβ;

[0062] If the angle β between ray Of1 and the spatial center line L is acute or right, select point F1 that is closer to the camera center point O; otherwise, select point G1 that is farther away from the camera center point O as F.

[0063] Step 7: If F is the intersection point G1 closest to the camera, then G1 moves R / sinβ along Of1 towards point O. If F is a point F1 far from the center point, then F1 moves R / sinβ in the opposite direction along Of1 towards point O, thus obtaining the coordinates of F.

[0064] Step 8: Project point F onto the center line of space; the projection point is the endpoint L1 of the end face on one side of the duct; endpoint L1(x L1 ,y L1 ,z L1 The formula for calculating ) is:

[0065]

[0066] Where X1(x1,y1,z1) are the coordinates of the sampling point X1, and F(x F ,y F ,z F Let f be the coordinates of point F on the circumference of the single-sided end face of the curved pipe in the reference camera coordinate system, which has a unique projection relationship with point f1 on the outer contour of the single-sided end face of the duct in the image plane. Let (m,n,p) be the direction vector of the spatial centerline, and (x) be the coordinates of point F on the circumference of the single-sided end face of the curved pipe in the reference camera coordinate system, which has a unique projection relationship with point f1 on L1 ,y L1 ,z L1 ) represents the endpoint to be determined.

[0067] The coordinates of the endpoints of the 16 catheter side faces obtained from the images at various angles are averaged and optimized to obtain the optimized three-dimensional coordinates of the endpoint L1 of the catheter side face based on the coordinate system of the first camera.

[0068] Following the steps described above, the three-dimensional coordinates of the endpoint L2 on the other side of the duct, optimized with reference to the coordinate system of the first camera, can be calculated.

[0069] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for accurately measuring the tip of a catheter, characterized in that, Includes the following steps: S1. Perform internal parameter calibration, external parameter calibration, and multi-view vision calibration of cameras deployed at multiple viewpoints, with any camera coordinate system selected as the reference. S2. Control multiple cameras from different angles to simultaneously capture images of the original bent tube placed on the backlight panel; S3. Perform image preprocessing on the above images from various perspectives in sequence, calculate the center line of the duct image, establish the center plane, find the point F on the circumference of the single end face of the curved tube in the reference camera coordinate system that has a unique projection relationship with the point f1 on the outer contour of the single end face of the duct on the image plane; calculate the endpoint L1 of one end face of the duct. The endpoint of the conduit is the center of the conduit end face; the establishment of the central plane, and finding a point F on the circumference of the single-sided end face of the bend in the reference camera coordinate system that has a unique projection relationship with the point f1 on the outer contour of the single-sided end face of the conduit on the image plane, includes: Step e: Sample n points on the spatial centerline L; Step f: Establish a central plane using the camera center O and two sampling points. The central plane intersects the circumference of the single-sided end face of the curved pipe at two points F1 and G1. According to the discrimination principle, select the point among the two intersection points F1 and G1 that has a unique projection relationship with the outer contour point f1 of the single-sided end face of the conduit on the image plane, and take it as F. The discrimination principle is: if the angle β between ray Of1 and the spatial center line L is acute or right, select the point F1 that is closer to the camera center point O; otherwise, select the point G1 that is farther away from the camera center point O, and take it as F. Step g: Calculate the coordinates of point F based on the intersection of centerline L and ray Of1 at point F1'; the process of solving for the coordinates of F is as follows: Calculate the intersection point F′1 of ray Of1 and the center line in space; If F is a point closer to the camera center point O, then move F′1 along Of1 towards point O by R / sinβ; if F is a point farther from the camera center point O, then move F′1 along Of1 in the opposite direction towards point O by R / sinβ; thus obtaining the coordinates F(x) F ,y F ,z F ); where R is the radius; Step h: Project the point F onto the spatial centerline L to obtain the projection point L1, which is the endpoint of one side of the duct face; S4. In the reference camera coordinate system, the endpoint coordinates of one side of the duct obtained from multiple angle images are optimized by mean averaging.

2. The method for accurately measuring the catheter tip according to claim 1, characterized in that, The intrinsic parameter is the transformation matrix from the industrial camera coordinate system to the image coordinate system, and the extrinsic parameter is the transformation matrix from the camera coordinate system to the world coordinate system.

3. The method for accurately measuring the catheter tip according to claim 1, characterized in that, The steps of image preprocessing, calculating the center line of the duct image, establishing the center plane, and finding a point F on the circumference of the single-sided end face of the curved tube in the reference camera coordinate system that has a unique projection relationship with the point f1 on the outer contour of the single-sided end face of the duct on the image plane include: Step a: Dedistort the image and perform image correction; Step b: Segment the pipeline region from the image and extract the pipeline edges using sub-pixel edge extraction; Step c: Fit the center line L of the catheter image; Step d: Perform 3D reconstruction in the reference camera coordinate system to obtain the spatial straight line equation of the image centerline.

4. The method for precise measurement of catheter tip according to claim 3, characterized in that, The distortions removed in step a include radial distortion caused by the lens shape and tangential distortion caused by the camera assembly process.

5. The method for precise measurement of catheter tip according to claim 3, characterized in that, The image correction mentioned in step a means: converting the image into an epipolar standard geometry.

6. The method for precise measurement of catheter tip according to claim 3, characterized in that, The discrimination principle is as follows: calculate J = |Of1×l|; If J≥0, then the point F on the circumference of one end face of the bend and the point f1 on the outer contour of the single end face of the duct on the image plane that has a unique projection relationship is the intersection point F1 closer to the camera side; if J<0, point F is the intersection point G1 farther away from the camera.

7. The method for accurately measuring the catheter tip according to claim 3, characterized in that, The point F is projected onto the spatial centerline L to obtain the projection point L1, and the endpoint L1(x) L ,y L ,z L Based on the coordinates of sampling point X1(x1,y1,z1) and F(x F ,y F ,z F Calculation of coordinates of ) Where X1(x1,y1,z1) are the coordinates of the sampling point X1, and F(x F ,y F ,z F Let f1 be the coordinate of point F1 on the circumference of the single-sided end face of the curved pipe in the reference camera coordinate system, which has a unique projection relationship with point f1 on the outer contour of the single-sided end face of the duct in the image plane. Let (m,n,p) be the direction vector of the spatial centerline, and (x L1 ,y L1 ,z L1 ) represents the endpoint to be determined.

Images