A method for measuring drill pipe joint height based on lidar and visible light fusion

Abstract

This invention proposes a method for measuring drill pipe joint height based on the fusion of lidar and visible light. The method involves simultaneously acquiring image data containing drill pipe information and 3D point cloud data; identifying drill pipe clamps and determining their positions; differentiating the current working conditions of the drill pipe; under drilling conditions, using 3D point cloud data to identify the joint and calculate its height relative to the drill platform plane; under retrieval conditions, using image data to identify the joint and calculate its height relative to the drill platform plane. This invention can accurately identify the positions of clamps and joints, facilitating the rapid guidance of drillers to the joint height for attaching and detaching operations. It has the advantages of improving the efficiency of drill retrieval and drilling operations, reducing human intervention, and improving operational safety.

Description

Technical Field

[0001] This invention relates to the field of intelligent petroleum equipment, and in particular to a method for measuring drill pipe joint height based on the fusion of lidar and visible light. Background Technology

[0002] As competition in the global oil drilling market intensifies, drilling contractors are increasingly demanding higher levels of automation in their drilling equipment to shorten drilling cycles, increase efficiency, and save costs. Consequently, more and more conventional drilling equipment is moving towards automation, using intelligent control and mechanization to replace manual, physically demanding tasks. This aims to reduce worker workload and decrease the incidence of accidents caused by harsh environments, fatigue, and negligence.

[0003] The drill stringer is an automated drilling tool mounted on the drill rig for connecting and disconnecting drill pipes. It can accommodate various types and sizes of drill pipes. The drill stringer can safely, quickly, and smoothly connect, disconnect, and tighten drill pipes. It enables the connection and disconnection of various drill pipe models, significantly reducing the labor intensity of drillers and improving drilling efficiency through mechanized operations.

[0004] Currently, when working with the drill string, most operators rely on visual guidance from the main driller to adjust the height of the tongs and back tongs assemblies, using hand gestures and intercoms to instruct the operator in the drill cab to ensure the tongs and back tongs are aligned with the drill pipe joint. Alternatively, a video surveillance camera is installed in front of the drill string, allowing the operator to monitor the footage. These procedures result in drillers working under high intensity for extended periods, with low efficiency and reliance on subjective judgment. Therefore, further improvements using intelligent technologies are needed.

[0005] Although Lanzhou University of Technology proposed the "Design of a Vision-Based Control System for Iron Drill Piles" in 2009, which initially realized the identification and positioning of drill rods by using visual feedback and ultrasonic sensors to integrate two-dimensional graphic information and distance information, it did not involve the measurement of joint height. Summary of the Invention

[0006] The purpose of this invention is to provide a method for measuring the height of drill pipe joints based on the fusion of lidar and visible light, thereby solving one or more of the problems in the prior art.

[0007] This invention proposes a method for measuring drill pipe joint height based on the fusion of lidar and visible light, comprising:

[0008] Simultaneously acquire image data and 3D point cloud data, both of which contain drill pipe information;

[0009] Identify the drill pipe clamps and determine their positions;

[0010] Distinguish the current working condition of the drill pipe, which includes two conditions: drill bit retrieval and drill bit running.

[0011] During drilling operations, the joint is identified using 3D point cloud data, and the height of the joint relative to the drilling platform plane is calculated.

[0012] During drilling operations, image data is used to identify the joint and calculate the height of the joint relative to the drilling platform plane.

[0013] In some implementations...

[0014] Image data and 3D point cloud data are acquired through a visible light and lidar fusion device, which includes a visible light module and a lidar. The visible light module is used to acquire image data, and the lidar is used to acquire 3D point cloud data.

[0015] Before the visible light and lidar fusion device is assembled but not installed, the visible light module and lidar are calibrated. The calibration includes the internal parameters of the visible light module and the external parameters of the visible light module and lidar.

[0016] After the visible light and lidar fusion device is installed, the whole system needs to be calibrated.

[0017] In some implementations...

[0018] The intrinsic parameters of the visible light module were calibrated using the Zhang Zhengyou calibration method with a checkerboard pattern to obtain the focal length f along the x and y axes. x and f y The pixel coordinates c of the image center along the x and y axes x and c y Radial distortion coefficients k1, k2, k3, tangential distortion coefficients p1, p2, let the three-dimensional coordinates of a point in the visible light module coordinate system be (X... c Y c Z c The formula for calculating the pixel coordinates u and v of this point in the visible light module coordinate system is as follows:

[0019] x = X c / Z c

[0020] y = Y c / Z c

[0021]

[0022] x corrected =x(1+k1r) 2+k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0023] y corrected =y(1+k1r) 2 +k2r 4 +k3r 6 )+2p2xy+p1(r 2 +2y 2 )

[0024] u = f x x corrected +c x

[0025] v = f y y corrected +c y ;

[0026] External parameters of visible light modules and radar During calibration, obtain the coordinates of the rectangular corner points of the calibration plate in the visible light module coordinate system and the radar coordinate system. Calculate the relative rotation radius R and relative translation T between the radar coordinate system and the visible light module coordinate system. After calibration, a point (X) in the radar coordinate system... l Y l Z l The formula for transforming to the visible light module coordinate system is:

[0027]

[0028] In some implementations, the process of calibrating the entire visible light and lidar fusion device after installation involves calculating the normal vector (a) of the drilling platform plane in the current radar coordinate system. x a y a z ) and a point (X) on the plane p Y p Z p By using spatial filtering, a point cloud of a selected region is fitted to a plane, and the plane equation is a. x *x+a y *y+a z *z+β=0, the fitting loss function is The normal vector of the plane is obtained directly through plane fitting, and X is set manually. p and Y p The value of Z is obtained through the plane equation. p The value of is used to obtain the coordinates (X, H) of a point on the plane. p Y pZ p ).

[0029] In some implementations, the process of identifying drill pipe clamps and determining their position includes the following steps:

[0030] S3.1 Select the radar spatial coordinate range containing the drill pipe area and extract the three-dimensional point cloud data of the drill pipe;

[0031] S3.2. Divide the data into intervals based on the step size λ and interval range δ over the height range, and extract the 3D point cloud data for each interval, specifically Z. min To Z max Choose the option that satisfies |Z min +i*λ|<δ,0≤i*λ <Z max -Z min The 3D point cloud data is the point cloud of the i-th interval;

[0032] S3.3. Perform circle fitting on the plane on the point cloud of the i-th interval to obtain the following fitting equation: The center of the i-th interval point cloud is obtained, and its three-dimensional coordinates are (x, y, y). ci y ci Z min +i*λ);

[0033] S3.4. Fit the circles in all intervals to obtain the center line and estimate the center line of the drill pipe cylinder.

[0034] S3.5 Calculate the distance from all points in the 3D point cloud data to the center line of the drill pipe. Filter out points outside the pre-defined range of values, and the remaining points are the points inside the drill pipe.

[0035] S3.6. Divide the drill pipe interior points into intervals according to the method in S3.2. Calculate the average distance from the interior points of each interval to the drill pipe centerline. The average distance d is the distance d from the drill pipe centerline to that interval. i ;

[0036] S3.7, for all intervals d i Perform statistics, set a threshold, and use the threshold to determine d. i The drill rod protrusions are divided into two categories, resulting in the drill rod protrusions on the three-dimensional point cloud data.

[0037] S3.8 Project the point cloud of the drill pipe protrusion based on the camera and radar calibration results to obtain the two-dimensional pixel coordinates in the visible light module coordinate system. Calculate the bounding box of the three-dimensional point cloud in the visible light module coordinate system. The upper and lower edges of the bounding box are the positions of the drill pipe coupling.

[0038] In some implementations, the method for distinguishing the current operating condition of the drill pipe includes the following steps:

[0039] S4.1 Using 3D point cloud data, obtain the parameter representation of the drill pipe centerline by point cloud height segmentation and circle fitting;

[0040] S4.2 Obtain the three-dimensional point cloud data and image data of the drill pipe protrusion by statistically analyzing the distance from the three-dimensional point cloud data to the drill pipe centerline;

[0041] S4.3. Based on the identification results of the drill pipe protrusion in S4.2, the distance from the point cloud to the drill pipe centerline is calculated for different intervals of the protrusion.

[0042] If the distance decreases significantly, the drilling condition is determined to be non-drilling; otherwise, the drilling condition is determined to be drilling.

[0043] In some implementations, during drilling operations, the process of identifying joints using 3D point cloud data and calculating the height of the joint relative to the drill platform plane is as follows:

[0044] By using the distances from the point cloud data of different regions of the protruding part in S4.3 to the drill pipe centerline, point cloud data within the range where the distance significantly decreases are obtained to form the point cloud region of the concave part. The point cloud data of the concave part are statistically analyzed, and their mean value is calculated, which is the location of the joint in the current radar coordinate system, denoted as (X). f Y f Z f The projection of the joint position onto the drilling platform plane, calculated using the calibration results of the previously used visible light and lidar fusion device, is the height of the joint relative to the drilling platform plane. The height H of the joint relative to the drilling platform plane is... f The calculation formula is:

[0045] H f =a x *(X f -X p )+a y *(Y f -Y p )+a z *(Z f -Z p ).

[0046] In some implementations, during drilling operations, image data is used to identify the joint. The image region of the drill pipe protrusion is obtained from the image data based on the identification result of the drill pipe clamp. The image of the clamp portion is removed, and the image of the remaining area is cropped to obtain the image region of the joint. Then, the 3D point cloud data is projected onto the camera coordinate system, and the 3D point cloud data corresponding to the image region of the joint is selected to obtain the 3D coordinates of the joint as (X... f Y f Z fThe projection of the joint position onto the drilling platform plane, calculated using the calibration results of the previously used visible light and lidar fusion device, is the height of the joint relative to the drilling platform plane. The height H of the joint relative to the drilling platform plane is... f The calculation formula is:

[0047] H f =a x *(X f -X p )+a y *(Y f -Y p )+a z *(Z f -Z p ).

[0048] A driller control system applying the aforementioned drill pipe joint height measurement method based on lidar and visible light fusion includes a visible light and lidar fusion device, a signal processing computer, a centralized control PLC, and a network switch. The visible light and lidar fusion device, the signal processing computer, and the centralized control PLC are all connected to the network switch.

[0049] The visible light and lidar fusion device is installed within 3-7 meters in front of the drill rod, facing the drill rod, to acquire image data and 3D point cloud data containing the measured drill rod information;

[0050] The signal processing computer includes an image processing module, a 3D point cloud processing module, and an algorithm module.

[0051] The image processing module is used to decode the video acquired by the visible light module through the network and switch to obtain real-time image data;

[0052] The 3D point cloud processing module is used to acquire 3D point cloud data collected by LiDAR through the network and switches;

[0053] The algorithm module is used to perform calculations and fusion processing algorithms, identify joints and calculate drill rod coupling height, and send the calculation results to the centralized control PLC via a switch;

[0054] The centralized control PLC is used to control the work of the iron drill operator.

[0055] In some implementations, the operation of the iron drill control system includes the following steps:

[0056] After the PLC-driven driller moves the drill rod to the waiting recognition position, it sends a joint recognition instruction to the signal processing computer.

[0057] After receiving the seam recognition instruction from the centralized control PLC, the signal processing computer acquires the image data and 3D point cloud data at the current moment.

[0058] Identify drill pipe clamps;

[0059] Differentiate working conditions and calculate the height of the joint relative to the drilling platform plane;

[0060] The calculated height of the joint relative to the drill rig plane is sent to the centralized control PLC;

[0061] The centralized control PLC drives the iron drill to the corresponding height based on the height of the joint relative to the drill platform plane.

[0062] The advantages of the drill pipe joint height measurement method based on lidar and visible light fusion described in this invention are as follows:

[0063] By fusing image data acquired by the visible light module with 3D point cloud data acquired by lidar, the location of couplings and joints can be accurately identified. This helps guide drillers to quickly reach the height of the joint for coupling and uncoupling operations. At the same time, real-time identification and automatic measurement effectively improve the efficiency of drilling operations during drill bit retrieval and drilling rig deployment, reduce the need for manual guidance, and improve the reliability and safety of the operation. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the structure of the iron driller control system in some embodiments of the present invention;

[0065] Figure 2 A flowchart illustrating the method and application of drill pipe joint height measurement based on lidar and visible light fusion in a driller control system according to some embodiments of the present invention.

[0066] Figure 3 This is a schematic diagram of three-dimensional point cloud data acquired by lidar in some embodiments of the present invention;

[0067] Figure 4 This is a schematic diagram of the connector protrusion identified by the fusion of lidar and visible light in some embodiments of the present invention;

[0068] Figure 5 This is a schematic diagram of a seam recognition frame in some embodiments of the present invention. Detailed Implementation

[0069] This embodiment is based on the principle of fusion measurement of lidar and visible light. It uses a visible light camera to acquire images of the wellhead operation scene on the drilling platform and uses lidar to acquire three-dimensional point cloud data of the scene in front. By fusing the image and point cloud information and combining the drill pipe joint height processing algorithm, the joint height of the drill pipe is automatically identified, guiding the driller to the joint quickly.

[0070] When the driller operates the drill pipe into the waiting recognition position (the waiting recognition position can be set in advance in the application of the central control PLC), the joint of the drill pipe is raised to the wellhead when the drill is connected. When the drill is lowered, the joint of the drill pipe is aligned vertically, ready for the driller to turn the thread.

[0071] This embodiment proposes applying a drill pipe joint height measurement method based on the fusion of lidar and visible light to the iron driller control system.

[0072] Combination Figure 1 The control system shown includes a visible light and lidar fusion device, a signal processing computer, a centralized control PLC, and a network switch.

[0073] Visible light and lidar fusion devices include visible light modules and lidar, for example:

[0074] The DJI Livox Mid-70 laser rangefinder sensor was selected as the lidar. It outputs a 905-nanometer laser beam, which is invisible to humans and avoids affecting the operator's line of sight. The lidar can detect blind spots as close as 0.05 meters and has a working distance of up to 260 meters. It has a 70.4-degree circular field of view and a point cloud density of 100,000 points / second, which can well meet the requirements of high-resolution measurement and observation at close range.

[0075] The IMX385 low-light sensor is selected as the visible light module to meet the shooting needs during nighttime operations. The imaging resolution is 1920*1080, and a 4.1mm focal length lens is selected to meet the shooting needs of small to medium field of view.

[0076] After the visible light and lidar fusion device was assembled, the intrinsic parameters of the camera (i.e., the intrinsic parameters of the visible light module) and the extrinsic parameters of the camera and lidar (the extrinsic parameters of the visible light module and the lidar) were calibrated in a laboratory environment.

[0077] The camera's intrinsic parameters were calibrated using the Zhang Zhengyou calibration method with a checkerboard pattern to obtain the focal length f along the x and y axes. x and f y The pixel coordinates c of the image center along the x and y axes x and c y Radial distortion coefficients k1, k2, k3; tangential distortion coefficients p1, p2; let the three-dimensional coordinates of a point in the camera coordinate system be (X... c Y c Z c The formula for calculating the pixel coordinates u and v of this point in the camera coordinate system is as follows:

[0078] x = X c / Zc

[0079] y = Y c / Z c

[0080]

[0081] x corrected =x(1+k1r) 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0082] y corrected =y(1+k1r) 2 +k2r 4 +k3r 6 )+2p2xy+p1(r 2 +2y 2 )

[0083] u = f x x corrected +c x

[0084] v = f y y corrected +c y ;

[0085] External parameters of cameras and radar During calibration, Aruco's rectangular calibration plates at different locations were used. The coordinates of the rectangular corner points of the calibration plate in the camera coordinate system were obtained through homography matrix. The coordinates of the rectangular corner points of the calibration plate in the radar coordinate system were obtained through plane estimation, edge extraction, and other operations. The relative rotation radius R and relative translation T between the radar and camera coordinate systems were calculated using 3D-ICP. After calibration, a point (X) in the radar coordinate system... l Y l Z l The formula for transforming to the camera coordinate system is:

[0086]

[0087] The algorithms used in this calibration process can all be implemented using existing technologies, so they will not be elaborated here.

[0088] After calibration, on-site installation is carried out. The visible light and lidar fusion device is installed within a range of 3-7 meters in front of the drill pipe, directly facing the drill pipe. After installation, the drill pipe being measured is included in the field of view of both the visible light module in the visible light and lidar fusion device. The visible light and lidar fusion device is then calibrated, and the normal vector (a) of the drill platform plane in the current radar coordinate system is calculated. x a y a z ) and a point (X) on the plane p Y p Z p By using spatial filtering, a point cloud of a selected region is fitted to a plane, and the plane equation is a. x *x+a y *y+a z *z+β=0, the fitting loss function is The normal vector of the plane is obtained directly through plane fitting, and then X is manually set. p and Y p The value of Z is obtained through the plane equation. p The value of is used to obtain the coordinates (X, Y) of a point on the plane. p Y p Z p ).

[0089] The visible light and lidar fusion device is connected to a switch located in the driller's cabin via two gigabit Ethernet cables. The signal processing computer and the centralized control PLC are also connected to the switch for communication. The centralized control PLC and the signal processing computer maintain connection reliability by receiving heartbeat packets from the signal processing computer. The visible light and lidar fusion device transmits 1080P video signals and lidar point cloud signals to the signal processing computer in real time. The signal processing computer uses a high-performance computing server to compute the fusion processing algorithm. The signal processing computer includes an image processing module, a 3D point cloud processing module, and an algorithm module.

[0090] The image processing module is used to decode the video acquired by the visible light module through the network and switch to obtain real-time image data;

[0091] The 3D point cloud processing module is used to acquire 3D point cloud data collected by LiDAR through the network and switches;

[0092] The algorithm module is used to perform calculations and fusion processing algorithms, identify joints and calculate drill rod coupling height, and send the calculation results to the centralized control PLC through the network and switch. The step size λ and interval range δ of the height range are preset in the algorithm module.

[0093] Combination Figure 2The content shown describes a method for measuring drill pipe joint height based on the fusion of lidar and visible light in a driller's control system, and the process of further applying the measured drill pipe joint height includes the following steps:

[0094] S1. After the PLC drives the drill bit to reach the pre-set waiting recognition position, it sends a joint recognition instruction to the signal processing computer.

[0095] S2. After receiving the seam recognition instruction from the centralized control PLC, the signal processing computer acquires the current image data and 3D point cloud data, for example: acquiring... Figure 3 The three-dimensional point cloud data shown;

[0096] S3. Perform drill pipe clamp identification. The specific identification process is as follows:

[0097] S3.1 Select the radar spatial coordinate range containing the drill pipe area and extract the three-dimensional point cloud data of the drill pipe (the point cloud data of other targets is allowed to contain point cloud noise).

[0098] S3.2. Divide the data into intervals based on the set step size λ and interval range δ within the height range, and extract the 3D point cloud data for each interval, specifically Z. min To Z max Choose the option that satisfies |Z min +i*λ|<δ,0≤i*λ <Z max -Z min The 3D point cloud data is the point cloud of the i-th interval;

[0099] S3.3. Perform circle fitting on the plane on the point cloud of the i-th interval to obtain the following fitting equation: The center of the i-th interval point cloud is obtained, and its three-dimensional coordinates are (x, y, y). ci y ci Z min +i*λ);

[0100] S3.4. Fit the circles in all intervals to obtain the center of the circle and estimate the line to obtain the centerline of the drill pipe cylinder;

[0101] S3.5 Calculate the distance from all points in the 3D point cloud data to the center line of the drill rod. Filter out points that are too far or too close by using a pre-set range of values. The remaining points are the points inside the drill rod.

[0102] S3.6. Divide the drill pipe interior points into intervals according to the method in S3.2. Calculate the average distance from the interior points of each interval to the drill pipe centerline. The average distance d is the distance d from the drill pipe centerline to that interval. i ;

[0103] S3.7, for all intervals di Perform histogram statistics, set a threshold, and use the threshold to analyze d. i Divided into two categories, and constrained on the continuity of protrusion recognition, the drill rod protrusion part on the three-dimensional point cloud data is obtained;

[0104] S3.8 Project the point cloud of the drill pipe protrusion based on the camera and radar calibration results to obtain the two-dimensional pixel coordinates in the camera coordinate system. Calculate the bounding box of the three-dimensional point cloud in the camera coordinate system based on the image segmentation algorithm. The upper and lower edges of the bounding box are the positions of the drill pipe coupling.

[0105] S4 and S3 simultaneously distinguish the working conditions, which include two types: drill bit retrieval and drill bit running. The method for distinguishing the working conditions includes the following steps:

[0106] S4.1 Using 3D point cloud data, obtain the parameter representation of the drill pipe centerline by point cloud height segmentation and circle fitting;

[0107] S4.2 Obtain the three-dimensional point cloud data and image data of the drill pipe protrusion by statistically analyzing the distance from the three-dimensional point cloud data to the drill pipe centerline;

[0108] S4.3 Based on the identification results of the drill pipe protrusions, for example: Figure 4 The schematic diagram shows the distance from the drill pipe centerline to the point cloud of different sections of the protrusion. Under drilling conditions, this distance decreases significantly; under retrieval conditions, it does not. Therefore, based on this and the changes in the statistical distance to the drill pipe centerline, the drill pipe condition is automatically determined. If it is drilling, proceed to S5; if it is retrieval, proceed to S6.

[0109] S5. During drilling operations, the 3D point cloud data acquired by the lidar is used to identify the joint. Using the distance from the point cloud data of different areas of the protruding part to the drill pipe centerline as described in S4.3, point cloud data within the range where the distance significantly decreases is obtained to form the point cloud region of the concave part. The point cloud data of the concave part is statistically analyzed, and its average value is calculated. This average value is the position of the joint in the current radar coordinate system, denoted as (X). f Y f Z f The projection of the joint position onto the drilling platform plane, calculated using the calibration results of the previously used visible light and lidar fusion device, is the height of the joint relative to the drilling platform plane. The height H of the joint relative to the drilling platform plane is... f The calculation formula is:

[0110] H f =a x *(X f -X p )+a y *(Y f -Yp )+a z *(Z f -Z p ).

[0111] S6. During drilling operations, the visible light module is used to identify the joint. At this time, the recessed area at the joint is small. If a lidar is used for joint identification, the resolution limitation of the lidar will result in poor identification. Therefore, image data acquired by the visible light module is used for joint identification. Based on the drill pipe coupling identification results obtained in S4, the image area of ​​the drill pipe protrusion in the image data acquired by the visible light module is obtained. The image of the coupling is removed, and the remaining area is cropped. The joint area is then labeled. A detection model is trained using deep learning. This detection model is used to detect the cropped image to obtain the image area at the joint, for example, forming an image like... Figure 5 The diagram shows the image area between the two light green stripes within the green box. The three-dimensional point cloud data is then projected onto the camera coordinate system using pre-calibrated parameters. By selecting the three-dimensional point cloud data corresponding to the image area of ​​the seam, the three-dimensional coordinates of the seam can be obtained. Combined with the calibration results of the visible light and lidar fusion device, the seam position and the height of the seam relative to the drilling platform plane are calculated. This calculation process is the same as the formula for calculating the height of the seam relative to the drilling platform plane in S6.

[0112] S7. The signal processing computer sends the height of the joint relative to the drill rig plane obtained from the above calculation to the centralized control PLC.

[0113] S8. The centralized control PLC drives the iron drill to the corresponding height according to the height of the joint relative to the drill platform plane. This step is not the focus of this invention and can be implemented using existing technology, so it will not be described in detail here.

[0114] The above description is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several similar modifications and improvements can be made without departing from the inventive concept of the present invention, and these should also be considered within the scope of protection of the present invention.

Claims

1. A method for measuring drill pipe joint height based on the fusion of lidar and visible light, characterized in that, include: Simultaneously acquire image data and 3D point cloud data. Both image data and 3D point cloud data contain drill rod information. The image data and 3D point cloud data are acquired through a visible light and lidar fusion device, which includes a visible light module and a lidar. The visible light module is used to acquire image data, and the lidar is used to acquire 3D point cloud data. Before the visible light and lidar fusion device is assembled but not installed, the visible light module and lidar are calibrated. The calibration includes the internal parameters of the visible light module and the external parameters of the visible light module and lidar. After the visible light and lidar fusion device is installed, the whole system needs to be calibrated. The intrinsic parameters of the visible light module were calibrated using the Zhang Zhengyou calibration method with a checkerboard pattern to obtain the focal lengths along the x and y axes. and Pixel coordinates of the image center along the x and y axes and Radial distortion coefficient , , Tangential distortion coefficient , Let the three-dimensional coordinates of a point in the visible light module coordinate system be ( , , The formula for calculating the pixel coordinates u and v of this point in the visible light module coordinate system is as follows: ； External parameters of visible light modules and radar , During calibration, obtain the coordinates of the rectangular corner points of the calibration plate in the visible light module coordinate system and the radar coordinate system. Calculate the relative rotation radius R and relative translation T between the radar coordinate system and the visible light module coordinate system. After calibration, a point in the radar coordinate system ( , , The formula for transforming to the visible light module coordinate system is: ； After the visible light and lidar fusion device is installed, the process of calibrating the whole system involves calculating the normal vector of the drilling platform plane in the current radar coordinate system. , , ) and a point on the plane ( , , By using spatial filtering, a point cloud of a selected region is fitted to a plane, and the plane equation is: The fitting loss function is The normal vector of the plane can be obtained directly through plane fitting, or manually set. and The value of is obtained through the plane equation. The value of is used to obtain the coordinates of a point on the plane. , , ); Identifying and determining the location of drill pipe clamps involves the following steps: S3.1 Select the radar spatial coordinate range containing the drill pipe area and extract the three-dimensional point cloud data of the drill pipe; S3.2, Based on the step size within the height range and interval range Divide the data into intervals and extract the 3D point cloud data for each interval, specifically... arrive Choose to satisfy , The 3D point cloud data is the point cloud of the i-th interval; S3.

3. Perform circle fitting on the plane on the point cloud of the i-th interval to obtain the following fitting equation: , obtained the The center of the point cloud of each interval has three-dimensional coordinates as ( , , ); S3.

4. Fit the circles in all intervals to obtain the center line and estimate the center line of the drill pipe cylinder. S3.5 Calculate the distance from all points in the 3D point cloud data to the center line of the drill pipe. Filter out points outside the pre-defined range of values, and the remaining points are the points inside the drill pipe. S3.

6. Divide the drill pipe interior points into intervals according to the method in S3.

2. Calculate the average distance from the interior points of each interval to the drill pipe centerline. The average distance obtained is the distance from the drill pipe centerline to that interval. ; S3.7, for all intervals Perform statistics, set thresholds, and use the thresholds to... The drill rod protrusions are divided into two categories, resulting in the drill rod protrusions on the three-dimensional point cloud data. S3.8 Project the point cloud of the drill pipe protrusion based on the camera and radar calibration results to obtain the two-dimensional pixel coordinates in the visible light module coordinate system. Calculate the bounding box of the three-dimensional point cloud in the visible light module coordinate system. The upper and lower edges of the bounding box are the positions of the drill pipe coupling. To determine the current operating condition of the drill pipe, which includes both drill bit retrieval and drill bit running conditions, the following steps are required: S4.1 Using 3D point cloud data, obtain the parameter representation of the drill pipe centerline by point cloud height segmentation and circle fitting; S4.2 Obtain the three-dimensional point cloud data and image data of the drill pipe protrusion by statistically analyzing the distance from the three-dimensional point cloud data to the drill pipe centerline; S4.

3. Based on the identification results of the drill pipe protrusion in S4.2, the distance from the point cloud to the drill pipe centerline is calculated for different intervals of the protrusion. If the distance decreases significantly, the drilling condition is determined to be non-drilling; otherwise, the drilling condition is determined to be drilling. During drilling operations, the joint is identified using 3D point cloud data, and the height of the joint relative to the drilling platform plane is calculated. During drilling operations, image data is used to identify the joint and calculate the height of the joint relative to the drilling platform plane.

2. The method for measuring drill pipe joint height based on lidar and visible light fusion according to claim 1, wherein, During drilling operations, the process of identifying joints using 3D point cloud data and calculating the height of the joint relative to the drill platform plane is as follows: By using the distance from the point cloud of different regions of the protruding part in S4.3 to the center line of the drill pipe, point cloud data in the range where the distance is significantly smaller is obtained to form the point cloud region of the concave part. The point cloud data of the concave part is statistically analyzed, and its mean value is calculated, which is the position of the joint in the current radar coordinate system, denoted as ( , , The projection of the joint position onto the drilling platform plane, calculated using the calibration results of the previously used visible light and lidar fusion device, is the height of the joint relative to the drilling platform plane. The calculation formula is: 。 3. The method for measuring drill pipe joint height based on lidar and visible light fusion according to claim 2, wherein, In drilling operations, image data is used to identify the joint. The image region of the drill pipe protrusion is obtained from the drill pipe clamp identification results. The image of the clamp is removed, and the remaining area is cropped to obtain the image region of the joint. Then, the 3D point cloud data is projected onto the camera coordinate system, and the 3D point cloud data corresponding to the image region of the joint is selected to obtain the 3D coordinates of the joint. , , The projection of the joint position onto the drilling platform plane, calculated using the calibration results of the previously used visible light and lidar fusion device, is the height of the joint relative to the drilling platform plane. The calculation formula is: 。 4. A drill pipe joint height measurement control system applying the method for measuring drill pipe joint height based on the fusion of lidar and visible light as described in any one of claims 1-3, characterized in that, This includes a visible light and lidar fusion device, a signal processing computer, a centralized control PLC, and a network switch. All of these components are connected to the network switch. The visible light and lidar fusion device is installed within 3-7 meters in front of the drill rod, facing the drill rod, to acquire image data and 3D point cloud data containing the measured drill rod information; The signal processing computer includes an image processing module, a 3D point cloud processing module, and an algorithm module. The image processing module is used to decode the video acquired by the visible light module through the network and switch to obtain real-time image data; The 3D point cloud processing module is used to acquire 3D point cloud data collected by LiDAR through the network and switches; The algorithm module is used to perform calculations and fusion processing algorithms, identify joints and calculate drill rod coupling height, and send the calculation results to the centralized control PLC via a switch; The centralized control PLC is used to control the work of the iron drill operator.

5. The iron drill control system according to claim 4, wherein, The working method of the iron drill control system includes the following steps: After the PLC-driven driller moves the drill rod to the waiting recognition position, it sends a joint recognition instruction to the signal processing computer. After receiving the seam recognition instruction from the centralized control PLC, the signal processing computer acquires the image data and 3D point cloud data at the current moment. Identify drill pipe clamps; Differentiate working conditions and calculate the height of the joint relative to the drilling platform plane; The calculated height of the joint relative to the drill rig plane is sent to the centralized control PLC; The centralized control PLC drives the iron drill to the corresponding height based on the height of the joint relative to the drill platform plane.

Images