Mining pipe grabbing machine pipeline grabbing method based on multi-sensor fusion

Abstract

The invention discloses a mining pipe grabber pipeline grabbing method based on multi-sensor fusion, a mining pipe grabbing robot is used for grabbing a pipeline, the mining pipe grabbing robot comprises a mechanical arm, a control unit and a driving unit for driving the mechanical arm to act, and a depth camera is arranged at the tail end of the mechanical arm; the pipeline grabbing method comprises the following steps of S1, model configuration, S2, hand-eye calibration, S3, coarse positioning, S4, fine positioning, S5, path planning, S6, motion control and S7, grabbing operation. By fusing data of various sensors such as the depth camera, the laser radar, the encoder and the pressure sensor, accurate positioning and grabbing of the pipeline are achieved, and the working efficiency and safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automation of mining machinery and equipment, and in particular to a pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion. Background Art

[0002] In mining operations, the grabbing and handling of pipelines is an important and complex task. Traditional methods often rely on manual operations, which are not only inefficient but also pose safety risks. With the development of sensor technology and automatic control technology, it has become possible to realize the automatic grabbing of pipelines. However, how to accurately and efficiently locate and grab pipelines is still an urgent problem to be solved. Therefore, it is particularly important to develop a mining pipe grabber pipeline grabbing method that can integrate multiple sensor data and accurately determine the position, posture and surrounding environment information of the pipeline. Summary of the invention

[0003] In view of the above problems, the purpose of the present invention is to propose a pipeline grabbing method for a mining pipe grabber based on multi-sensor fusion, so as to solve the problem that the existing pipeline grabbing and transportation in underground coal mines mainly relies on manual operation, which is very time-consuming and poses personal safety issues.

[0004] In order to solve the above problem, the present invention is achieved as follows:

[0005] A pipe grabbing method for a mining pipe grabber based on multi-sensor fusion, using a mining pipe grabbing robot to grab the pipe. The mining pipe grabbing robot includes a mechanical arm, a control unit and a driving unit for driving the mechanical arm. A depth camera is installed at the end of the mechanical arm. The pipe grabbing method includes the following steps:

[0006] S1. Model configuration: Export the URDF model of the mining pipe grabbing robot, generate a configuration file through the MoveIt configuration assistant, and improve the configuration file to control the mechanical arm of the mining pipe grabbing robot;

[0007] S2. Hand-eye calibration: solve the conversion relationship between the depth camera coordinate system and the robotic arm coordinate system, and TF publishes the coordinate transformation;

[0008] S3. Coarse positioning: Obtain point cloud data in the tunnel, segment the point cloud data using the RANSAC algorithm, extract the pipeline point cloud after filtering out noise points through voxel downsampling and statistical filtering, and calculate the coordinates of the pipeline center point to achieve coarse positioning of the pipeline. The control unit controls the mining pipe grabbing robot to move to the center of the pipeline;

[0009] S4. Precise positioning: The driving unit moves the robotic arm to the initial position, uses ROS to drive the depth camera to obtain the RGB image and depth map of the pipeline, and accurately identifies the position of the pipeline grabbing point based on the YOLOv8 target segmentation and positioning method, and transforms it from the camera coordinate system to the robotic arm coordinate system according to the coordinate transformation relationship in step S2;

[0010] S5. Path planning: According to the state of the robot arm in step S4 and the position of the pipeline grabbing point, use MoveIt to generate an efficient motion trajectory, fit a smooth trajectory curve through interpolation, and publish the trajectory path to form ROS topic information;

[0011] S6. Motion control: The drive unit receives the ROS topic information from MoveIt, which includes the position and speed of each axis. The drive unit also receives the encoder feedback information on the robot arm and converts it into the joint_state topic, which is sent back to MoveIt to achieve closed-loop control.

[0012] S7. Grasping operation: The drive unit controls the robot arm to move to the pipeline position according to the trajectory path planned in step S5. When it reaches the last waypoint, the gripper is closed and the pressure value is detected by the pressure sensor. The grasping is completed when the threshold is reached.

[0013] Furthermore, in step S2, hand-eye calibration is performed, the depth camera is installed at the end of the robotic arm and its coordinate system moves with the robotic arm, the calibration plate is fixed, and the coordinate relationship between the depth camera and the robotic arm is determined by coordinate transformation.

[0014] Furthermore, the calculation method in step S2 is:

[0015]

[0016] In the formula, Represents the pose transformation matrix between the first calibrated robot end coordinate system and the robot base coordinate system, Represents the pose transformation matrix between the second calibrated robot end coordinate system and the robot base coordinate system, Represents the pose transformation matrix between the first calibrated camera coordinate system and the first calibrated robot end coordinate system, Represents the pose transformation matrix between the camera coordinate system of the second calibration and the robot arm end coordinate system of the second calibration, Represents the pose transformation matrix between the calibration plate coordinate system and the first calibrated camera coordinate system, Represents the pose transformation matrix between the calibration plate coordinate system and the second calibration camera coordinate system;

[0017] The deduction of the public statement can be obtained:

[0018]

[0019] Since there is no change between the camera coordinate system and the robot end coordinate system, we can get:

[0020]

[0021] Arranging the formula yields:

[0022] AX=XB

[0023] Among them, A represents the product result of the pose transformation of the robot base and its robot end for two movements. This matrix is ​​the product result of the pose transformation obtained when the robot end observes the robot base at two different positions. It not only covers the spatial position information of the robot end relative to the robot base, but also accurately describes its directional characteristics; X represents the pose transformation matrix between the camera coordinate system and the robot end coordinate system. This matrix is ​​the key bridge connecting the robot end coordinate system and the camera coordinate system, and describes in detail the exact position and orientation of the depth camera in the robot end coordinate system; B represents the product result of the pose transformation of the depth camera and the calibration plate for two movements. This matrix is ​​the product result of the pose transformation obtained when the camera observes the calibration plate at two different positions.

[0024] Furthermore, the voxel downsampling method in step S3 aims to effectively reduce the amount of point cloud data while ensuring that the overall structural features are preserved. First, the point cloud data obtained by the lidar is spatially divided to construct a fine grid structure, in which each grid unit is called a voxel. These voxels are closely arranged to form a cube array in a three-dimensional space. The feature points of the voxel are represented by calculating the centroid position of all points in each small cube. The calculation formula is:

[0025]

[0026] Among them, p c is the centroid coordinates of all points in the cube, p i is the coordinate value of the midpoint of the cube, n t is the number of points in the cube. First, the centroid position of each small cube is calculated according to the distribution of points in each small cube. Then, all small cubes are traversed and a new sampling point is placed at the centroid of each small cube to construct the downsampled point cloud. For small cubes without points, they are kept empty to ensure that the downsampled point cloud can accurately reflect the distribution characteristics of the original point cloud.

[0027] Furthermore, the principle of statistical filtering in step S3 is based on the assumption of Gaussian distribution, where for the spatial point cloud set Point Cloud = {a1, a2, a3 ..., an}, point a i The k-neighborhood of (i=1,2,3…,n) is the distance a in space. i The nearest k points a m (m=1,2,3...,k), define a i The coordinates are (x i ,y i ,z i ), a m The coordinates are (x m ,y m ,z m ), then a m to a i The distance is:

[0028]

[0029] These k points to a i The average distance D is:

[0030]

[0031] Assume a i To this k neighboring point a m The average distance D conforms to the Gaussian distribution, then the expected value μ and standard deviation σ are:

[0032]

[0033] Assume that the distance threshold θ=μ+σ·s, where s is the specified multiple of the standard deviation, and perform denoising on the distance threshold θ. When θ<D, remove the point, and when θ>D, retain the point.

[0034] Furthermore, in the RANSAC plane segmentation method in step S3, a sufficient number of points are randomly selected as candidate inliers in each iteration, and then these points are used to fit a plane model. The plane equation can usually be expressed as:

[0035] Ax+By+Cz+D=0

[0036] Where A, B, C and D are the coefficients of the plane equation. According to the fitted plane model, the distances from all points to the plane are calculated, and the set threshold is used to determine which points are in-plane points. If the distance from a point to the plane is less than the threshold, the point is considered to belong to the plane. The above process is repeated multiple times, and each iteration will obtain a new plane model and a set of in-plane points. Finally, the plane model with the most in-plane points is retained as the optimal result.

[0037] Furthermore, in step S4, precise positioning is performed, and a binary mask image is obtained through YOLOv8 target segmentation and positioning. The coordinates of the target pixel points on the RGB image can be determined, and the depth values ​​corresponding to the pixel coordinates are obtained from the aligned depth map. Combined with the camera central transmission model, the 3D coordinates (X, Y, Z) corresponding to the target pixel points in the RGB image are calculated. The calculation formula is:

[0038]

[0039] In the formula, c x 、c y 、f x 、f y is the internal parameter of the RGB camera, u is the horizontal coordinate of the pixel, v is the vertical coordinate of the pixel, and Z is the depth value corresponding to the pixel (u, v).

[0040] Furthermore, in the grasping operation of step S7, the pressure sensor is used to detect the pressure value of the clamping claw on the robot arm on the pipeline, which is achieved by measuring the output signal of the pressure sensor. The output signal is in a certain proportional relationship with the pressure value, and the calculation formula is:

[0041] P=(V-V0) / S

[0042] Where P is the pressure value, V is the output signal of the pressure sensor, V0 is the zero point output of the pressure sensor, and S is the sensitivity of the sensor.

[0043] Furthermore, in step S1, SolidWorks is used to export the URDF model of the mining pipe grabbing robot.

[0044] The beneficial effects of the present invention are:

[0045] The present invention achieves precise positioning and grasping of pipelines by integrating data from multiple sensors such as depth cameras, lidars, encoders and pressure sensors, thereby improving operational efficiency and safety.

[0046] The MoveIt software platform is used for path planning and motion control to optimize the motion trajectory and ensure the smoothness and speed of the robot arm's movement.

[0047] Closed-loop control is achieved, the accuracy and stability of the robot arm's movement are improved, and the operating difficulty and safety hazards are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the present invention;

[0049] Figure 2 It is a construction framework diagram of the present invention;

[0050] Figure 3 It is a schematic diagram of an application scenario of the present invention.

[0051] Description of reference numerals:

[0052] 1. Mining pipe grabbing robot; 2. Depth camera; 3. Pipeline; 4. LiDAR. DETAILED DESCRIPTION

[0053] The technical scheme in the embodiment of the present invention is described clearly and completely below in conjunction with the accompanying drawings in the embodiment of the present invention. In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0054] like Figure 1-Figure 3 As shown, the present invention provides a pipe grabbing method for a mining pipe grabber based on multi-sensor fusion, using a mining pipe grabbing robot 1 to grab a pipe 3, the mining pipe grabbing robot 1 includes a mechanical arm, a control unit and a driving unit for driving the mechanical arm to move, and a depth camera 2 is installed at the end of the mechanical arm. In this embodiment, the control unit is used to control the movement of the mining pipe grabbing robot 1, and the driving unit is used to control the movement of each joint of the mechanical arm.

[0055] like Figure 1 As shown, the pipeline grabbing method includes the following steps:

[0056] S1. Model configuration: export the URDF model of the mining pipe grabbing robot 1, and generate a configuration file through the MoveIt configuration assistant, and improve the configuration file to control the mechanical arm of the mining pipe grabbing robot 1. In this embodiment, SolidWorks is used to export the URDF model of the mining pipe grabbing robot 1, and other three-dimensional drawing software can also be used, which is not limited here.

[0057] S2. Hand-eye calibration: solve the conversion relationship between the depth camera 2 coordinate system and the robotic arm coordinate system, and TF publishes the coordinate transformation. Specifically, in step S2, hand-eye calibration is performed, the depth camera 2 is installed at the end of the robotic arm and its coordinate system moves with the robotic arm, the calibration plate is fixed, and the coordinate relationship between the depth camera 2 and the robotic arm is determined by coordinate transformation.

[0058] The specific calculation method is:

[0059]

[0060] In the formula, Represents the pose transformation matrix between the first calibrated robot end coordinate system and the robot base coordinate system, Represents the pose transformation matrix between the second calibrated robot end coordinate system and the robot base coordinate system, Represents the pose transformation matrix between the first calibrated camera coordinate system and the first calibrated robot end coordinate system, Represents the pose transformation matrix between the camera coordinate system of the second calibration and the robot arm end coordinate system of the second calibration, Represents the pose transformation matrix between the calibration plate coordinate system and the first calibrated camera coordinate system, Represents the pose transformation matrix between the calibration plate coordinate system and the second calibration camera coordinate system.

[0061] The deduction of the public statement can be obtained:

[0062]

[0063] Since there is no change between the camera coordinate system and the robot end coordinate system, we can get:

[0064]

[0065] Arranging the formula yields:

[0066] AX=XB

[0067] Among them, A represents the product result of the pose transformation of the robot base and its robot end for two movements. This matrix is ​​the product result of the pose transformation obtained when the robot end observes the robot base at two different positions. It not only covers the spatial position information of the robot end relative to the robot base, but also accurately describes its directional characteristics; X represents the pose transformation matrix between the camera coordinate system and the robot end coordinate system. This matrix is ​​the key bridge connecting the robot end coordinate system and the camera coordinate system, and describes in detail the exact position and orientation of the depth camera 2 in the robot end coordinate system; B represents the product result of the pose transformation of the depth camera 2 and the calibration plate for two movements. This matrix is ​​the product result of the pose transformation obtained when the camera observes the calibration plate at two different positions.

[0068] S3. Coarse positioning: obtain the point cloud data in the tunnel, use the RANSAC algorithm to segment the point cloud data, first reduce the data volume by voxel downsampling, then filter out the noise by statistical filtering, finally extract the pipeline point cloud, and calculate the coordinates of the center point of pipeline 3 to achieve coarse positioning of pipeline 3. The control unit controls the mining pipe grabbing robot 1 to move to the center position of pipeline 3. Figure 3 As shown, the front end of the mining pipe grabbing robot 1 is provided with a laser radar 4 to collect data in the tunnel. The laser radar 4 can be Figure 2 The model of the mid-chassis sensor may also be other models, and there is no limitation here.

[0069] The following is a detailed introduction to RANSAC's plane segmentation method, voxel downsampling method, and statistical filtering to remove noise:

[0070] (1) RANSAC plane segmentation method, in each iteration, randomly selects a sufficient number of points as candidate inliers, and then uses these points to fit a plane model. The plane equation can usually be expressed as:

[0071] Ax+By+Cz+D=0

[0072] Where A, B, C and D are the coefficients of the plane equation. According to the fitted plane model, the distances from all points to the plane are calculated, and the set threshold is used to determine which points are in-plane points. If the distance from a point to the plane is less than the threshold, the point is considered to belong to the plane. The above process is repeated multiple times, and each iteration will obtain a new plane model and a set of in-plane points. Finally, the plane model with the most in-plane points is retained as the optimal result.

[0073] (2) Voxel downsampling method aims to effectively reduce the amount of point cloud data while ensuring that the overall structural features are preserved. First, the point cloud data collected by the LiDAR 4 is spatially divided to construct a fine grid structure, where each grid unit is called a voxel. These voxels are closely arranged to form a cube array in three-dimensional space. The feature points of the voxel are represented by calculating the centroid position of all points in each small cube. The calculation formula is:

[0074]

[0075] Among them, p c is the centroid coordinates of all points in the cube, p i is the coordinate value of the midpoint of the cube, n t is the number of points in the cube. First, the centroid position of each small cube is calculated according to the distribution of points in each small cube. Then, all small cubes are traversed and a new sampling point is placed at the centroid of each small cube to construct the downsampled point cloud. For small cubes without points, they are kept empty to ensure that the downsampled point cloud can accurately reflect the distribution characteristics of the original point cloud.

[0076] (3) The principle of statistical filtering is based on the assumption of Gaussian distribution, where for a spatial point cloud set Point Cloud = {a1, a2, a3..., a n}, point a i The k-neighborhood of (i=1,2,3…,n) is the distance a in space. i The nearest k points a m (m=1,2,3...,k), define a i The coordinates are (x i,y i ,z i ), a m The coordinates are (x m ,y m ,z m ), then a m to a i The distance is:

[0077]

[0078] These k points to a i The average distance D is:

[0079]

[0080] Assume a i To this k neighboring point a m The average distance D conforms to the Gaussian distribution, then the expected value μ and standard deviation σ are:

[0081]

[0082]

[0083] Assume that the distance threshold θ=μ+σ·s, where s is the specified multiple of the standard deviation, and perform denoising on the distance threshold θ. When θ<D, remove the point, and when θ>D, retain the point.

[0084] S4. Precise positioning: The driving unit moves the robot to the initial position, uses ROS to drive the depth camera 2 to obtain the RGB image and depth map of the pipeline 3, and accurately identifies the position of the pipeline grabbing point based on the YOLOv8 target segmentation and positioning method, and converts it from the camera coordinate system to the robot base coordinate system according to the coordinate transformation relationship in step S2. In step S4, precise positioning is performed, and a binary mask image is obtained through YOLOv8 target segmentation and positioning. The coordinates of the target pixel points on the RGB image can be determined, and the depth value corresponding to the pixel point coordinates is obtained from the aligned depth map. Combined with the camera central transmission model, the 3D coordinates (X, Y, Z) corresponding to the target pixel point in the RGB image are calculated. The calculation formula is:

[0085]

[0086] In the formula, c x 、c y 、f x 、f y is the internal parameter of the RGB camera, u is the horizontal coordinate of the pixel, v is the vertical coordinate of the pixel, and Z is the depth value corresponding to the pixel (u, v).

[0087] S5. Path planning: According to the current state of the robot arm and the position of the pipe grabbing point in step S4, use MoveIt to generate an efficient motion trajectory, fit a smooth trajectory curve through interpolation, and publish the trajectory path to form ROS topic information.

[0088] S6. Motion control: The drive unit receives the ROS topic information of MoveIt, which includes the position and speed of each axis. The drive unit also receives the encoder feedback information on the robot arm and converts it into the joint_state topic, which is sent back to MoveIt to achieve closed-loop control.

[0089] S7. Grasping operation: The drive unit controls the robot arm to move to the position of pipeline 3 according to the trajectory path planned in step S5. When it reaches the last waypoint, the gripper is closed and the pressure value is detected by the pressure sensor. The gripping is completed after reaching the threshold. The pressure sensor is used to detect the pressure value of the gripper on pipeline 3. This is achieved by measuring the output signal of the sensor. The signal is proportional to the pressure value. The calculation formula is:

[0090] P=(V-V0) / S

[0091] Where P is the pressure value, V is the output signal of the sensor, S is the sensitivity of the sensor, and V0 is the zero point output of the sensor.

[0092] like Figure 1-Figure 3 As shown, a pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion of the present invention first obtains the three-dimensional point cloud data inside the tunnel through the laser radar 4 carried by the mining pipe grabbing robot 1, extracts the point cloud data of the pipeline 3, and calculates the center point coordinates of the pipeline 3 to achieve rough positioning of the pipeline 3, moves the mining pipe grabbing robot 1 to the center position of the pipeline 3, and then moves the mechanical arm to the initial posture, uses ROS to drive the depth camera 2 to obtain the RGB image and depth map of the pipeline 3, and accurately identifies the posture of the pipeline grabbing point based on the YOLOv8 target segmentation and positioning technology, and converts it from the camera coordinate system to the mechanical arm base coordinate system. According to the current state of the mechanical arm and the posture of the pipeline grabbing point, MoveIt is used to generate an efficient motion trajectory, and a smooth trajectory curve is fitted by interpolation, and the trajectory path is released. The driving unit controls the mechanical arm to move to the position of the pipeline 3 according to the planned path. When it reaches the last waypoint, the gripper is closed and the pressure value is detected by the pressure sensor. The grabbing is completed after reaching the threshold.

[0093] Although the present invention discloses preferred specific embodiments to achieve the above-mentioned objectives, it is not intended to limit the structural features of the present invention. Any person skilled in the art should know that within the technical spirit of the present invention, any easily conceivable changes or modifications are possible and are all covered by the scope of the patent application of the present invention.

Claims

1. A pipeline grabbing method for a mining pipe grabber based on multi-sensor fusion, characterized in that: The pipe is grabbed by a mining pipe grabbing robot, which includes a mechanical arm, a control unit and a driving unit for driving the mechanical arm to move. A depth camera is installed at the end of the mechanical arm. The pipe grabbing method includes the following steps: S1. Model configuration: Export the URDF model of the mining pipe grabbing robot, generate a configuration file through the MoveIt configuration assistant, and improve the configuration file to control the mechanical arm of the mining pipe grabbing robot; S2. Hand-eye calibration: solve the conversion relationship between the depth camera coordinate system and the robotic arm coordinate system, and TF publishes the coordinate transformation; S3. Coarse positioning: Obtain point cloud data in the tunnel, segment the point cloud data using the RANSAC algorithm, extract the pipeline point cloud after filtering out noise points through voxel downsampling and statistical filtering, and calculate the coordinates of the pipeline center point to achieve coarse positioning of the pipeline. The control unit controls the mining pipe grabbing robot to move to the center of the pipeline; S4. Precise positioning: The driving unit moves the robotic arm to the initial position, uses ROS to drive the depth camera to obtain the RGB image and depth map of the pipeline, and accurately identifies the position of the pipeline grabbing point based on the YOLOv8 target segmentation and positioning method, and transforms it from the camera coordinate system to the robotic arm coordinate system according to the coordinate transformation relationship in step S2; S5. Path planning: According to the state of the robot arm in step S4 and the position of the pipeline grabbing point, use MoveIt to generate an efficient motion trajectory, fit a smooth trajectory curve through interpolation, and publish the trajectory path to form ROS topic information; S6. Motion control: The drive unit receives the ROS topic information from MoveIt, which includes the position and speed of each axis. The drive unit also receives the encoder feedback information on the robot arm and converts it into the joint_state topic, which is sent back to MoveIt to achieve closed-loop control. S7. Grasping operation: The drive unit controls the robot arm to move to the pipeline position according to the trajectory path planned in step S5. When it reaches the last waypoint, the gripper is closed and the pressure value is detected by the pressure sensor. The grasping is completed when the threshold is reached.

2. The pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion according to claim 1 is characterized in that: In step S2, hand-eye calibration is performed. The depth camera is installed at the end of the robotic arm and its coordinate system moves with the robotic arm. The calibration plate is fixed and the coordinate relationship between the depth camera and the robotic arm is determined by coordinate transformation.

3. The pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion according to claim 2 is characterized in that: The calculation method in step S2 is: In the formula, Represents the pose transformation matrix between the first calibrated robot end coordinate system and the robot base coordinate system, Represents the pose transformation matrix between the second calibrated robot end coordinate system and the robot base coordinate system, Represents the pose transformation matrix between the first calibrated camera coordinate system and the first calibrated robot end coordinate system, Represents the pose transformation matrix between the camera coordinate system of the second calibration and the robot arm end coordinate system of the second calibration, Represents the pose transformation matrix between the calibration plate coordinate system and the first calibrated camera coordinate system, Represents the pose transformation matrix between the calibration plate coordinate system and the second calibration camera coordinate system; The deduction of the public statement can be obtained: Since there is no change between the camera coordinate system and the robot end coordinate system, we can get: Arranging the formula yields: AX=XB Among them, A represents the product result of the pose transformation of the robot base and its robot end for two movements. This matrix is ​​the product result of the pose transformation obtained when the robot end observes the robot base at two different positions. It not only covers the spatial position information of the robot end relative to the robot base, but also accurately describes its directional characteristics; X represents the pose transformation matrix between the camera coordinate system and the robot end coordinate system. This matrix is ​​the key bridge connecting the robot end coordinate system and the camera coordinate system, and describes in detail the exact position and orientation of the depth camera in the robot end coordinate system; B represents the product result of the pose transformation of the depth camera and the calibration plate for two movements. This matrix is ​​the product result of the pose transformation obtained when the camera observes the calibration plate at two different positions.

4. The pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion according to claim 1 is characterized in that: The voxel downsampling method in step S3 aims to effectively reduce the amount of point cloud data while ensuring that the overall structural features are preserved. First, the point cloud data obtained by the lidar is spatially divided to construct a fine grid structure, in which each grid unit is called a voxel. These voxels are closely arranged to form a cube array in a three-dimensional space. The feature points of the voxel are represented by calculating the centroid position of all points in each small cube. The calculation formula is: Among them, p c is the centroid coordinates of all points in the cube, p i is the coordinate value of the midpoint of the cube, n t is the number of points in the cube. First, the centroid position of each small cube is calculated according to the distribution of points in each small cube. Then, all small cubes are traversed and a new sampling point is placed at the centroid of each small cube to construct the downsampled point cloud. For small cubes without points, they are kept empty to ensure that the downsampled point cloud can accurately reflect the distribution characteristics of the original point cloud.

5. The pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion according to claim 1 is characterized in that: The principle of statistical filtering in step S3 is based on the assumption of Gaussian distribution, where for the spatial point cloud set Point Cloud = {a1, a2, a3..., a n }, point a i The k-neighborhood of (i=1,2,3…,n) is the distance a in space. i The nearest k points a m (m=1,2,3...,k), define a i The coordinates are (x i ,y i ,z i ), a m The coordinates are (x m ,y m ,z m ), then a m to a i The distance is: These k points to a i The average distance D is: Assume a i To this k neighboring point a m The average distance D conforms to the Gaussian distribution, then the expected value μ and standard deviation σ are: Assume that the distance threshold θ=μ+σ·s, where s is the specified multiple of the standard deviation, and perform denoising on the distance threshold θ. When θ<D, remove the point, and when θ>D, retain the point.

6. The pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion according to claim 1, characterized in that: In the RANSAC plane segmentation method in step S3, a sufficient number of points are randomly selected as candidate inliers in each iteration, and then these points are used to fit a plane model. The plane equation can usually be expressed as: Ax+By+Cz+D=0 Where A, B, C and D are the coefficients of the plane equation. According to the fitted plane model, the distances from all points to the plane are calculated, and the set threshold is used to determine which points are in-plane points. If the distance from a point to the plane is less than the threshold, the point is considered to belong to the plane. The above process is repeated multiple times, and each iteration will obtain a new plane model and a set of in-plane points. Finally, the plane model with the most in-plane points is retained as the optimal result.

7. The pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion according to claim 1, characterized in that: In step S4, precise positioning is performed. The binary mask image is obtained through YOLOv8 target segmentation and positioning. The coordinates of the target pixel points on the RGB image can be determined. At the same time, the depth value corresponding to the pixel coordinates is obtained from the aligned depth map. Combined with the camera central transmission model, the 3D coordinates (X, Y, Z) corresponding to the target pixel points in the RGB image are calculated. The calculation formula is: In the formula, c x 、c y 、f x 、f y is the internal parameter of the RGB camera, u is the horizontal coordinate of the pixel, v is the vertical coordinate of the pixel, and Z is the depth value corresponding to the pixel (u, v).

8. The pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion according to claim 1, characterized in that: In the grasping operation of step S7, the pressure sensor is used to detect the pressure value of the clamping claw on the robot arm on the pipeline, which is achieved by measuring the output signal of the pressure sensor. The output signal is in a certain proportional relationship with the pressure value. The calculation formula is: P=(V-V0) / S Where P is the pressure value, V is the output signal of the pressure sensor, V0 is the zero point output of the pressure sensor, and S is the sensitivity of the sensor.

9. The pipeline grabbing method of a mining pipe grabber based on multi-sensor fusion according to claim 1, characterized in that: In step S1, SolidWorks is used to export the URDF model of the mining pipe grabbing robot.

Images