Automatic Lithotripsy Method, Device and Medium Based on 3D LiDAR and Camera

Abstract

An automatic gravel crushing method, device and medium based on 3D lidar and camera, which relates to the field of gravel crushers. By installing the 3D lidar and camera on the device, the 3D point cloud data and image data of the ore are obtained in real time. The calculation unit realizes the segmentation and positioning of the ore and obtains the position coordinates of the center of the ore shape through the processing and analysis of the data of the 3D lidar and camera. The control system converts the coordinate information into motion information and transmits it to the actuator, and the actuator realizes the automatic gravel crushing operation of the ore. The position of the ore is automatically identified by using the data fusion of the image and the radar point cloud, and the position of the ore is converted into the motion information of the actuator through inverse kinematics analysis, so that the breaker automatically crushes the gravel. The safety and production efficiency of ore mining are improved.

Description

Technical Field

[0001] The present invention relates to the field of crushers, and particularly to an automatic crushing method, device and medium based on a three-dimensional lidar and a camera. Background Art

[0002] In the traditional mining process, crushing is an important link. Traditional crushing operations usually require a large amount of manpower and equipment investment, with low operation efficiency and potential safety hazards. Therefore, it is imperative to develop an automatic crushing system that can improve the operation efficiency of mine crushing, reduce manpower input and enhance safety. Summary of the Invention

[0003] In order to overcome the above deficiencies in technology, the present invention provides an automatic crushing method, device and medium based on a three-dimensional lidar and a camera, which improve the safety and production efficiency of ore mining.

[0004] The technical solution adopted by the present invention to overcome its technical problems is as follows:

[0005] An automatic crushing method based on a three-dimensional lidar and a camera includes:

[0006] Install a three-dimensional lidar and a camera on the rotating platform of the crusher;

[0007] Start the device, and transmit the images captured by the camera and the 3D point cloud collected by the lidar to the computing unit;

[0008] The computing unit performs target detection and segmentation on the ore based on the fusion of the image and the 3D point cloud into a target detection model, and obtains the position center coordinates of the ore;

[0009] The control system converts the position center coordinates of the ore into the motion information of the actuator;

[0010] According to the motion information, the breaker performs the crushing action.

[0011] Furthermore, the method for the computing unit to perform target detection and segmentation on the ore based on the fusion of the image and the 3D point cloud into a target detection model, and obtain the position center coordinates of the ore is as follows:

[0012] Locate the stone area in the image captured by the camera, and perform a segmentation operation after location to obtain the stone area image;

[0013] Preprocess the 3D point cloud collected by the three-dimensional lidar to obtain 2D point cloud data;

[0014] Calibrate the camera and the 3D lidar jointly to obtain the external parameter matrix of the camera and the lidar. Use the external parameter matrix to map the 2D point cloud data onto the stone area image to obtain the distance point cloud segmentation of the stone area; obtain the shape information of the stone point cloud according to the distance point cloud segmentation of the stone area, and obtain the position center coordinates of the ore according to the shape information of the stone point cloud.

[0015] Further, preprocess the 3D point cloud by voxel grid downsampling or Euclidean point cloud segmentation.

[0016] Further, the motion information includes the target angle θ of the crusher boom 1 , the target angle θ of the crusher forearm 2 , the target angle θ of the breaker 3 , the target angle θ of the crusher rotating platform (6) 4 .

[0017] Further, the method for the control system to convert the position center coordinates of the ore into the motion information of the actuator is: through the formula Calculate to obtain the target angle θ of the crusher boom 1 , where Y is the Y-axis coordinate value of the position center of the ore, and H 1 is the Y-axis coordinate of the fulcrum of the crusher boom, H 0 is the length of the breaker, Z is the Z-axis coordinate value of the position center of the ore, and H 2 is the height of the fulcrum of the crusher boom, L 2 is the distance from the fulcrum of the crusher boom to the fulcrum of the breaker, L 0 is the distance from the fulcrum of the crusher boom to the fulcrum of the crusher forearm, L 1 is the distance from the fulcrum of the crusher forearm to the fulcrum of the breaker;

[0018] Through the formula Calculate to obtain the target angle θ of the crusher forearm 2 ;

[0019] Through the formula Calculate to obtain the target angle θ of the breaker 3 , where

[0020] Through the formula Calculate to obtain the target angle θ of the crusher rotating platform 4 , where X is the X-axis coordinate value of the position center of the ore.

[0021] An automatic stone crushing device based on a 3D lidar and a camera, comprising:

[0022] At least one processor, and

[0023] A memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to:

[0024] Transmit the images captured by the camera and the 3D point cloud collected by the lidar to the computing unit;

[0025] The computing unit performs object detection and segmentation on the ore based on the fusion of the image and the 3D point cloud into the object detection model to obtain the position center coordinates of the ore;

[0026] The control system converts the position center coordinates of the ore into the motion information of the actuator;

[0027] According to the motion information, the breaker performs the gravel crushing action.

[0028] A non-volatile computer storage medium stores computer-executable instructions, and the computer-executable instructions are configured to:

[0029] Transmit the images captured by the camera and the 3D point cloud collected by the lidar to the computing unit;

[0030] The computing unit performs object detection and segmentation on the ore based on the fusion of the image and the 3D point cloud into the object detection model to obtain the position center coordinates of the ore;

[0031] The control system converts the position center coordinates of the ore into the motion information of the actuator;

[0032] According to the motion information, the breaker performs the gravel crushing action.

[0033] The beneficial effects of the present invention are as follows: Real-time acquisition of the three-dimensional point cloud data and image data of the ore, through the processing and analysis of the three-dimensional lidar and camera data, realizing the detection, segmentation and positioning of the ore, through the inverse kinematics analysis of the ore position and using the fuzzy PID control model to realize the automatic gravel crushing operation of the ore. Improve the automation degree of the gravel crushing operation, reduce the labor input and lower the labor intensity, based on the real-time data acquisition of the three-dimensional lidar and camera, improve the positioning and detection accuracy of the gravel crushing equipment, improve the efficiency and stability of the gravel crushing operation, reduce unnecessary resource waste, improve the safety of the mine exploitation process, and reduce the occurrence of accidents and injuries. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic three-dimensional structure diagram of a crusher

[0035] Figure 2 It is a flowchart of the image and point cloud data fusion of the present invention;

[0036] 1. Crusher 2. Crusher arm 3. Crusher arm 4. 3D laser radar 5. Camera 6. Crusher rotating platform. DETAILED DESCRIPTION

[0037] The following is combined with Figure 1 , Attachment Figure 2 The present invention is further described.

[0038] An automatic stone crushing method based on three-dimensional laser radar and camera, comprising:

[0039] A three-dimensional laser radar 4 and a camera 5 are installed on the crusher rotating platform 6 .

[0040] The device is started, and the image captured by the camera 5 and the 3D point cloud collected by the laser radar 4 are transmitted to the computing unit. The computing unit performs target detection and segmentation on the ore based on the fusion of the image and the 3D point cloud into the target detection model to obtain the position center coordinates of the ore.

[0041] The control system converts the center coordinates of the ore into motion information of the actuator.

[0042] According to the motion information, the breaker 1 performs a rock-breaking action.

[0043] By installing 3D laser radar and camera on the equipment, the 3D point cloud data and image data of the ore can be obtained in real time. The computing unit processes and analyzes the 3D laser radar and camera data to segment and locate the ore and obtain the position coordinates of the center of the ore shape. The control system converts the coordinate information into motion information and transmits it to the actuator, which realizes the automatic crushing operation of the ore.

[0044] In one embodiment of the present invention, the computing unit performs target detection and segmentation on the ore based on the fusion of the image and the 3D point cloud into the target detection model, and the method for obtaining the position center coordinates of the ore is:

[0045] The stone area in the image taken by the camera 5 is located, and after the location is located, a segmentation operation is performed to obtain an image of the stone area.

[0046] The 3D point cloud collected by the three-dimensional laser radar 4 is pre-processed to obtain 2D point cloud data.

[0047] The camera 5 and the three-dimensional laser radar 4 are jointly calibrated to obtain the external parameter matrices of the camera and the laser radar, and the extrinsic parameter matrix is ​​used to map the 2D point cloud data to the stone area image to obtain the distance point cloud segmentation of the stone area; the stone point cloud shape information is obtained according to the distance point cloud segmentation to the stone area, and the position center coordinates of the ore are obtained according to the stone point cloud shape information.

[0048] In one embodiment of the present invention, the 3D point cloud is preprocessed by voxel network downsampling or Euclidean point cloud segmentation.

[0049] In one embodiment of the present invention, the motion information includes the target angle θ of the crusher boom 3 1 , the target angle θ of the crusher forearm 2 2 , the target angle θ of the breaker hammer 1 3 , and the target angle θ of the crusher rotating platform 6 4 . The motion information including the target angle θ of the crusher boom 3 1 , the target angle θ of the crusher forearm 2 2 , the target angle θ of the breaker hammer 1 3 , and the target angle θ of the crusher rotating platform 6 4 is input into the fuzzy PID control model to output the motion information of the actuator.

[0050] In this embodiment, the method for the control system to convert the position center coordinates of the ore into the motion information of the actuator is as follows:

[0051] By the formula Calculate the target angle θ of the crusher boom 3 1 , where Y is the Y-axis coordinate value of the position center of the ore, and H 1 is the Y-axis coordinate of the fulcrum of the crusher boom 3, H 0 is the length of the breaker hammer 1, Z is the Z-axis coordinate value of the position center of the ore, and H 2 is the height of the fulcrum of the crusher boom 3, L 2 is the distance from the fulcrum of the crusher boom 3 to the fulcrum of the breaker hammer 1, L 0 is the distance from the fulcrum of the crusher boom 3 to the fulcrum of the crusher forearm 2, L 1 is the distance from the fulcrum of the crusher forearm 2 to the fulcrum of the breaker hammer 1;

[0052] By the formula Calculate the target angle θ of the crusher forearm 2 2 ; By the formula Calculate the target angle θ of the breaker hammer 1 3 , where

[0053] By the formula Calculate the target angle θ of the crusher rotating platform 6 4 , where X is the X-axis coordinate value of the position center of the ore.

[0054] The present invention also relates to an automatic gravel crushing device based on a three-dimensional lidar and a camera, comprising:

[0055] at least one processor, and

[0056] a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0057] transmit an image captured by a camera and 3D point cloud collected by a lidar to a computing unit;

[0058] the computing unit performs object detection and segmentation on ore based on the fusion of the image and the 3D point cloud into an object detection model to obtain the position center coordinates of the ore;

[0059] a control system converts the position center coordinates of the ore into motion information of an actuator;

[0060] According to the motion information, a breaker performs a gravel crushing action.

[0061] The present invention also relates to a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are configured to:

[0062] transmit an image captured by a camera and 3D point cloud collected by a lidar to a computing unit;

[0063] the computing unit performs object detection and segmentation on ore based on the fusion of the image and the 3D point cloud into an object detection model to obtain the position center coordinates of the ore;

[0064] a control system converts the position center coordinates of the ore into motion information of an actuator;

[0065] According to the motion information, a breaker performs a gravel crushing action.

[0066] Finally, it should be noted that: The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automatic gravel crushing method based on a 3D lidar and a camera, characterized in that, it includes: Install a 3D lidar (4) and a camera (5) on the crusher rotating platform (6); Start the device and transmit the image captured by the camera (5) and the 3D point cloud collected by the lidar (4) to the calculation unit; The calculation unit performs target detection and segmentation on the ore based on the fusion of the image and the 3D point cloud into the target detection model to obtain the position center coordinates of the ore; The control system converts the position center coordinates of the ore into the motion information of the actuator; According to the motion information, the breaker (1) performs the gravel crushing action; The motion information includes the target angle θ of the crusher boom (3) 1 , the target angle θ of the crusher forearm (2) 2 , the target angle θ of the breaker (1) 3 , the target angle θ of the crusher rotating platform (6) 4 ; The method by which the control system converts the position center coordinates of the ore into the motion information of the actuator is: The target angle θ of the large arm (3) of the crusher is calculated through the formula , where Y is the Y-axis coordinate value of the position center of the ore, and H 1 is the Y-axis coordinate of the fulcrum of the large arm (3) of the crusher, 1 H is the length of the breaker (1), Z is the Z-axis coordinate value of the position center of the ore, and H 0 is the height of the fulcrum of the large arm (3) of the crusher, 2 L 2 is the distance from the fulcrum of the large arm (3) of the crusher to the fulcrum of the breaker (1), and L 0 is the distance from the fulcrum of the large arm (3) of the crusher to the fulcrum of the small arm (2) of the crusher, and L 1 is the distance from the fulcrum of the small arm (2) of the crusher to the fulcrum of the breaker (1);​ Calculate the target angle θ of the crusher's small arm (2) through the formula ; Calculate the target angle θ of the breaker (1) through the formula 2 ; In the formula Calculate the target angle θ of the crusher's rotating platform (6) through the formula 3 , where Calculate the target angle θ of the crusher's rotating platform (6) through the formula , where X is the X-axis coordinate value of the position center of the ore 4 .

2. The automatic gravel crushing method based on a 3D lidar and a camera according to claim 1, characterized in that, The method by which the calculation unit performs target detection and segmentation on the ore based on the fusion of the image and the 3D point cloud into the target detection model to obtain the position center coordinates of the ore is: Locate the stone area in the image captured by the camera (5), and perform a segmentation operation after positioning to obtain the stone area image; Preprocess the 3D point cloud collected by the 3D lidar (4) to obtain 2D point cloud data; Perform joint calibration on the camera (5) and the 3D lidar (4) to obtain the external parameter matrix of the camera and the lidar, and use the external parameter matrix to map the 2D point cloud data onto the stone area image to obtain the distance point cloud segmentation of the stone area; Obtain the stone point cloud shape information according to the distance point cloud segmentation of the stone area, and obtain the position center coordinates of the ore according to the stone point cloud shape information.

3. The automatic gravel crushing method based on a 3D lidar and a camera according to claim 2, characterized in that: Preprocess the 3D point cloud by voxel network downsampling or Euclidean point cloud segmentation.

Images