A Slag Detection System and Method Based on Video Detection Device and 3D LiDAR

Abstract

This invention discloses a slag detection system and method based on a video detection device and a 3D LiDAR. The system collects video data of a grid using a camera and employs a deep learning algorithm to identify the presence of slag on the grid. If no slag is present, the hydraulic shut-off gate is not required for slag crushing. If slag is present, the 3D LiDAR acquires the 3D laser point cloud data of the slag, and a slag height detection module identifies the slag height. Based on the height, it determines whether slag crushing is necessary. If the slag is too high on the grid, slag crushing is required. If the slag height indicates that crushing is unnecessary, a slag position detection module determines the slag position. If slag is found to be spread evenly on the grid, crushing is required to prevent blockage. This invention detects the amount of slag deposited on the grid and then controls the hydraulic shut-off gate to crush the slag, preventing blockage of the steel belt conveyor and avoiding energy waste.

Description

Technical Field

[0001] This invention relates to intelligent identification of slag discharge systems, and more particularly to a slag detection system and method based on a video detection device and a three-dimensional lidar. Background Technology

[0002] The air-cooled dry slag removal system uses a screen to block larger slag chunks, preventing blockage of the conveyor belt. Waste crushing is a common process in this system. A hydraulic shut-off gate on the screen crushes larger slag chunks, allowing them to fall into the conveyor belt below, thus reducing equipment blockage and accumulation.

[0003] Currently, waste residue crushing mainly relies on setting timers to periodically open and close the hydraulic shut-off gate. This method results in the hydraulic shut-off gate opening and closing even when there are no residue blocks on the screen, leading to energy waste.

[0004] Therefore, existing technologies have shortcomings and need to be improved. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a material slag detection system and method based on video detection device and three-dimensional lidar to overcome the shortcomings of the prior art.

[0006] The technical solution of the present invention is as follows:

[0007] A method for detecting slag based on a video detection device and a three-dimensional lidar includes the following steps:

[0008] S1: Collect video data of the grid through the camera, and use deep learning algorithm to identify whether there is slag on the grid. If there is no slag, there is no need to use the hydraulic shut-off gate to crush the slag. If there is slag, proceed to step S2.

[0009] S2: The three-dimensional laser point cloud data of the slag is acquired by the three-dimensional lidar, and then the height of the slag is identified by the slag height detection module. Based on the height, it is determined whether the slag needs to be crushed. If the slag is too high on the grid, the slag needs to be crushed. If the slag height is determined and it is found that the slag does not need to be crushed, the slag position detection module needs to determine the slag position. If it is determined that the slag is spread flat on the grid, the slag needs to be crushed to prevent the slag from spreading flat on the grid and causing blockage.

[0010] The aforementioned slag detection method, specifically step S2 includes the following steps:

[0011] The two sets of 3D lidars are spaced l apart along the horizontal x-axis. x The horizontal y-axis interval is l y The heights are h d1 and hd2 The height of the grid is h; the distance between the grid and the positions of the two sets of 3D LiDARs in the y-axis direction is [l]. 1dy1 ,l 1dy2 ] and [l 2dy1 ,l 2dy2 The distance along the x-axis is l. 1dx and l 2dx ;

[0012] S21, Acquisition and Processing of 3D Laser Point Cloud Data

[0013] First, a 3D lidar is used to scan the grid and its carrying space to obtain 3D lidar point cloud data of the slag, and then the distance S from the 3D lidar to the slag target is obtained. i Pitch angle α i , deviation angle β i ;

[0014] The horizontal distance between the slag point and the y-axis of the 3D lidar is: S xi =S i ×cosα i ×cosβ i ;

[0015] The horizontal distance between the slag point and the x-axis of the 3D lidar is: S yi =S i ×cosα i ×sinβ i ;

[0016] The vertical distance from the slag point to the 3D lidar is: S zi =S i ×sinα i ;

[0017] Therefore, the data obtained by scanning the slag with the first three-dimensional lidar is: the distance S from the lidar to the target. 1i Pitch angle α 1i , deviation angle β 1i ;

[0018] The location of the slag calculated using the first three-dimensional lidar is:

[0019] The horizontal distance between the slag point and the y-axis of the first 3D lidar is: S 1xi =S 1i ×cosα 1i ×cosβ 1i ;

[0020] The horizontal distance between the slag point and the x-axis of the first three-dimensional lidar is: S 1yi =S 1i ×cosα1i ×sinβ 1i ;

[0021] The vertical distance from the slag point to the first three-dimensional lidar is: S 1zi =S 1i ×sinα 1i ;

[0022] The actual height of the slag point is h. q1 =S 1zi +h d1 ;

[0023] The data obtained by scanning the slag with a second-dimensional lidar is: the distance S from the lidar to the target. 2i Pitch angle α 2i , deviation angle β 2i ;

[0024] The location of the slag calculated using the second-dimensional lidar is:

[0025] The horizontal distance between the slag point and the y-axis of the second 3D lidar is: S 2xi =S 2i ×cosα 2i ×cosβ 2i ;

[0026] The horizontal distance between the slag point and the x-axis of the second 3D lidar is: S 2yi =S 2i ×cosα 2i ×sinβ 2i ;

[0027] The vertical distance from the slag point to the second 3D lidar is: S 2zi =S 2i ×sinα 2i ;

[0028] The actual height of the slag point is h. q2 =S 2zi +h d2 ;

[0029] S22, Material slag height detection, to determine whether waste slag crushing is required;

[0030] First, determine the height of the slag material, and then set the threshold value for the horizontal x-axis direction of the grid as x. T The threshold value of the horizontal y-axis direction of the grille is y. T slag height threshold h T1 ;

[0031] The first three-dimensional lidar point cloud data, in the horizontal x-axis direction [l] 1dx +x T ,lx -l 2dx -x T ], y-axis direction [l 1dy1 +y T ,l 1dy2 -y T Search within the range, select h q1 The largest point h q1max This is the highest point of the slag;

[0032] The second three-dimensional lidar point cloud data, in the horizontal x-axis direction [l] 2dx +x T ,l x -l 1dx -x T ], y-axis direction [l 2dy1 +y T ,l 2dy2 -y T Search within the range, select h q2 The largest point h q2max This is the highest point of the slag;

[0033] If h q1max or h q2max It is greater than h T1 If the material is at a certain point, then waste residue crushing is required; otherwise, material residue crushing is not necessary.

[0034] S23, material residue location detection, to determine whether waste residue crushing is required;

[0035] If it is determined by judging the height of the slag that no slag crushing is required, then it is necessary to further determine whether slag crushing is required based on the position of the slag to prevent the slag from spreading flat on the grid and causing blockage.

[0036] Set a height threshold h for determining the presence of slag. T2 ;

[0037] First 3D laser scanner point cloud data: Select the point cloud data of the first 3D laser scanner, in the horizontal x-axis direction [l 1dx +x T ,l x -l 2dx -x T ], y-axis direction [l 1dy1 +y T ,l 1dy2 -y T Search within the range, searching for slag height h. q1 >h T2 Find the location of the slag along the x-axis and locate the position range [x]. 1x1 ,x 1x2 The position interval in the y-axis direction [y 1y1 ,y1y2 ];

[0038] Second 3D laser scanner point cloud data: Select the point cloud data of the second 3D laser scanner, in the horizontal x-axis direction [l 2dx +x T ,l x -l 1dx -x T ], y-axis direction [l 2dy1 +y T ,l 2dy2 -y T Search within the range, searching for slag height h. q2 >h T2 Find the location of the slag along the x-axis and locate the position range [x]. 2x1 ,x 2x2 The position interval in the y-axis direction [y 2y1 ,y 2y2 ];

[0039] Set the size threshold x of the slag in the x-axis direction. c The size threshold y in the y-axis direction c The size of the slag is determined based on the positional range of the two 3D laser scanners. The size of the slag in the x-axis direction is w. x =l x -x 1x1 -x 2x1 The magnitude of the data from the first 3D laser scanner in the y-axis direction is w. 1y =y 1y2 -y 1y1 The magnitude of the second 3D laser scanner data calculated in the y-axis direction is w. 2y =y 2y2 -y 2y1 If w x >x c or w 1y >y c or w 2y >y c If necessary, the slag needs to be crushed to prevent it from spreading evenly on the grid and causing blockage.

[0040] The aforementioned slag detection method uses the YOLOv5 algorithm as the deep learning algorithm: continuously acquiring video data on the grid through a camera, and using a deep learning model to identify the slag in the video frames in real time, so as to achieve automated slag detection.

[0041] (1) Data collection and preprocessing

[0042] Collect video data of the grid during its working state;

[0043] The video is segmented into single-frame images, and the images are preprocessed to crop the region of interest.

[0044] (2) Data labeling

[0045] Image annotation tools are used to annotate the material residue in a single frame image to generate the label data required for training;

[0046] (3) Model Training

[0047] YOLOv5 was selected as the base model and adjusted to meet the specific needs of slag identification.

[0048] Train the model on the segmented training set, and evaluate and fine-tune the model using the validation set;

[0049] (4) Model Deployment and Application

[0050] Deploy the trained model to a real production environment;

[0051] The system processes video data captured by cameras in real time, identifies and classifies the material residue on the grating, and provides feedback on the identification results.

[0052] The slag detection system according to any of the methods includes: a set of cameras installed in the observation window of the slag discharge system, two sets of three-dimensional LiDARs, a slag height detection module, and a slag position detection module. A set of three-dimensional LiDARs is installed before and after the observation window. The system collects video data of the grid using the cameras and uses the YoloV5 deep learning algorithm to identify whether there is slag on the grid. If there is no slag, the hydraulic shut-off gate is not required for slag crushing. If there is slag, the three-dimensional LiDARs acquire three-dimensional laser point cloud data of the slag, and the slag height detection module identifies the height of the slag. Based on the height, it is determined whether slag crushing is necessary. If the slag is too high on the grid, slag crushing is required. If the slag height indicates that slag crushing is not necessary, the slag position detection module determines the slag position. If it is determined that the slag is spread flat on the grid, slag crushing is required to prevent blockage caused by the slag spreading flat on the grid.

[0053] This invention detects the amount of slag deposited on the grid and then controls the hydraulic shut-off gate to crush the slag, which prevents the steel belt conveyor from getting clogged and avoids energy waste. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the slag detection system; A1 is the observation hole in front of the observation window; A2 is the installation position of the 3D LiDAR 1; A3 is the observation hole behind the observation window; A4 is the installation position of the 3D LiDAR 2.

[0055] Figure 2This is a diagram showing the positional relationship between the 3D lidar and the slag target. Detailed Implementation

[0056] The present invention will be described in detail below with reference to specific embodiments.

[0057] A slag detection system based on video detection device and three-dimensional lidar, such as Figure 1 As shown, the system includes: a camera and two sets of 3D LiDARs installed in the observation window of the slag discharge system, a slag height detection module, and a slag position detection module. A set of 3D LiDARs is installed before and after the observation window. The camera collects video data of the grid, and the deep learning algorithm YoloV5 is used to identify whether there is slag on the grid. If there is no slag, the hydraulic shut-off gate is not needed for slag crushing. If there is slag, the 3D LiDARs acquire the 3D laser point cloud data of the slag, and the slag height detection module identifies the height of the slag. If the slag is too large or too high on the grid, slag crushing is required. First, the height is used to determine whether slag crushing is necessary. If the height determines that slag crushing is not necessary, the slag position detection module determines the slag position and whether crushing is required to prevent slag from spreading evenly on the grid and causing blockage.

[0058] A method for detecting slag based on video detection devices and 3D lidar includes the following steps:

[0059] S1: Collect video data of the grid through the camera, and use the deep learning algorithm YoloV5 to identify whether there is slag on the grid. If there is no slag, there is no need to use the hydraulic shut-off gate to crush the slag. If there is slag, proceed to step S2.

[0060] The YOLOv5 deep learning algorithm continuously collects video data from the grid using a camera and uses a deep learning model to identify slag in the video frames in real time, thereby achieving automated slag detection.

[0061] (1) Data collection and preprocessing

[0062] Collect video data of the grid during its working state.

[0063] The video is segmented into single-frame images, and the images are preprocessed to crop the regions of interest.

[0064] (2) Data labeling

[0065] Image annotation tools are used to annotate the material residue in a single frame image to generate the label data required for training.

[0066] (3) Model Training

[0067] YOLOv5 was chosen as the base model and adjusted to meet the specific needs of slag identification.

[0068] The model is trained on the segmented training set and evaluated and tuned using the validation set.

[0069] (4) Model Deployment and Application

[0070] Deploy the trained model to the actual production environment.

[0071] The system processes video data captured by cameras in real time, identifies and classifies the material residue on the grating, and provides feedback on the identification results.

[0072] S2: Determine whether slag crushing is necessary based on the height or size of the slag; obtain slag data by scanning the grid with a 3D LiDAR to determine the height or position of the slag on the grid, and then determine whether slag crushing is necessary; the horizontal x-axis spacing between the two sets of 3D LiDAR is l. x The horizontal y-axis interval is l y The heights are h d1 and h d2 The height of the grille is h.

[0073] The distance between the grid and the positions of the two sets of 3D LiDARs along the y-axis is [l]. 1dy1 ,l 1dy2 ] and [l 2dy1 ,l 2dy2 The distance along the x-axis is l. 1dx and l 2dx .

[0074] S21, Acquisition and Processing of 3D Laser Point Cloud Data

[0075] First, a 3D lidar is used to scan the grid and its carrying space to obtain 3D lidar point cloud data of the slag, and then the distance S from the 3D lidar to the slag target is obtained. i Pitch angle α i , deviation angle β i ,like Figure 2 As shown in the figure, the black dots represent the slag target.

[0076] The horizontal distance between the slag point and the y-axis of the 3D lidar is: S xi =S i ×cosα i ×cosβ i .

[0077] The horizontal distance between the slag point and the x-axis of the 3D lidar is: S yi =S i ×cosα i ×sinβi .

[0078] The vertical distance from the slag point to the 3D lidar is: S zi =S i ×sinα i .

[0079] Therefore, the data obtained by scanning the slag with a 3D lidar is: the distance S from the lidar to the target. 1i Pitch angle α 1i , deviation angle β 1i .

[0080] The location of the slag calculated by 3D lidar 1 is:

[0081] The horizontal distance between the slag point and the y-axis of the 3D lidar 1 is: S 1xi =S 1i ×cosα 1i ×cosβ 1i .

[0082] The horizontal distance between the slag point and the x-axis of the 3D lidar 1 is: S 1yi =S 1i ×cosα 1i ×sinβ 1i .

[0083] The vertical distance from the slag point to the 3D lidar 1 is: S 1zi =S 1i ×sinα 1i .

[0084] The actual height of the slag point is h. q1 =S 1zi +h d1 .

[0085] The data obtained by scanning the slag with a 3D lidar 2 is: the distance S from the lidar to the target. 2i Pitch angle α 2i , deviation angle β 2i .

[0086] The location of the slag calculated using 3D lidar 2 is:

[0087] The horizontal distance between the slag point and the y-axis of the 3D lidar 2 is: S 2xi =S 2i ×cosα 2i ×cosβ 2i .

[0088] The horizontal distance between the slag point and the x-axis of the 3D lidar 2 is: S 2yi =S 2i×cosα 2i ×sinβ 2i .

[0089] The vertical distance from the slag point to the 3D lidar 2 is: S 2zi =S 2i ×sinα 2i .

[0090] The actual height of the slag point is h. q2 =S 2zi +h d2 .

[0091] S22, material slag height detection, to determine whether waste slag crushing is required.

[0092] If the slag is too large or too high on the screen, it needs to be crushed.

[0093] First, determine the height of the slag material, and then set the threshold value for the horizontal x-axis direction of the grid as x. T The threshold value of the horizontal y-axis direction of the grille is y. T slag height threshold h T1 .

[0094] 3D LiDAR point cloud data, in the horizontal x-axis direction [l 1dx +x T ,l x -l 2dx -x T ], y-axis direction [l 1dy1 +y T ,l 1dy2 -y T Search within the range, select h q1 The largest point h q1max This is the highest point of the slag.

[0095] 3D LiDAR point cloud data, in the horizontal x-axis direction [l 2dx +x T ,l x -l 1dx -x T ], y-axis direction [l 2dy1 +y T ,l 2dy2 -y T Search within the range, select h q2 The largest point h q2max This is the highest point of the slag.

[0096] If h q1max or h q2max It is greater than h T1 If the material is at a certain point, then waste residue crushing is required; otherwise, material residue crushing is not necessary.

[0097] S23, material residue position detection, to determine whether waste residue crushing is required.

[0098] If judging the height of the slag reveals that slag crushing is unnecessary, then it is necessary to further determine whether slag crushing is required based on the location of the slag to prevent the slag from spreading evenly on the grid and causing blockage.

[0099] Set a height threshold h for determining the presence of slag. T2 .

[0100] 3D Laser Scanner 1 Point Cloud Data: Select the point cloud data of 3D Laser Scanner 1, in the horizontal x-axis direction [l 1dx +x T ,l x -l 2dx -x T ], y-axis direction [l 1dy1 +y T ,l 1dy2 -y T Search within the range, searching for slag height h. q1 >h T2 Find the location of the slag along the x-axis and locate the position range [x]. 1x1 ,x 1x2 The position interval in the y-axis direction [y 1y1 ,y 1y2 ].

[0101] 3D Laser Scanner 2 Point Cloud Data: Select the point cloud data of 3D Laser Scanner 2, in the horizontal x-axis direction [l 2dx +x T ,l x -l 1dx -x T ], y-axis direction [l 2dy1 +y T ,l 2dy2 -y T Search within the range, searching for slag height h. q2 >h T2 Find the location of the slag along the x-axis and locate the position range [x]. 2x1 ,x 2x2 The position interval in the y-axis direction [y 2y1 ,y 2y2 ].

[0102] Set the size threshold x of the slag in the x-axis direction. c The size threshold y in the y-axis direction c Based on the positional range of the two 3D laser scanners, the size of the slag is determined, and the size of the slag in the x-axis direction is w. x =l x -x 1x1-x 2x1 The magnitude of the data calculated from the 3D laser scanner 1 in the y-axis direction is w. 1y =y 1y2 -y 1y1 The magnitude of the data calculated from the 3D laser scanner 2 in the y-axis direction is w. 2y =y 2y2 -y 2y1 If w x >x c or w 1y >y c or w 2y >y c If necessary, the slag needs to be crushed to prevent it from spreading evenly on the grid and causing blockage.

[0103] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for detecting slag based on a video detection device and a three-dimensional lidar, characterized in that, Includes the following steps: S1: Collect video data of the grid through the camera, and use deep learning algorithm to identify whether there is slag on the grid. If there is no slag, there is no need to use the hydraulic shut-off gate to crush the slag. If there is slag, proceed to step S2. S2: The three-dimensional laser point cloud data of the slag is acquired by the three-dimensional LiDAR, and then the height of the slag is identified by the slag height detection module. Based on the height, it is determined whether the slag needs to be crushed. If the slag is too high on the grid, the slag needs to be crushed. If the slag height is determined and it is found that the slag does not need to be crushed, the slag position detection module needs to determine the slag position. If it is determined that the slag is spread flat on the grid, the slag needs to be crushed to prevent the slag from spreading flat on the grid and causing blockage. Step S2 specifically includes the following steps: the two sets of three-dimensional lidars are spaced apart along the horizontal x-axis. The horizontal y-axis interval is The heights are respectively and The height of the grille is The distance between the grid and the positions of the two sets of 3D LiDARs along the y-axis is... and The distance along the x-axis is and ; S21, Acquisition and processing of 3D laser point cloud data; First, a 3D lidar scan is used to scan the grid and its carrying space to obtain 3D lidar point cloud data of the slag, and then the distance of the 3D lidar to the slag target is obtained. Pitch angle , deviation angle The horizontal distance between the slag point and the y-axis of the 3D lidar is: The horizontal distance between the slag point and the x-axis of the 3D lidar is: The vertical distance from the slag point to the 3D lidar is: Therefore, the data obtained by scanning the slag with the first three-dimensional lidar is: the distance of the lidar to the target. Pitch angle , deviation angle The location of the slag, calculated by the first three-dimensional lidar, is as follows: The horizontal distance of the slag point from the y-axis of the first three-dimensional lidar is: The horizontal distance from the slag point to the x-axis of the first 3D lidar is: The vertical distance from the slag point to the first three-dimensional lidar is: The actual height of the slag point is The data obtained by scanning the slag with a second-dimensional lidar is: the distance the lidar reaches the target. Pitch angle , deviation angle The location of the slag, calculated using the second 3D lidar, is as follows: The horizontal distance from the slag point to the y-axis of the second 3D lidar is: The horizontal distance from the slag point to the x-axis of the second 3D lidar is: The vertical distance from the slag point to the second 3D lidar is: The actual height of the slag point is ; S22, Slag Height Detection: Determines whether waste slag crushing is necessary; First, determine the slag height, and set the horizontal x-axis threshold of the grid as follows. The threshold value for the horizontal y-axis direction of the grille is Slag Height Threshold The first three-dimensional lidar point cloud data, in the horizontal x-axis direction. y-axis direction Search within range, select The largest point The highest point of the slag; the second three-dimensional lidar point cloud data, in the horizontal x-axis direction. y-axis direction Search within range, select The largest point This is the highest point of the slag; if or It is greater than If the material is at a certain point, then waste residue crushing is required; otherwise, material residue crushing is not necessary. S23, Slag position detection to determine if slag crushing is necessary; if the slag height is determined and crushing is not required, further determination based on slag position is needed to prevent slag from spreading evenly on the grid and causing blockage; a slag height threshold is set for determining slag presence. Point cloud data from the first 3D laser scanner: Select the point cloud data from the first 3D laser scanner, along the horizontal x-axis. y-axis direction Search within range, search for slag height Find the location of the slag along the x-axis. The position range in the y-axis direction Point cloud data from a second 3D laser scanner: Select the point cloud data from a second 3D laser scanner, along the horizontal x-axis. y-axis direction Search within range, search for slag height Find the location of the slag along the x-axis. The position range in the y-axis direction Set the size threshold of the slag in the x-axis direction. Size threshold in the y-axis direction Based on the positional range of the two 3D laser scanners, the size of the slag is determined. The size of the slag in the x-axis direction is... The magnitude in the y-axis direction calculated from the data of the first 3D laser scanner is The magnitude in the y-axis direction calculated from the data from the second 3D laser scanner is ,like or or If necessary, the slag needs to be crushed to prevent it from spreading evenly on the grid and causing blockage.

2. The method for detecting slag according to claim 1, characterized in that, The deep learning algorithm mentioned is the YOLOv5 algorithm: video data on the grid is continuously collected by the camera, and the deep learning model is used to identify the slag in the video frames in real time to achieve automated slag detection. (1) Data collection and preprocessing; acquiring video data of the grid in operation; The video is segmented into single-frame images, and the images are preprocessed to crop the region of interest. (2) Data labeling; Image annotation tools are used to annotate the material residue in a single frame image to generate the label data required for training; (3) Model training; YOLOv5 was selected as the base model and adjusted to meet the specific needs of slag identification; the model was trained on the segmented training set and evaluated and optimized using the validation set; (4) Model deployment and application; deploying the trained model to the actual production environment; The system processes video data captured by cameras in real time, identifies and classifies the material residue on the grating, and provides feedback on the identification results.

3. The slag detection system according to any one of claims 1-2, characterized in that, include: A camera, two sets of 3D LiDAR, a slag height detection module, and a slag position detection module are installed in the observation window of the slag discharge system. A set of 3D LiDAR is installed before and after the observation window. The camera collects video data from the grid, and the YoloV5 deep learning algorithm is used to identify whether there is slag on the grid. If there is no slag, the hydraulic shut-off gate is not needed for slag crushing. If there is slag, the 3D LiDAR acquires the 3D laser point cloud data of the slag, and the slag height detection module identifies the height of the slag. Based on the height, it is determined whether slag crushing is necessary. If the slag is too high on the grid, slag crushing is required. If the slag height indicates that slag crushing is not necessary, the slag position detection module determines the slag position. If the slag is found to be spread flat on the grid, slag crushing is required to prevent blockage.

Images