A Belt Deviation Detection and Coal Piling Detection Method Based on 3D Point Cloud

Abstract

The present invention provides a method for detecting belt deviation and coal accumulation based on 3D point cloud, which relates to the technology of belt conveyor fault detection. The present invention realizes the segmentation and processing of the semantic information of the point cloud; through the processing of noise points and the deep extraction of the local features of the point set, the segmentation result is more accurate and stable, and based on the processed point cloud, the accurate detection of belt deviation and coal accumulation is realized, avoiding the influence of the belt conveyor operation environment and ensuring the safety of the belt conveyor operation.

Description

Technical Field

[0001] The present invention relates to the technical field of belt conveyor fault detection, and particularly to a belt deviation detection and coal stacking detection method based on three-dimensional point cloud. Background Art

[0002] At present, belt conveyors are the main transportation equipment in coal mine production. The characteristics of long distance and multi-section installation make the influence of belt deviation and coal stacking faults extremely large. Often, a problem with one section of the belt affects the entire transportation line and requires a shutdown inspection.

[0003] In practical applications, contact sensors are mostly installed on belt conveyors to detect belt deviation or coal stacking faults. However, due to the harsh coal mine production environment, the sensitivity of sensors is affected by coal ash, vibration during material transportation, transportation pressure, etc., resulting in failure and serious decline in durability. Sensors need to be frequently replaced, and they cannot meet the requirements of coal mine safety production in terms of reliability.

[0004] Another method is for belt deviation or coal stacking faults of belt conveyors based on image processing. Belt deviation detection is mostly achieved by identifying changes in the belt edge or the relative position relationship between the idler and the belt edge. Coal stacking detection is calculated by identifying the proportion of materials in the overall area. However, due to the large interference of dust and water vapor on the image, and the poor illumination conditions in the underground environment, in the case of material spilling around the belt conveyor, the change in color information affects the image segmentation effect, resulting in misidentification. Summary of the Invention

[0005] In view of the above problems in the prior art, the present invention provides a belt deviation detection and coal stacking detection method based on three-dimensional point cloud.

[0006] In order to achieve the above invention object, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, a three-dimensional point cloud processing method includes the following steps:

[0008] S1. Obtain three-dimensional point cloud data;

[0009] S2. Based on the three-dimensional point cloud data, select N' points through the iterative farthest point sampling algorithm to define N' central points of the local area;

[0010] S3. Based on the N' central points, search for k adjacent points within the range r of each central point through the ball query algorithm to construct a local area;

[0011] S4. Extract features from the local area, and perform multi-scale feature extraction based on the N' central points to obtain multi-scale features;

[0012] S5. Use the PCL clustering algorithm to filter out the noise in the multi-scale features and obtain the point cloud filtered data.

[0013] Preferably, in step S1, the three-dimensional point cloud data is obtained by the lidar installed directly above the belt conveyor.

[0014] Preferably, in step S2, based on the three-dimensional point cloud data, the method of selecting N' points by the iterative farthest point sampling algorithm to define the N' central points of the local area includes the following steps:

[0015] Step S21. Establish a rectangular coordinate system OXYZ of the three-dimensional point cloud data with the center of the equipment support of the belt conveyor as the origin O, the belt running direction of the belt conveyor as the X axis, the belt width direction of the belt conveyor as the Y axis, and the belt height direction of the belt conveyor as the Z axis;

[0016] Step S22. Determine a sampling point A from the three-dimensional point cloud data, and determine a sampling point B that is the farthest from the sampling point from the remaining points, and obtain a sampling point set S = {A, B};

[0017] Step S23. Determine a sampling point P that is the farthest from the sampling point set from the remaining points, and update the sampling point set;

[0018] Step 24. Repeat step 23 to obtain N' points.

[0019] Preferably, in step S3, based on the N' central points, the method of searching for the k adjacent points within the range r of each central point by the ball query algorithm to construct the local area includes the following steps:

[0020] Step S31. Set the parameters r and k;

[0021] Step S32. Respectively use the N' central points as the centers of the spheres, draw spheres with the r as the radius, and find k points within the spheres to construct the local area.

[0022] Preferably, in step S4, perform feature extraction on the local area, and perform multi-scale feature extraction based on the N' central points to obtain the multi-scale features, including the following steps:

[0023] Step S41. Normalize the points within each local area, and encode each local area with the N' central points as the centers to obtain the first multi-scale feature;

[0024] Step S42. For each central point, sample the multi-scale multi-level structure, construct n sampling areas, and perform feature extraction on each sampling area to obtain the second multi-scale feature.

[0025] Second aspect, a belt deviation detection method based on 3D point cloud, comprising the following steps:

[0026] Step A1: Obtain point cloud filtering data during coal transportation by the belt conveyor based on a 3D point cloud processing method, and divide the point cloud filtering data into multiple data segments;

[0027] Step A2: Take the maximum and minimum values of the Y-axis coordinates of the points in each data segment as the left and right edge bands of the belt, calculate the central value of each left and right edge band, calculate the average central value of each data segment according to the central value, and determine whether the belt of the belt conveyor is deviated according to the average central value.

[0028] Third aspect, a coal stacking detection method based on 3D point cloud, comprising the following steps:

[0029] Step B1: Obtain the first point cloud filtering data during coal transportation by the belt conveyor and the first point cloud filtering data during no-load based on a 3D point cloud processing method;

[0030] Step B2: Determine n detection points from the first point cloud filtering data and the second point cloud filtering data respectively, calculate the average value of the Z-axis coordinates of the n detection points to obtain a first equivalent height and a second equivalent height;

[0031] Step B3: Judge whether the belt of the belt conveyor has a coal stacking fault according to the difference between the first equivalent height and the second equivalent height.

[0032] The beneficial effects of the present invention are as follows: The present invention realizes the segmentation processing of the semantic information of the point cloud; through the processing of noise points and the deep extraction of the local features of the point set, the segmentation result is more accurate and stable, and based on the processed point cloud, accurate detection of belt deviation and coal stacking is realized, avoiding the influence of the belt conveyor operation environment, ensuring the safe operation of the belt conveyor, increasing the service life, and reducing the accident shutdown loss. Description of the Drawings

[0033] Figure 1 It is a flowchart of a 3D point cloud processing method provided by an embodiment of the present invention;

[0034] Figure 2 It is a sub-flowchart of a 3D point cloud processing method provided by an embodiment of the present invention;

[0035] Figure 3 It is a sub-flowchart of a 3D point cloud processing method provided by an embodiment of the present invention;

[0036] Figure 4 It is a sub-flowchart of a 3D point cloud processing method provided by an embodiment of the present invention;

[0037] Figure 5 Schematic flow chart of a belt deviation detection method based on 3D point cloud provided by an embodiment of the present invention;

[0038] Figure 6 Schematic flow chart of a coal heap detection method based on 3D point cloud provided by an embodiment of the present invention. Specific embodiments

[0039] The following describes the specific embodiments of the present invention to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concept of the present invention are within the scope of protection.

[0040] Embodiment 1

[0041] As Figure 1 shown, Figure 1 is a flow chart of a 3D point cloud processing method provided by an embodiment of the present invention, including the following steps:

[0042] S1. Obtain 3D point cloud data;

[0043] In the embodiment of the present invention, 3D point cloud data is obtained by a lidar installed directly above the belt conveyor; since the lidar is installed directly above the belt conveyor, the lidar is not affected by the materials running on the belt conveyor and can collect accurate 3D point cloud data.

[0044] S2. Based on the 3D point cloud data, select N' points through the iterative farthest point sampling algorithm to define N' center points of the local area;

[0045] Refer to Figure 2 , preferably, in step S2, the method of selecting N' points through the iterative farthest point sampling algorithm based on the 3D point cloud data to define N' center points of the local area includes the following steps: Step S21. Establish a rectangular coordinate system OXYZ of the 3D point cloud data with the center of the equipment support of the belt conveyor as the origin O, the belt running direction of the belt conveyor as the X axis, the belt width direction of the belt conveyor as the Y axis, and the belt height direction of the belt conveyor as the Z axis; Step S22. Determine a sampling point A from the 3D point cloud data, and determine a sampling point B that is the farthest from the sampling point from the remaining points, to obtain a sampling point set S = {A, B}; Step S23. Determine a sampling point P that is the farthest from the sampling point set from the remaining points, and update the sampling point set; Step 24. Repeat step 23 to obtain N' points.

[0046] The above method can be used to sample 3D point cloud data, facilitating the further abstraction of the subsequent 3D point cloud data.

[0047] S3. Based on the N' center points, search for the k adjacent points within the range r of each center point through a ball query algorithm to construct a local area;

[0048] Reference Figure 3 , preferably, in step S3, the method of searching for the k adjacent points within the range r of each center point through a ball query algorithm to construct a local area based on the N' center points includes the following steps: Step S31. Set the parameters r and k; Step S32. Respectively use the N' center points as the centers of the balls, draw balls with the r as the radius, and find k points within the balls to construct a local area.

[0049] The above method can be used to group 3D point cloud data, facilitating the further abstraction of the subsequent 3D point cloud data.

[0050] S4. Extract features from the local area, perform multi-scale feature extraction based on the N' center points to obtain multi-scale features;

[0051] Reference Figure 4 , preferably, in step S4, the method of extracting features from the local area and performing multi-scale feature extraction based on the N' center points to obtain multi-scale features includes the following steps: Step S41. Normalize the points within each local area, encode each local area with the N' center points as the centers to obtain the first multi-scale feature; Step S42. For each center point, sample a multi-scale multi-level structure, construct n sampling areas, and extract features from each sampling area to obtain the second multi-scale feature.

[0052] After extracting features from the local area, the points in the local area can be transformed into relative features around the center point. Since the points in the 3D point cloud data are unevenly distributed, the features obtained from the dense area and the sparse area are very different. Therefore, it is also necessary to use different scales for grouping operations, perform feature extraction separately at the same time, and then stack these features to obtain multi-scale features. These multi-scale features contain global features and local features, providing basic support for subsequent belt deviation detection or coal heap detection.

[0053] S5. Use the PCL clustering algorithm to filter out the noise in the multi-scale features to obtain point cloud filtered data.

[0054] Specifically, the PCL clustering method is as follows: Find a point p11 in space, use the KD-Tree to find the n points closest to it, and judge the distances from these n points to p11. Place the points p12, P13, P14... whose distances are less than the value r into the class Q; Find a point p12 in Q(p11) and repeat step 1; Find a point in Q(P11, P12) and repeat step 1 to find P22, P23, P24...; When no new points are added to Q anymore, the search is completed.

[0055] Using the above method, the three-dimensional point cloud data after multiple abstractions can be effectively filtered, and the redundant three-dimensional point cloud data such as the belt conveyor frame can be filtered out, which is convenient for subsequent calls during belt deviation detection or coal stacking detection.

[0056] In summary, the embodiment of the present invention provides a three-dimensional point cloud processing method, which realizes the segmentation processing of the semantic information of the point cloud; Through the processing of noise points and the deep extraction of the local features of the point set, the segmentation result is more accurate and stable, and the fine-grained recognition ability and generalization ability of the point cloud are improved.

[0057] Embodiment 2

[0058] As Figure 5 shown, Figure 5 is a schematic flowchart of a belt deviation detection method based on three-dimensional point cloud provided by an embodiment of the present invention, including the following steps:

[0059] Step A1: Obtain the point cloud filtering data during the coal transportation of the belt conveyor based on the three-dimensional point cloud processing method, and divide the point cloud filtering data into multiple data segments;

[0060] Step A2: Use the maximum and minimum values of the Y-axis coordinates of the points in each data segment as the left and right edge bands of the belt, calculate the central value of each left and right edge band, calculate the average central value of each data segment according to the central value, and judge whether the belt of the belt conveyor is deviated according to the average central value.

[0061] Specifically, find the points with the largest and smallest Y-axis coordinates among the points with the same X coordinate, calculate the average value of the Y-axis coordinates of these two points to obtain the central value, calculate the average value of all the central values of a data segment to obtain the average central value. In the embodiment of the present invention, the sampling average central value is used to measure the position of a data segment (one end of the belt), compare the average central values of each data segment, and if the average central values of two adjacent data segments decrease slowly, it indicates that the belt has a left or right deviation.

[0062] In summary, the embodiment of the present invention provides a method for detecting belt deviation based on 3D point cloud. Through the 3D point cloud processing method, the point cloud filtering data during coal transportation can be accurately obtained, and then the accurate detection of belt deviation can be realized, which can timely discover belt deviation information and ensure the safety of coal mine operations.

[0063] Embodiment 2

[0064] As Figure 6 shown Figure 6 is a schematic flowchart of a method for detecting coal stacking based on 3D point cloud provided by an embodiment of the present invention. The method includes the following steps:

[0065] Step B1: Obtain the first point cloud filtering data during coal transportation by the belt conveyor and the first point cloud filtering data during no-load by the 3D point cloud processing method;

[0066] Step B2: Determine n detection points from the first point cloud filtering data and the second point cloud filtering data respectively, and calculate the average value of the Z-axis coordinates of the n detection points to obtain the first equivalent height and the second equivalent height;

[0067] Specifically, the formulas for calculating the first equivalent height and the second equivalent height are:

[0068]

[0069] In the formula, h f is the equivalent height, and Z q is the Z-axis coordinate of the qth detection point.

[0070] Step B3: Judge whether the belt of the belt conveyor has a coal stacking fault according to the difference between the first equivalent height and the second equivalent height.

[0071] The first equivalent height is the height during coal transportation, and the second equivalent height is the height during no-load. When the difference between the first equivalent height and the second equivalent height is greater than the preset coal stacking threshold, it is considered that a coal stacking rule occurs; otherwise, no coal stacking fault occurs.

[0072] In summary, the present invention provides a method for detecting coal stacking based on 3D point cloud. Through the 3D point cloud processing method, the point cloud filtering data during coal transportation and no-load can be accurately obtained, and then the accurate coal stacking height can be obtained, which can timely discover the coal stacking phenomenon during belt coal transportation and ensure the safety of coal mine operations.

[0073] Those skilled in the art will understand that although the preferred embodiments of the present invention have been described, additional changes and modifications can be made to these embodiments once the basic creative concepts are known to those skilled in the art. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the equivalent technologies of the claims of the present invention, the present invention is also intended to include these modifications and variations.

Claims

1. A three-dimensional point cloud processing method, characterized in that, It includes the following steps: S1. Obtain three-dimensional point cloud data; S2. Based on the three-dimensional point cloud data, select N' points through the iterative farthest point sampling algorithm to define N' central points of the local area; S3. Based on the N' central points, search for k adjacent points within the range r of each central point through the ball query algorithm to construct a local area; S4. Extract features from the local area, perform multi-scale feature extraction based on the N' central points to obtain multi-scale features; S5. Use the PCL clustering algorithm to filter out the noise in the multi-scale features to obtain point cloud filtered data; In step S1, the three-dimensional point cloud data is obtained by a lidar installed directly above the belt conveyor; In step S2, the method of selecting N' points through the iterative farthest point sampling algorithm based on the three-dimensional point cloud data to define N' central points of the local area includes the following steps: Step S21. Establish a rectangular coordinate system OXYZ of the three-dimensional point cloud data with the center of the equipment support of the belt conveyor as the origin O, the belt running direction of the belt conveyor as the X axis, the belt width direction of the belt conveyor as the Y axis, and the belt height direction of the belt conveyor as the Z axis; Step S22. Determine a sampling point A from the three-dimensional point cloud data, and determine a sampling point B that is the farthest from the sampling point A from the remaining points to obtain a sampling point set S = {A, B}; Step S23. Determine a sampling point P that is the farthest from the sampling point A set from the remaining points, and update the sampling point set; Step S24. Repeat step S23 to obtain N' points; In step S3, the method of searching for k adjacent points within the range r of each central point through the ball query algorithm based on the N' central points to construct a local area includes the following steps: Step S31. Set parameters r and k; Step S32. Respectively use the N' central points as the centers of the spheres, draw spheres with the radius r, and find k points within the spheres to construct a local area.

2. The 3D point cloud processing method according to claim 1, characterized in that Step S4. The method of extracting features from the local area and performing multi-scale feature extraction based on the N' central points to obtain multi-scale features includes the following steps: Step S41. Normalize the points in each local area, encode each local area with the N' central points as the centers to obtain the first multi-scale feature; Step S42. For each central point, sample a multi-scale multi-level structure, construct n sampling areas, and extract features from each sampling area to obtain the second multi-scale feature.

3. A belt deviation detection method based on 3D point cloud, characterized in that, It includes the following steps: Step A1. Obtain the point cloud filtered data during the coal transportation of the belt conveyor based on the three-dimensional point cloud processing method described in claim 1 or 2, and divide the point cloud filtered data into multiple data segments; Step A2. Use the maximum and minimum values of the Y-axis coordinates of the points in each data segment as the left and right edge bands of the belt, calculate the central value of each left and right edge band, calculate the average central value of each data segment according to the central value, and judge whether the belt of the belt conveyor is running off track according to the average central value.

4. A coal heap detection method based on 3D point cloud, characterized in that, It includes the following steps: Step B1: Obtain the first point cloud filtering data during coal transportation by the belt conveyor and the second point cloud filtering data during no-load operation based on the 3D point cloud processing method described in Claim 1 or 2; Step B2: Determine n detection points from the first point cloud filtering data and the second point cloud filtering data respectively, calculate the average value of the Z-axis coordinates of the n detection points, and obtain the first equivalent height and the second equivalent height; Step B3: Judge whether there is a coal piling fault in the belt of the belt conveyor according to the difference between the first equivalent height and the second equivalent height.

Images