A Detection Method for the Opening and Closing Angle of the Hydraulic Support Guard Plate of a LiDAR

Abstract

The present invention discloses a method for detecting the opening and closing angle of the hydraulic support rib protection plate of a lidar, which relates to the technical field of monitoring the working state of fully mechanized coal mining equipment, and includes: Step 1, obtaining the first trajectory of the robot's movement according to the data collected by the first lidar; Step 2, obtaining the first point cloud of the three-dimensional fully mechanized coal mining face by combining the data collected by the second lidar and the first trajectory; Step 3, performing the first screening and clustering on the first point cloud to obtain the second point cloud containing the point cloud of the rib protection plate; Step 4, performing the second screening on the second point cloud to obtain the point cloud clusters of the rib protection plates of each hydraulic support; Step 5, calculating each cluster of point clouds and performing the third screening to obtain the opening and closing angle of the rib protection plate. The present invention has high working efficiency, can ensure the safety of the acquisition work, reduce the influence of interference factors, improve the accuracy of detecting the opening and closing angle of the rib protection plate, and achieve the accurate detection of the opening and closing angle of the rib protection plate.

Description

Technical Field

[0001] The invention belongs to the technical field of monitoring the working state of fully-mechanized coal mining equipment, and particularly relates to a detection method for the opening and closing angle of the rib protection plate of a hydraulic support based on lidar. Background Art

[0002] As an important part of the hydraulic support, the rib protection plate is hinged with the rib protection jack. When the rib protection is required, the lower part of the rib protection plate is made to closely adhere to the coal wall by operating the rib protection jack. When actually working, when the rib spalling occurs in the front and the end beam distance is too large, the rib protection plate must be pushed out first, but the rib protection plate must be retracted before the shearer passes through; when the roof collapses or the end beam distance is too large, the rib protection plate can be flipped to support the roof above the coal wall to prevent rib spalling, which plays an important role in ensuring the normal operation of the shearer, scraper conveyor and personnel safety.

[0003] Traditionally, workers rely on visual inspection with the naked eye to check the state of the rib protection plate. However, this method has certain defects both in terms of accuracy and the personal safety guarantee of workers. Subsequently, the contact measurement method using sensors has problems such as the need for a large number of sensors, high failure rate, and difficult maintenance.

[0004] In addition, the patent "Method and device for measuring the telescopic angle of a rib protection plate based on an image sequence (CN114511799A)" discloses real-time monitoring and visual display of the telescopic angle of the rib protection plate through an image sequence; the patent "Method for real-time decision-making and control of fully-mechanized mining equipment based on lidar (CN114251092A)" discloses the identification of the rib protection plate area in the fused image through a preset target detection model to obtain a target image; however, the above methods for monitoring the state of the rib protection plate by collecting images are easily affected by light, resulting in low-quality images taken in the fully-mechanized coal mining face, thereby reducing the calculation accuracy; the patent "Method for real-time decision-making and control of fully-mechanized mining equipment based on lidar (CN114251092A)" discloses calculating the opening and closing state of the rib protection plate based on the positional relationship between the rib protection plate and the top beam scanned by two lidars. Although this method can realize the detection of the rib protection plate of a single support, it is difficult to extract all rib protections from multiple supports and cannot achieve full autonomous operation. Summary of the Invention

[0005] Aiming at the defects and problems existing in the existing detection methods for the opening and closing angle of the rib protection plate, such as low image quality and poor calculation accuracy due to being affected by light when collecting information through images; and most of them can only realize the detection of a single rib protection plate and it is difficult to extract all rib protections from multiple supports and cannot achieve full autonomous operation, the present invention provides a detection method for the opening and closing angle of the rib protection of a hydraulic support based on lidar.

[0006] The solution adopted by the present invention to solve its technical problems is as follows: A method for detecting the opening and closing angle of the hydraulic support rib protection based on lidar, comprising the following steps:

[0007] Step 1: Place a robot equipped with a first lidar and a second lidar under the hydraulic support for information collection work, and calculate the travel trajectory of the robot using the point cloud data within the horizontal range scanned by the first lidar;

[0008] Step 2: Scan the point cloud data within the vertical range through the second lidar, and calculate the complete point cloud of the coal mine fully mechanized mining face using the scanning result and combining with the travel trajectory of the robot in Step 1, and perform downsampling to obtain the first point cloud;

[0009] Step 3: Perform the first screening and the first clustering process on the first point cloud obtained in Step 2 to obtain a second point cloud containing the rib protection plate point cloud;

[0010] Step 4: Perform the second screening on the second point cloud obtained in Step 3 to obtain the point cloud clusters of each hydraulic support rib protection plate;

[0011] Step 5: Calculate the point cloud clusters obtained in Step 4 and perform the third screening to obtain the final opening and closing angle of the rib protection plate.

[0012] For the above method for detecting the opening and closing angle of the hydraulic support rib protection based on lidar, the steps for obtaining the travel trajectory of the robot in Step 1 include the following steps:

[0013] S101: Extract all corner points and plane points in the point cloud data of the previous and subsequent frames;

[0014] S102: Calculate the pose increment Δp through the registration and parameter optimization of the corner points and plane points;

[0015] S103: Iteratively update the pose, p(t) = p(t - 1)+Δp,

[0016] where p(t) = (x(t), y(t), z(t), rx(t), ry(t), rz(t)); use X(t) to represent its position: X(t) = (x(t), y(t), z(t)), and the first trajectory is denoted as {X(t)}.

[0017] For the above method for detecting the opening and closing angle of the hydraulic support rib protection based on lidar, the first screening in Step 3 includes: Select a height threshold, and screen out the data points in the first point cloud PC1 whose z-axis coordinates satisfy the height relationship to obtain PC2′, that is: PC2′ = {p|h0 < p z <

[0018] h1, p ∈ PC1};

[0019] Where h0 and h1 are the high and low thresholds of the first screening respectively, and the selection of h0 and h1 is related to the physical size of the rib protection plate. Let the z-axis coordinate of the centroid of the rib protection plate in the second lidar coordinate be M z , then the set value of h0 is 0.8M z , and the set value of h1 is 1.2M z .

[0020] In the above method for detecting the opening and closing angle of the rib protection of the hydraulic support based on lidar, the first clustering in step three includes: calculating the curvature of each point according to the plane formed by v points near the rib protection plate, and calculating the curvature change of the w points closest to a point in the space that meets the distance threshold range requirement. If it is less than the set curvature threshold, it can be classified into one category to obtain the second point cloud PC2.

[0021] In the above method for detecting the opening and closing angle of the rib protection of the hydraulic support based on lidar, the second screening of the second point cloud PC2 in step four includes setting the high and low thresholds of the theoretical point cloud quantity. If the number N i of all data points in the i-th cluster of point cloud C i meets the threshold range, it can be preliminarily determined as the rib protection plate to obtain the third point cloud PC3.

[0022] In the above method for detecting the opening and closing angle of the rib protection of the hydraulic support based on lidar, in step five, the opening and closing angle of the final rib protection plate is obtained by performing PCA decomposition on the data of the i-th cluster of point cloud in the third point cloud PC3 and then through the third screening, including the following steps:

[0023] S501. Arrange the x, y, and z-axis coordinates p jx , p jy , p jy of the data points in the i-th cluster of point cloud to obtain a matrix X i with 3 rows and N i columns, and zero-mean each row of X i ;

[0024] S502. Calculate the covariance matrix G i and find its eigenvalues and corresponding eigenvectors;

[0025] S503. Obtain the direction corresponding to the normal vector of the rib protection plate plane. The opening and closing angle of the rib protection plate is the angle between the rib protection plane and the horizontal plane, that is:

[0026]

[0027] In the above method for detecting the opening and closing angle of the rib protection of the hydraulic support based on lidar, in step S502, the covariance matrix G is calculatedi The methods for obtaining its eigenvalues and the corresponding eigenvectors are as follows:

[0028] (1) Define the matrix X i The average value of each row of data of

[0029] where X i (e, f) is the element in the e-th row and f-th column of the matrix X i ;

[0030] (2) Zero-mean normalization means that each element of the matrix X i is subtracted by the average value of the data in its row to obtain the matrix Y i , that is:

[0031]

[0032] Define the number of all elements in the matrix Y i as m i Then: m i = 3N i ;

[0033] And the covariance matrix G i is the covariance of the matrix Y i and its transpose, that is

[0034]

[0035] In the formula, Cov is the covariance operation, and y′ i is the transpose of the matrix Y i ;

[0036] (3) For the eigenvalues obtained by PCA decomposition of the i-th cluster of point cloud data, if the following requirements are met, it can be determined as the rib protection plate, and the third screening is completed, that is:

[0037] In the formula, is the eigenvalue obtained by PCA decomposition, and it is defined to satisfy t l , t h is the set threshold parameter; u is a parameter related to the physical size of the rib protection plate, and is defined as follows:

[0038] In the formula, l1 and l2 are the side lengths of the long side and the short side of the rib protection plate respectively.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides a method for detecting the opening and closing angle of the hydraulic support rib protection plate based on lidar. The lidar is used to collect relevant data of the underground fully mechanized coal mining face, and then point cloud filtering is used to obtain the point cloud data containing the rib protection plate. Whether it is a rib protection plate is judged by the number of points of the segmented rib protection plate and the flatness after PCA decomposition. Finally, the opening and closing angle of the rib protection plate is determined by the normal vector of the rib protection plate plane and the horizontal plane, so as to realize the detection of the opening and closing angle of the underground hydraulic support rib protection plate.

[0040] By collecting point cloud data with the lidar, the present invention can obtain the spatial information of the internal area of the hydraulic support in a short time, with high working efficiency, high accuracy of the obtained data, and can ensure the safety of the collection work. It can also avoid the problems of low image quality and poor calculation accuracy caused by being easily affected by light when collecting images through image acquisition. And based on the complete point cloud of the coal mine three-dimensional fully mechanized mining face, the present invention obtains the point cloud data containing only the rib protection plate through multiple screenings, reduces the influence of interference factors, and improves the accuracy of detecting the opening and closing angle of the rib protection plate. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is the overall flow chart of the method of the present invention;

[0042] Figure 2 is the specific flow chart of the method of the present invention;

[0043] Figure 3 is the three-dimensional structure diagram of the lidar installation position of the present invention;

[0044] Figure 4 is the first point cloud display diagram obtained in step two of the present invention;

[0045] Figure 5 is the second point cloud display diagram obtained in step three of the present invention;

[0046] Figure 6 is the point cloud cluster display diagram of the rib protection plate of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0047] The present invention will be further described below with reference to the drawings and embodiments.

[0048] Please refer to Figures 1-5 , the present invention provides a method for detecting the opening and closing angle of the hydraulic support rib protection plate based on lidar. The lidar is used to collect relevant data of the underground fully mechanized coal mining face, and then point cloud filtering is used to obtain the point cloud data containing the rib protection plate. Whether it is a rib protection plate is judged by the number of points of the segmented rib protection plate and the flatness after PCA decomposition. Finally, the opening and closing angle of the rib protection plate is determined by the normal vector of the rib protection plate plane and the horizontal plane, so as to realize the detection of the opening and closing angle of the underground hydraulic support rib protection plate.

[0049] Example 1:

[0050] This embodiment provides a method for detecting the opening and closing angle of the hydraulic support rib protection based on lidar, including the following steps:

[0051] Step 1: Place the robot equipped with the first lidar and the second lidar under the hydraulic support for information collection. The first lidar is installed above the robot and is used to scan the environmental information within the horizontal range (i.e., front, back, left, and right) under the hydraulic support. The second lidar is installed in front of the robot, and the scanning range is the environmental information within the vertical range (i.e., up, down, left, and right). Run the robot to collect data using the first lidar, and calculate the pose increment based on the front and back two frames of point cloud data scanned by it, so as to obtain the first trajectory of the robot's movement, including the following steps:

[0052] S101: Extract all corner points and plane points from the front and back two frames of point cloud data;

[0053] S102: Calculate the pose increment Δp through the registration and parameter optimization of the corner points and plane points;

[0054] S103: Iteratively update the pose, p(t) = p(t - 1)+Δp;

[0055] Among them, p(t)=(x(t), y(t), z(t), rx(t), ry(t), rz(t). Since only the position coordinates of the robot are required, X(t) is used to represent its position: X(t)=(x(t), y(t), z(t)), and the first trajectory is denoted as {X(t)}.

[0056] Step 2: Scan the point cloud data within the vertical range through the second lidar, and use the scanning result and combine it with the movement trajectory of the robot in Step 1 to calculate the complete point cloud of the coal mine three-dimensional fully mechanized mining face, and obtain the first point cloud through downsampling:

[0057] Since the second lidar is installed in front of the robot, the distance and angle relationship between the first lidar and the second lidar can be obtained through measurement, so that the change of the three-dimensional coordinate points in the coordinate system of the second lidar to the three-dimensional coordinate points in the coordinate system of the first lidar can be obtained, that is: Among them, is the rotation matrix for converting the coordinate system of the second lidar to the coordinate system of the first lidar, and and θ x , θ y , θ z are the rotation angles of the x, y, and z axes of the second lidar to the x, y, and z axes of the first lidar respectively; is the translation vector for converting the second lidar coordinate system to the first lidar coordinate system, and while t x , t y , t z are respectively the translation distances of the origin of the second lidar coordinate to the x, y, and z axes of the origin of the first lidar coordinate;

[0058] Then, the point cloud data scanned by the second lidar each time is spliced to obtain the complete point cloud trajectory PC of the three-dimensional fully-mechanized coal mining face all , that is,

[0059] After that, downsampling is performed. That is, by constructing a three-dimensional voxel grid, the centroid of all points within each voxel approximately represents other points in the voxel. In this way, all points within the voxel are represented by a centroid point, and downsampling is performed to achieve the filtering effect, which greatly reduces the data volume and can also improve the running speed of the program. In the process of downsampling in this embodiment, voxels with a side length of r meters are used, and the r value is set to 0.02, thereby obtaining the first point cloud PC1.

[0060] Step 3: Perform the first screening and the first clustering process on the first point cloud PC1 obtained in Step 2 to obtain the second point cloud containing the point cloud of the rib protection plate, including:

[0061] (1) First screening: Select the height threshold, and screen out the data points in the first point cloud PC1 whose z-axis coordinates satisfy the height relationship to obtain PC2′, that is: PC2′ = {p|h0 < p z < h1, p ∈ PC1};

[0062] where h0 and h1 are respectively the high and low thresholds of the first screening, and the selection of h0 and h1 is related to the physical size of the rib protection plate. Let the z-axis coordinate of the centroid of the rib protection plate in the second lidar coordinate be M z , then the set value of h0 is 0.8M z , and the set value of h1 is 1.2M z ;

[0063] (2) First clustering: Calculate the curvature of each point according to the plane formed by v points near the rib protection plate, and calculate the curvature change of the w points closest in space to a point within the range of the distance threshold requirement. If it is less than the set curvature threshold, it can be classified into one category to obtain the second point cloud PC2, that is:

[0064] where the distance threshold d treshold is set to 10 pixels, and the curvature change |Δn| tresholdSet the threshold value to 5°, the set value of the number of points v to 5, and the set value of the number of points w to 10.

[0065] Step 4: Perform a second screening on the second point cloud PC2 obtained in Step 3 to obtain the point cloud clusters of each hydraulic support rib protection plate, including:

[0066] Set the high and low thresholds of the theoretical point cloud quantity. If the number N i of all data points in the i-th cluster of point cloud C i meets the threshold range, it can be preliminarily determined as the rib protection plate to obtain the third point cloud PC3, that is:

[0067] PC3 = { { C i}| aβ < N i < bβ, C i ∈ PC2}, where β is the theoretical point cloud quantity, a and b are set threshold parameters, the set value of a is 0.8, the set value of b is 1.2, and β is related to the physical size of the rib protection plate, that is: And in the formula, S is the physical area of the working surface of the rib protection plate, r is the side length of the voxel in downsampling, and r = 0.02m.

[0068] Step 5: Perform PCA decomposition on the data of the i-th cluster of point cloud in the third point cloud PC3 obtained in Step 4, and then perform a third screening to obtain the opening and closing angle of the final rib protection plate, including the following steps:

[0069] S501: Arrange the x, y, and z axis coordinates p jx , p jy , p jy of the data points in the i-th cluster of point cloud according to the following method to obtain a 3-row N i -column matrix X i , and zero-mean each row of X i , that is:

[0070]

[0071] S502: Calculate the covariance matrix G i and find its eigenvalues and their corresponding eigenvectors, including:

[0072] (1) Define the average value of each row of data of matrix X i as

[0073] where X i (e, f) is the element in the e-th row and f-th column of matrix X i ;

[0074] (2) Zero-mean is matrix X iSubtract the average value of the data in each row from each element of the matrix to obtain matrix Y i , that is:

[0075]

[0076] Define matrix Y i Let the number of all elements in be m i Then: m i = 3N i ;

[0077] And the covariance matrix G i is the covariance of matrix Y i and its transpose, that is

[0078]

[0079] In the formula, Cov is the covariance operation, y′ i is the transpose of matrix Y i ;

[0080] (3) For the eigenvalues obtained by performing PCA decomposition on the i-th cluster of point cloud data, if the following requirements are met, it can be determined as the guard plate, and the third screening is completed, that is:

[0081] In the formula, is the eigenvalue obtained by PCA decomposition, and it is defined to satisfy t l , t h is the set threshold parameter; u is a parameter related to the physical size of the guard plate, and is defined as follows:

[0082] In the formula, l1 and l2 are the side lengths of the long side and the short side of the guard plate respectively;

[0083] S503. Obtain the direction corresponding to the normal vector of the guard plate plane. The opening and closing angle of the guard plate is the angle between the guard plate plane and the horizontal plane, that is:

[0084]

[0085] In the formula, v3 is the eigenvector corresponding to the eigenvalue λ3 obtained by PCA decomposition, is the length of the vector v3.

[0086] The above are only the preferred embodiments of the present invention, which do not limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A detection method for the opening and closing angle of the hydraulic support rib protection plate based on lidar, characterized in that: It includes the following steps: Step 1: Place the robot equipped with the first lidar and the second lidar below the inside of the hydraulic support frame for information collection work, and calculate the first travel trajectory of the robot using the point cloud data scanned by the first lidar within the horizontal range; Step 2: Scan the point cloud data within the vertical range through the second lidar, and calculate the complete point cloud of the coal mine three-dimensional fully mechanized mining face using the scanning result and combining the first travel trajectory of the robot in Step 1, and perform downsampling to obtain the first point cloud; Step 3: Perform the first screening and the first clustering process on the first point cloud obtained in Step 2 to obtain the second point cloud containing the point cloud of the rib protection plate; The first screening includes: Select a height threshold, and screen out the data points in the first point cloud PC1 whose z-axis coordinates satisfy the height relationship to obtain PC′2, that is: PC′2 = {p | h0 < p z < h1, p ∈ PC1}; Where: h0 and h1 are respectively the high and low thresholds of the first screening, and the selection of h0 and h1 is related to the physical size of the side guard plate. Let the z-axis coordinate of the centroid of the side guard plate in the second lidar coordinate system be M z , then the set value of h0 is 0.8Mz, and the set value of h1 is 1.2Mz; The first clustering includes: Calculate the curvature of each point according to the plane formed by v points near the rib protection plate, calculate the curvature change of the w points closest in distance within the space where a point satisfies the distance threshold range requirement, and if it is less than the set curvature threshold, it can be classified into one category to obtain the second point cloud PC2; Step 4: Perform the second screening on the second point cloud obtained in Step 3 to obtain the point cloud clusters of the rib protection plates of each hydraulic support frame; Step 5: Calculate the point cloud clusters obtained in Step 4 and perform the third screening to obtain the final opening and closing angle of the rib protection plate.

2. The detection method for the opening and closing angle of the hydraulic support rib protection plate based on lidar according to claim 1, characterized in that: Obtaining the travel trajectory of the robot in Step 1 includes the following steps: S101: Extract all corner points and plane points in two consecutive frames of point cloud data; S102: Calculate the pose increment Δp through the registration and parameter optimization of the corner points and plane points; S103: Iteratively update the pose, p(t) = p(t - 1)+Δp, where p(t) = (x(t), y(t), z(t), rx(t), ry(t), rz(t)); Use X(t) to represent the robot position: X(t) = (x(t), y(t), z(t)), and the first trajectory is denoted as {X(t)}.

3. The detection method for the opening and closing angle of the hydraulic support rib protection plate based on lidar according to claim 1, wherein: In the fourth step, the second screening of the second point cloud PC2 includes setting high and low thresholds for the theoretical number of point clouds. If the number N i of all data points in the i-th cluster of point clouds C i meets the threshold range, it can be preliminarily determined as the rib protection plate, and the third point cloud PC3 is obtained.

4. The detection method for the opening and closing angle of the hydraulic support rib protection plate based on lidar according to claim 1, wherein: In Step 5, the final opening and closing angle of the rib protection plate is obtained by performing PCA decomposition on the i-th cluster of point cloud data in the third point cloud PC3 and then performing the third screening, including the following steps: S501, the x, y, z axis coordinates p of the data points in the i-th cluster point cloud jx , p jy , p jz By arranging, we get 3 rows N i The column matrix X i , and X i Each row of is zero-meaned; S502. Calculate the covariance matrix G i and find its eigenvalues and the corresponding eigenvectors; S503: Obtain the direction corresponding to the normal vector of the rib protection plate plane. The opening and closing angle of the rib protection plate is the angle between the rib protection plane and the horizontal plane, that is: Where: v3 is the eigenvector corresponding to the eigenvalue λ3 decomposed by PCA, is the length of the vector v3.

5. The detection method for the opening and closing angle of the hydraulic support rib protection plate based on lidar according to claim 4, characterized in that: The S502 calculates the covariance matrix G i and finds its eigenvalues and the corresponding eigenvectors, including the following methods: (1) Define the matrix X i The average value of each row of data where X i (e, f) is the element in the e-th row and f-th column of the matrix X i ; (2) Zero-mean normalization means subtracting the average value of each row of matrix X i from each of its elements to obtain matrix Y i , that is: Define matrix Y i The number of all elements in i is m i Then: m i ; while the covariance matrix G i is the covariance of matrix Y i and its transpose, i.e.: where Cov is the covariance operation, and Y i ′ is the transpose of matrix Y i ; (3) For the eigenvalues obtained by performing PCA decomposition on the i-th cluster of point cloud data, if the following requirements are met, it can be determined as the rib protection plate to complete the third screening, that is: In the formula, is the eigenvalue obtained by PCA decomposition, and is defined to satisfy t1, t h is the set threshold parameter; u is a parameter related to the physical size of the rib protection plate and is defined as follows: In the formula, l1 and l2 are the side lengths of the long side and the short side of the rib protection plate respectively.

Images