Three-dimensional point cloud-based deviation detection system and method for underground long-distance belt conveyor
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- SHENHUA SHENDONG COAL GRP
- Filing Date
- 2025-07-16
- Publication Date
- 2026-06-04
Smart Images
Figure CN2025108762_04062026_PF_FP_ABST
Abstract
Description
A System and Method for Detecting Misalignment of Long-Distance Belt Conveyors in Underground Mining Based on 3D Point Clouds Technical Field
[0001] This disclosure relates to the field of coal mine monitoring technology, and in particular to a system and method for detecting misalignment of long-distance underground belt conveyors based on three-dimensional point clouds. Background Technology
[0002] In the coal industry, especially in underground coal mining, belt conveyors are widely used for transportation. Belt misalignment is the most common problem encountered during belt conveyor operation. Severe issues can lead to damage to the production equipment due to intense friction between the belt and the frame, with potentially disastrous consequences. This is particularly prevalent in long-distance underground belt conveyors. Therefore, it is necessary to analyze and study the causes of belt conveyor misalignment and implement corresponding measures to improve its reliable and safe operation, reduce wear and tear on equipment and components, extend its service life, and increase its transport capacity. Summary of the Invention
[0003] This disclosure aims to at least partially address one of the technical problems in the related art.
[0004] Therefore, the first objective of this disclosure is to propose a method for detecting misalignment of long-distance underground belt conveyors based on three-dimensional point clouds, so as to achieve high accuracy in detecting misalignment of long-distance underground belt conveyors.
[0005] The second objective of this disclosure is to propose a misalignment detection system for long-distance underground belt conveyors based on three-dimensional point clouds.
[0006] The third objective of this disclosure is to propose a misalignment detection system for long-distance underground belt conveyors based on three-dimensional point clouds.
[0007] The fourth objective of this disclosure is to provide a computer-readable storage medium.
[0008] The fifth objective of this disclosure is to provide a computer program product.
[0009] To achieve the above objectives, the first aspect of this disclosure proposes a method for detecting misalignment of a long-distance underground belt conveyor based on three-dimensional point clouds, comprising:
[0010] The current frame contour point cloud data of the cross-section of the belt of a long-distance underground belt conveyor is acquired using camera equipment.
[0011] Determine the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section;
[0012] If the included angle exceeds the included angle threshold, a secondary determination of belt deviation is made based on the current frame contour point cloud data to obtain the belt deviation detection result.
[0013] Optionally, before determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section, the method further includes:
[0014] The initial position of the belt cross-section is determined, and the reference frame contour point cloud data of the belt cross-section is obtained by calibrating the initial position using a camera device.
[0015] Optionally, before obtaining the reference frame contour point cloud data of the belt cross-section by calibrating the initial position using a camera device, the method further includes:
[0016] The camera parameters of the camera device are set so that the reference frame contour point cloud data meets the requirements for belt misalignment detection. The camera parameters include region of interest, brightness, exposure rate, and frame rate.
[0017] Optionally, determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section includes:
[0018] Data preprocessing is performed on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data to obtain preprocessed reference frame contour point cloud data and preprocessed current frame contour point cloud data.
[0019] Determine the angle between the preprocessed reference frame contour point cloud data, the camera device, and the preprocessed current frame contour point cloud data.
[0020] Optionally, the data preprocessing of the reference frame contour point cloud data of the belt cross-section and the current frame contour point cloud data includes at least one of the following:
[0021] The reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data are subjected to pass-through filtering.
[0022] Statistical outlier removal processing is performed on the reference frame contour point cloud data of the belt cross-section and the current frame contour point cloud data.
[0023] Optionally, the step of performing pass-through filtering on the reference frame contour point cloud data of the belt cross-section and the current frame contour point cloud data includes:
[0024] Determine the filter coordinate axes and the corresponding filter data ranges;
[0025] Based on the filtering data range corresponding to the filtering coordinate axis, pass-through filtering is performed on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data.
[0026] Optionally, the statistical outlier removal process for the reference frame contour point cloud data of the belt cross-section and the current frame contour point cloud data includes:
[0027] Determine the average distance between each data point in the point cloud data and the set of K neighboring points corresponding to the data point, wherein the point cloud data is the reference frame contour point cloud data of the belt cross section or the current frame contour point cloud data.
[0028] Determine the standard deviation between the average distances of all data points in the point cloud data;
[0029] The confidence level range is determined based on the standard deviation and the average distance, and data points in the point cloud data whose confidence level is outside the confidence level range are removed.
[0030] Optionally, determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section includes:
[0031] Based on Euclidean distance, determine the left belt edge point and the right belt edge point in the current frame of the current frame contour point cloud data;
[0032] Based on Euclidean distance, determine the left and right edge points of the belt in the reference frame contour point cloud data of the belt cross-section.
[0033] Determine the first included angle between the left edge point of the belt in the current frame, the camera device, and the left edge point of the belt in the reference frame;
[0034] Determine the second included angle between the right edge point of the belt in the current frame, the camera device, and the right edge point of the belt in the reference frame.
[0035] Optionally, the step of performing a secondary determination of belt misalignment based on the current frame contour point cloud data to obtain the belt misalignment detection result includes:
[0036] If the left frame contour point cloud data in the current frame contour point cloud data is located to the left of the belt edge point in the reference frame, and the right frame contour point cloud data in the current frame contour point cloud data is located to the right of the belt edge point in the reference frame, then the initial belt deviation detection result corresponding to the current frame contour point cloud data is determined to be belt deviation.
[0037] If the initial belt misalignment detection result is that the number of consecutive belt misalignments is greater than the misalignment number threshold, then the belt misalignment detection result for the underground long-distance belt conveyor is determined to be belt misalignment.
[0038] To achieve the above objectives, a second aspect of this disclosure provides a system for detecting misalignment of a long-distance underground belt conveyor based on three-dimensional point clouds, comprising:
[0039] The data acquisition module is used to acquire the current frame contour point cloud data of the cross-section of the belt of a long-distance underground belt conveyor through camera equipment;
[0040] The back-end processing unit is used to determine the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section;
[0041] The back-end processing unit is further configured to perform a secondary determination of belt misalignment based on the current frame contour point cloud data if the included angle exceeds the included angle threshold, and obtain the belt misalignment detection result.
[0042] To achieve the above objectives, a third aspect of this disclosure provides a long-distance downhole belt conveyor deviation detection system based on three-dimensional point clouds, comprising: a processor, and a memory communicatively connected to the processor;
[0043] The memory stores the instructions that the computer executes;
[0044] The processor executes computer execution instructions stored in memory to implement the method shown in any of the first aspects above.
[0045] To achieve the above objectives, a fourth aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method shown in any of the first aspects above.
[0046] To achieve the above objectives, a fifth aspect of this disclosure provides a computer program product including a computer program that, when executed by a processor, implements the method shown in any of the first aspects above.
[0047] In summary, the method, system, and storage medium provided in this disclosure acquire current frame contour point cloud data of the cross-section of a belt conveyor in a long-distance underground mine using a camera device; determine the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section; if the angle exceeds a threshold, a secondary determination of belt misalignment is performed based on the current frame contour point cloud data to obtain the belt misalignment detection result. Therefore, belt misalignment detection of long-distance underground belt conveyors can be achieved with high accuracy.
[0048] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description
[0049] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:
[0050] Figure 1 is a flowchart illustrating a method for detecting misalignment of a long-distance underground belt conveyor based on three-dimensional point clouds, provided in an embodiment of this disclosure.
[0051] Figure 2 is a schematic diagram of the operation of a camera device provided in an embodiment of this disclosure;
[0052] Figure 3 is a schematic diagram of the delineation of an ROI region provided in an embodiment of this disclosure;
[0053] Figure 4 is an imaging schematic diagram of a single frame of ordered three-dimensional point cloud data obtained based on the ROI region provided in an embodiment of this disclosure;
[0054] Figure 5 is a schematic diagram of the included angle of belt misalignment provided in an embodiment of this disclosure;
[0055] Figure 6 is a schematic diagram of the cross-section of a belt conveyor under different coal carrying capacities provided in an embodiment of this disclosure;
[0056] Figure 7 is a schematic diagram of a belt connection object and its corresponding point cloud imaging provided in an embodiment of this disclosure;
[0057] Figure 8 is a schematic diagram of a long-distance underground belt conveyor deviation detection system based on three-dimensional point cloud provided in an embodiment of this disclosure. Detailed Implementation
[0058] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.
[0059] It should be noted that in the entire underground coal conveying system, based on 6 belt conveyors with a total belt length of 6000 meters, the estimated annual replacement cost due to belt wear and misalignment is 2000 meters, totaling 800,000 to 1 million yuan. Since belt misalignment affects normal production, assuming a 0.5-hour impact per day and a coal output of 300 tons per hour, at 300 yuan per ton, the estimated loss in revenue over 13 million yuan per year (300 working days) is significant. In other words, preventing belt misalignment not only improves safety but also yields substantial economic benefits.
[0060] Among related technologies, the main detection techniques for belt misalignment include:
[0061] Contact detection mainly uses contact sensors for detection, such as vertical roller devices, and the amount of deflection of the device is used to detect whether the belt is running off-track.
[0062] Infrared detection involves evenly distributing several infrared generators and receivers on both sides of the belt. When belt misalignment occurs, the belt will block some of the infrared beams, and the displacement of the misalignment is determined based on the beam width information.
[0063] Machine vision inspection uses two-dimensional images and machine learning algorithms to detect the edges of belts and determine their deviation.
[0064] However, due to the complex underground environment, traditional contact and infrared deviation detection methods cannot meet the requirements of coal mine safety production in terms of durability, sensitivity, and reliability. On the other hand, the detection effect of image processing methods is greatly affected by light, lacks depth information, and the accuracy of the algorithm is insufficient, resulting in a high false detection rate and a high false negative rate.
[0065] The present disclosure will now be described in detail with reference to specific embodiments.
[0066] In the first embodiment, as shown in Figure 1, which is a flowchart illustrating a method for detecting misalignment of a long-distance underground belt conveyor based on three-dimensional point clouds provided in this disclosure, the method can be implemented using a computer program and can run on a system for detecting misalignment of a long-distance underground belt conveyor based on three-dimensional point clouds. This computer program can be integrated into an application or run as a standalone utility application.
[0067] The method for detecting misalignment of long-distance underground belt conveyors based on three-dimensional point clouds can be executed by a system for detecting misalignment of long-distance underground belt conveyors based on three-dimensional point clouds.
[0068] For example, the method for detecting misalignment of a long-distance underground belt conveyor based on 3D point clouds includes the following steps:
[0069] S101, acquires the current frame contour point cloud data of the cross-section of the belt of the long-distance belt conveyor in the mine through camera equipment;
[0070] According to some embodiments, a camera device refers to a device used to acquire three-dimensional point cloud data.
[0071] In some embodiments, the current frame contour point cloud data refers to the single-frame ordered 3D contour point cloud data acquired by the camera device at the current moment.
[0072] It should be noted that the 3D point cloud data includes point cloud data along the coordinate axes (X, Y, Z). However, since the X-axis has the same value in all single-frame ordered 3D point cloud data and is unrelated to edge detection, the data in the X-axis direction can be discarded, and the remaining data is the single-frame ordered 3D contour point cloud data.
[0073] S102, determine the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section;
[0074] According to some embodiments, the reference frame contour point cloud data refers to the initial single-frame ordered three-dimensional point cloud data of the belt cross-section.
[0075] S103, if the included angle exceeds the included angle threshold, then perform a secondary determination of belt deviation based on the current frame contour point cloud data to obtain the belt deviation detection result.
[0076] In some embodiments, the included angle threshold does not specifically refer to a fixed threshold. The included angle threshold can be determined, for example, based on the actual application scenario.
[0077] In some embodiments, belt misalignment detection results are used to indicate whether the belt of a long-distance underground belt conveyor has misaligned.
[0078] In summary, the method provided in this embodiment acquires the current frame contour point cloud data of the cross-section of the belt of a long-distance underground belt conveyor using a camera device; determines the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section; if the angle exceeds a threshold, a secondary belt misalignment determination is performed based on the current frame contour point cloud data to obtain the belt misalignment detection result. Therefore, it can realize the misalignment detection of long-distance underground belt conveyors with high accuracy.
[0079] This embodiment also provides another method for detecting belt misalignment in long-distance underground belt conveyors based on three-dimensional point clouds. This method can be executed by a three-dimensional point cloud-based system for detecting belt misalignment in long-distance underground belt conveyors.
[0080] For example, the method for detecting misalignment of a long-distance underground belt conveyor based on 3D point clouds may include the following steps:
[0081] S201, determine the initial position of the belt cross-section, and obtain the reference frame contour point cloud data of the belt cross-section by calibrating the initial position using camera equipment;
[0082] According to some embodiments, the camera device can be, for example, a binocular laser camera. Binocular laser cameras can still acquire high-precision point cloud data in dim environments and retain the three-dimensional shape information of the conveyor belt. Compared with image processing methods, they have better anti-interference performance and higher robustness, making them more suitable for the complex environment of coal mines. When obtaining the reference frame contour point cloud data of the conveyor belt cross-section by calibrating the initial position using a binocular laser camera, this data can be obtained through a camera software development kit (SDK).
[0083] In some embodiments, the camera device can be equipped with an explosion-proof housing to improve the safety of the camera device during use.
[0084] In some embodiments, FIG2 is a schematic diagram of the operation of a camera device provided in this disclosure. As shown in FIG2, when using the installed camera device to acquire laser line contour point cloud data of the belt cross-section for belt deviation detection, the camera device can be installed above the belt, specifically on a gantry or fixed to the top of the aisle above the belt, and the camera device takes pictures of the belt vertically downward. The camera device can be installed at a position greater than 1 meter above the center line of the belt, and the camera is installed horizontally.
[0085] According to some embodiments, before obtaining the reference frame contour point cloud data of the belt cross-section by calibrating the initial position using a camera device, the camera parameters of the camera device can be set so that the reference frame contour point cloud data meets the belt deviation detection requirements.
[0086] In some embodiments, camera parameters include, but are not limited to, region of interest (ROI), brightness, exposure rate, and frame rate.
[0087] Since the light absorption of the belt material and the overall dimness of the scene vary, adjusting the brightness and exposure rate of the laser line can enable the camera equipment to adapt to various scenes and obtain accurate belt contour point cloud data.
[0088] The frame rate refers to the number of point cloud laser lines acquired per second (each laser line has approximately 2000 points, which represent the entire monitoring area). The frame rate affects the final distance between each line. For example, it is preferable to set it to 500 frames per second to achieve a balance between timeliness and accuracy.
[0089] Since the entire laser line area of the camera device includes not only the entire belt area but also other areas of no interest, defining the ROI area can ensure that the acquired point cloud data only includes point clouds of a portion of the non-belt area. The width of the retained portion of the non-belt area only needs to cover the range of the belt deviation movement area.
[0090] As an example, Figure 3 is a schematic diagram of the delineation of a Region of Interest (ROI) provided in an embodiment of this disclosure. As shown in Figure 3, the binocular laser camera includes a left-eye camera and a right-eye camera. Figure 3(a) shows the ROI area corresponding to the left-eye camera, and Figure 3(b) shows the ROI area corresponding to the right-eye camera. Only the side railings on both sides of the belt and the entire belt area are retained as ROI areas.
[0091] It should be noted that when performing belt edge detection, the camera can be installed at any position above the belt (vertically and at an angle) while ensuring a complete ROI area is obtained. This is to consider situations where: 1) coal flow data exists inside the belt; and 2) the belt is unloaded. Furthermore, in practical applications, the camera installation needs to be flexible according to the actual scenario. Due to the influence of the on-site environment, the installation of the camera may not meet the standard requirements, and there may be some offset or tilt compared to the vertical downward installation method directly above the belt. Positional offsets can be corrected through calibration and algorithm optimization.
[0092] S202, acquire the current frame contour point cloud data of the cross-section of the belt of the long-distance belt conveyor in the mine through camera equipment;
[0093] It should be noted that after the baseline frame contour point cloud data is acquired, the position and camera parameters of the camera device remain fixed, and the acquisition of the current frame contour point cloud data continues.
[0094] For example, in one scenario, the initial position of the belt can be recorded by a binocular laser camera, and the initial edge position of the belt can be calibrated as a reference frame. Then, the binocular laser camera can be used to detect the three-dimensional coordinate information of the belt edge in real time.
[0095] S203, perform data preprocessing on the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross section to obtain the preprocessed reference frame contour point cloud data and the preprocessed current frame contour point cloud data.
[0096] It should be noted that due to factors such as lighting and reflective materials on the belt itself, even after adjusting exposure and brightness settings, camera equipment will still exhibit some noise (outliers); or the belt may shift, resulting in point cloud data with large distances (which can be referred to as noise). Therefore, data preprocessing, i.e., noise removal, is necessary before processing the point cloud data to ensure the accuracy of belt misalignment detection.
[0097] In some embodiments, when preprocessing the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross-section, at least one of the following processing methods can be used:
[0098] The reference frame contour point cloud data and the current frame contour point cloud data of the belt cross section are subjected to pass-through filtering.
[0099] Statistical outlier removal processing is performed on the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross-section.
[0100] According to some embodiments, pass-through filtering can filter the original point cloud data (including the reference frame contour point cloud data and the current frame contour point cloud data) according to the specified coordinate axes (X,Y,Z) and the corresponding coordinate range, and select whether to retain point cloud data within the set coordinate range or point cloud data outside the coordinate range according to actual needs, thereby achieving the filtering of corresponding noise data.
[0101] In some embodiments, when performing pass-through filtering on the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross-section, the filter coordinate axis and the filter data range corresponding to the filter coordinate axis can be determined; and the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross-section are subjected to pass-through filtering based on the filter data range corresponding to the filter coordinate axis.
[0102] Taking a scenario as an example, when performing pass-through filtering on the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross-section, the Z-axis data can be filtered to retain the point cloud data within the range of the closest and farthest distances between the installed camera and the belt. That is, the filtering coordinate axis is set to the Z-axis, the Z-axis filtering data range is specified, the point cloud data within the Z-axis filtering data range is selected, and the reference frame contour point cloud data and the current frame contour point cloud data after pass-through filtering are output.
[0103] According to some embodiments, statistical outlier removal refers to determining whether a point is noisy data by measuring the average distance and variance from a specified point to data points in a selected neighborhood. The threshold for determining outliers is dynamically changing.
[0104] In some embodiments, when performing statistical outlier removal processing on the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross-section, the following steps may be adopted:
[0105] S2031, determine the average distance between each data point in the point cloud data and the set of K neighboring points corresponding to the data point, wherein the point cloud data is the reference frame contour point cloud data of the belt cross section or the current frame contour point cloud data.
[0106] According to some embodiments, the K-Nearest Neighbor (KNN) classification algorithm can be used to determine the set of K-neighbor points corresponding to each data point in point cloud data.
[0107] In some embodiments, the distance between each data point and each K-neighbor point in the set of K-neighbor points corresponding to that data point is d. ij (i∈[1,…,m],j∈[1,…,k]), where k is the number of K-neighbor points in the K-neighbor set, m is the number of data points in the point cloud data, and d ij Let be the distance between the i-th data point in the point cloud data and its corresponding j-th K-neighbor point.
[0108] In some embodiments, the average distance d between each data point and its corresponding K-neighborhood set is:
[0109]
[0110] S2032, determine the standard deviation between the average distances of all data points in the point cloud data;
[0111] According to some embodiments, due to d ij It can be considered as following a Gaussian distribution with standard deviation σ and expected value μ. Therefore, the standard deviation between the mean distances of all data points in the point cloud data can be determined according to the following formula:
[0112]
[0113] S2033, determine the confidence range based on the standard deviation and average distance, and remove data points in the point cloud data whose confidence level is outside the confidence range.
[0114] In some embodiments, the confidence level range can be (dc*σ, d+c*σ), where c can be determined according to the actual application scenario, for example, c can be 3.
[0115] S204, determine the angle between the preprocessed reference frame contour point cloud data, the camera device, and the preprocessed current frame contour point cloud data;
[0116] It should be noted that Figure 4 is an imaging schematic diagram of a single-frame ordered 3D point cloud data obtained based on the ROI region provided in this embodiment of the present disclosure. As shown in Figure 4, the distances between the edge points P1 and P2 on both sides of the conveyor belt and the adjacent endpoints P0 and P3 of the side frame railings on both sides of the conveyor belt are significantly different from the distances between other adjacent points. Therefore, the conveyor belt edge detection can be performed based on the Euclidean distance between two adjacent points. Furthermore, in order to accurately locate the edge points on both sides of the conveyor belt even when there is coal flow in the belt, conveyor belt edge detection can be performed on M points before and after each frame of ordered point cloud data, thereby obtaining points P1 and P2 in the single-frame ordered point cloud data.
[0117] In some embodiments, the Euclidean distance between two adjacent points can be determined using the following formula:
[0118]
[0119] Where N is the number of points in a single frame of ordered 3D point cloud data; M is the number of points selected for belt edge detection. The value of M can be determined according to the actual application scenario. For example, M can be 200.
[0120] Among them, dis Pi The i-th point (y) adjacent to the left i , z i ) and the (i-1)th point (y i-1 , z i-1 The Euclidean distance between ) and dis Pj For the j-th point adjacent to the right (y j , z j ) and the (j+1)th point (y j+1 , z j+1 The Euclidean distance between the points is given by y, where y represents the coordinates of the point cloud on the Y-axis and z represents the coordinates of the point cloud on the Z-axis.
[0121] It should be noted that when dis Pi When the distance is greater than the distance threshold dis, the point is the left edge point of the belt, P1; similarly, when dis Pj When the distance exceeds the threshold dis, the point is designated as point P2 on the right edge of the belt. The distance threshold dis is not a fixed threshold; it can be adjusted based on the specific application scenario. For example, the distance threshold dis could be 20mm.
[0122] In other words, the left edge point P of the belt in the current frame can be determined based on the Euclidean distance in the current frame contour point cloud data. L Point P on the right edge of the belt in the current frame R Based on Euclidean distance, determine the left edge point P of the belt in the reference frame of the reference frame contour point cloud data for the belt cross-section.L_0 Point P on the right edge of the belt in the reference frame R_0 Determine the left edge point P of the belt in the current frame. L Camera equipment and the left edge point P of the reference frame belt L_0 The first included angle α1 between them; determine the right edge point P of the belt in the current frame. R Camera equipment and the right edge point P of the reference frame belt. R_0 The second included angle α2 between them.
[0123] In some embodiments, Figure 5 is a schematic diagram of the included angle of belt misalignment provided in this disclosure. As shown in Figure 5, the position of the camera device can be set as the origin P. src (0,0,0).
[0124] In some embodiments, the first included angle α1 and the second included angle α2 can be determined according to the following formula:
[0125]
[0126] Among them, P L_0y P represents L_0 The coordinate value on the Y-axis; P L_0z P represents L_0 The coordinate value on the Z-axis; P R_0y P represents R_0 The coordinate value on the Y-axis; P R_0z P represents R_0 The coordinate value on the Z-axis; P Ly P represents L The coordinate value on the Y-axis; P Lz P represents L The coordinate value on the Z-axis; P Ry P represents R The coordinate value on the Y-axis; P Rz P represents R The coordinate value on the Z-axis; P srcy P represents src The coordinate value on the Y-axis; P srcz P represents src The coordinate value on the Z-axis; |·| represents the distance between the corresponding two coordinate points.
[0127] S205, if the included angle exceeds the included angle threshold, then a secondary determination of belt deviation is made based on the current frame contour point cloud data to obtain the belt deviation detection result.
[0128] According to some embodiments, a first included angle threshold can be set for the first included angle α1, and a second included angle threshold can be set for the second included angle α2. If the first included angle α1 exceeds the first included angle threshold and the second included angle α2 exceeds the second included angle threshold, a secondary determination of belt misalignment can be made based on the current frame contour point cloud data. Alternatively, if the first included angle α1 exceeds the first included angle threshold or the second included angle α2 exceeds the second included angle threshold, a secondary determination of belt misalignment can be made based on the current frame contour point cloud data.
[0129] The first included angle threshold and the second included angle threshold can be the same or different, and can be determined according to the actual application scenario.
[0130] It should be noted that Figure 6 is a schematic diagram of the cross-section of a belt conveyor under different coal carrying capacities provided in the embodiments of this disclosure. As shown in Figure 6, the overall profile of the belt conveyor will change under different coal carrying capacities. The positions of the side frame railings on both sides of the belt will not change under different carrying conditions, while the belt as a whole will be concave downward as the coal carrying capacity increases. That is, the overall point cloud of the belt will increase the overall Z-axis value and decrease the overall belt width as the coal carrying capacity increases.
[0131] Additionally, Figure 7 is a schematic diagram of a belt connection object and its corresponding point cloud imaging provided in an embodiment of this disclosure. As shown in Figure 7, under real-world conditions, reflective materials may be present at the belt, causing point cloud noise in the camera's imaging process.
[0132] Therefore, to address the impact of the above two situations on belt misalignment detection, during the secondary determination of belt misalignment, firstly, considering the changes in belt cross-section for different coal loads, if the left frame contour point cloud data in the current frame contour point cloud data is located to the left of the belt edge point in the reference frame, and the right frame contour point cloud data in the current frame contour point cloud data is located to the right of the belt edge point in the reference frame, then the initial belt misalignment detection result corresponding to the current frame contour point cloud data is determined to be belt misalignment; regarding the point cloud noise interference from reflective materials, if the number of consecutive belt misalignments in the initial belt misalignment detection result is greater than the misalignment number threshold, then the belt misalignment detection result corresponding to the underground long-distance belt conveyor is determined to be belt misalignment.
[0133] The deviation count threshold is not a fixed threshold; it can be determined based on the actual application scenario. For example, deviation detection can be performed every 10 frames. When the number of consecutive records indicating belt deviation reaches 8, it is determined that the belt deviation detection result for the underground long-distance belt conveyor has occurred, and a relevant alarm signal is output.
[0134] In summary, the method provided in this embodiment determines the initial position of the belt cross-section and obtains the reference frame contour point cloud data of the belt cross-section by calibrating the initial position using a camera device; therefore, it can improve the accuracy of acquiring the reference frame contour point cloud data. Next, the current frame contour point cloud data of the belt cross-section of the underground long-distance belt conveyor is acquired using a camera device; the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross-section are preprocessed to obtain preprocessed reference frame contour point cloud data and preprocessed current frame contour point cloud data; the angle between the preprocessed reference frame contour point cloud data, the camera device, and the preprocessed current frame contour point cloud data is determined; therefore, the accuracy of angle determination can be improved. Then, if the angle exceeds an angle threshold, a secondary belt misalignment determination is performed based on the current frame contour point cloud data to obtain the belt misalignment detection result; therefore, it can realize the misalignment detection of underground long-distance belt conveyors with high accuracy.
[0135] To achieve the above embodiments, this disclosure also proposes a long-distance underground belt conveyor deviation detection system based on three-dimensional point clouds.
[0136] For example, this downhole long-distance belt conveyor deviation detection system based on 3D point cloud includes:
[0137] The data acquisition module is used to acquire the current frame contour point cloud data of the cross-section of the belt of a long-distance underground belt conveyor through camera equipment;
[0138] The back-end processing unit is used to determine the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section;
[0139] The back-end processing unit is also used to perform a secondary determination of belt deviation based on the current frame contour point cloud data if the included angle exceeds the included angle threshold, so as to obtain the belt deviation detection result.
[0140] It should be noted that the foregoing explanation of the embodiment of the method for detecting misalignment of long-distance underground belt conveyors based on three-dimensional point clouds also applies to the system for detecting misalignment of long-distance underground belt conveyors based on three-dimensional point clouds in this embodiment, and will not be repeated here.
[0141] The camera device in the data acquisition module can collect data at a fixed frequency and send it to the back-end processing unit. The camera device is only responsible for setting necessary camera parameters, Internet Protocol (IP) settings, and output frequency settings.
[0142] The back-end processing unit can be composed of edge computing devices to complete algorithm processing and linkage control signal output. For example, it can be based on an RK1126 CPU, supplemented by analog input / output modules and a 24V DC input / output module. Internally, it can port relevant algorithms for detecting misalignment of long-distance underground belt conveyors based on 3D point clouds, as well as an interactive interface, to complete three basic functions: data acquisition from camera equipment, algorithm processing (see the "Method for Detecting Misalignment of Long-Distance Underground Belt Conveyors Based on 3D Point Clouds" for specific algorithm processing), and algorithm result expression. The algorithm result expression includes its own relay output and analog output, while also being compatible with common network communication outputs such as Modbus TCP, Transmission Control Protocol (TCP), and databases.
[0143] As an example, Figure 8 is a schematic diagram of a long-distance underground belt conveyor deviation detection system based on three-dimensional point cloud provided in an embodiment of this disclosure. As shown in Figure 8, the system includes a data acquisition module (including camera equipment), a back-end processing unit, a control unit, an underground ring network, a ground server, a display, a power supply, a belt, and alarm lights.
[0144] After the camera equipment collects data, it can transmit the data to the back-end processing unit via network cable. When the back-end processing unit detects that the deviation exceeds a given threshold (the threshold can be flexibly configured according to the site conditions, such as setting an angle threshold; if the left / right angle exceeds 2°, it is considered a deviation warning; if the left / right deviation exceeds 4°, it is considered a serious deviation and the unit will intervene to stop the machine), it will report the alarm signal to the ground server platform to remind the monitoring room personnel to handle it in real time. At the same time, it will output an alarm signal through the IO board to control the audible and visual alarm lights to trigger an on-site alarm, and transmit the control signal to the control unit through the Modbus TCP protocol for shutdown linkage processing.
[0145] The control unit can be, for example, a programmable logic controller (PLC). The PLC can receive analog signals from the back-end processing unit for linkage control or alarm output. Linkage control includes belt start and stop control. For example, when the back-end processing unit detects that the belt deviation exceeds a given threshold, it can output an analog control signal to the PLC, which then controls the belt to stop.
[0146] The ground server can store historical belt speed, deviation data, etc., and can be set for shift, month, year, etc. to generate reports. Through data analysis, it can continuously optimize and adjust algorithm parameters and control strategies.
[0147] The camera equipment, back-end processing unit, control unit, and alarm lights can be powered by 12V or 24V power.
[0148] In summary, the system provided in this disclosure has millimeter-level accuracy in detecting belt misalignment and can effectively reduce the impact of factors such as uneven installation position and belt shape changes on misalignment detection. It has higher detection accuracy, sensitivity, and adaptability, faster misalignment detection processing speed, good system stability, high fault tolerance, and is not affected by electromagnetic interference. The embedded algorithm is ported to support edge computing, which can significantly reduce the computing pressure on the central server. It can still maintain normal operation when problems occur in the underground ring network, and the system has high stability.
[0149] To implement the above embodiments, this disclosure also proposes a long-distance downhole belt conveyor deviation detection system based on three-dimensional point clouds, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided in the foregoing embodiments.
[0150] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.
[0151] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.
[0152] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0153] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.
[0154] This disclosure is intended to provide implementation schemes for users to selectively prevent the use or access to their personal information data. Specifically, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.
[0155] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0156] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0157] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.
[0158] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.
[0159] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0160] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.
[0161] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0162] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.
Claims
1. A method for detecting misalignment of a long distance underground belt conveyor based on a three-dimensional point cloud, characterized in that, include: The current frame contour point cloud data of the cross-section of the belt of a long-distance underground belt conveyor is acquired using camera equipment. Determine the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section; If the included angle exceeds the included angle threshold, a secondary determination of belt deviation is made based on the current frame contour point cloud data to obtain the belt deviation detection result.
2. The method of claim 1, wherein, Before determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section, the method further includes: The initial position of the belt cross-section is determined, and the reference frame contour point cloud data of the belt cross-section is obtained by calibrating the initial position using a camera device.
3. The method of claim 2, wherein, Before obtaining the reference frame contour point cloud data of the belt cross-section by calibrating the initial position using a camera device, the method further includes: The camera parameters of the camera device are set so that the reference frame contour point cloud data meets the requirements for belt misalignment detection. The camera parameters include region of interest, brightness, exposure rate, and frame rate.
4. The method of claim 1, wherein, Determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section includes: Data preprocessing is performed on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data to obtain preprocessed reference frame contour point cloud data and preprocessed current frame contour point cloud data. Determine the angle between the preprocessed reference frame contour point cloud data, the camera device, and the preprocessed current frame contour point cloud data.
5. The method of claim 4, wherein, The data preprocessing of the reference frame contour point cloud data of the belt cross-section and the current frame contour point cloud data includes at least one of the following: The reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data are subjected to pass-through filtering. Statistical outlier removal processing is performed on the reference frame contour point cloud data of the belt cross-section and the current frame contour point cloud data.
6. The method of claim 5, wherein, The process of performing pass-through filtering on the reference frame contour point cloud data of the belt cross-section and the current frame contour point cloud data includes: Determine the filter coordinate axes and the corresponding filter data ranges; Based on the filtering data range corresponding to the filtering coordinate axis, pass-through filtering is performed on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data.
7. The method of claim 5, wherein, The statistical outlier removal process for the reference frame contour point cloud data of the belt cross-section and the current frame contour point cloud data includes: Determine the average distance between each data point in the point cloud data and the set of K neighboring points corresponding to the data point, wherein the point cloud data is the reference frame contour point cloud data of the belt cross section or the current frame contour point cloud data; Determine the standard deviation between the average distances of all data points in the point cloud data; The confidence range is determined based on the standard deviation and the average distance, and data points in the point cloud data whose confidence levels are outside the confidence range are removed.
8. The method of claim 1, wherein, Determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section includes: Based on Euclidean distance, determine the left belt edge point and the right belt edge point in the current frame of the current frame contour point cloud data; Based on Euclidean distance, determine the left and right edge points of the belt in the reference frame contour point cloud data of the belt cross-section. Determine the first included angle between the left edge point of the belt in the current frame, the camera device, and the left edge point of the belt in the reference frame; Determine the second included angle between the right edge point of the belt in the current frame, the camera device, and the right edge point of the belt in the reference frame.
9. The method of claim 1, wherein, The secondary determination of belt misalignment based on the current frame contour point cloud data to obtain the belt misalignment detection result includes: If the left frame contour point cloud data in the current frame contour point cloud data is located to the left of the belt edge point in the reference frame, and the right frame contour point cloud data in the current frame contour point cloud data is located to the right of the belt edge point in the reference frame, then the initial belt deviation detection result corresponding to the current frame contour point cloud data is determined to be belt deviation. If the initial belt misalignment detection result is that the number of consecutive belt misalignments is greater than the misalignment number threshold, then the belt misalignment detection result for the underground long-distance belt conveyor is determined to be belt misalignment.
10. A three-dimensional point cloud based long distance underground belt conveyor misalignment detection system, characterized in that, include: The data acquisition module is used to acquire the current frame contour point cloud data of the cross-section of the belt of a long-distance underground belt conveyor through camera equipment; The back-end processing unit is used to determine the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross-section; The back-end processing unit is further configured to perform a secondary determination of belt misalignment based on the current frame contour point cloud data if the included angle exceeds the included angle threshold, and obtain the belt misalignment detection result.