Underground long-distance belt conveyor deviation detection system and method based on three-dimensional point cloud

Abstract

The invention relates to the technical field of coal mine monitoring, in particular to an underground long-distance belt conveyor deviation detection system and method based on three-dimensional point cloud. The method comprises the following steps: acquiring current frame contour point cloud data of a belt cross section of an underground long-distance belt conveyor through camera equipment; determining an included angle among the current frame contour point cloud data, the camera equipment and the reference frame contour point cloud data of the cross section of the belt; if the included angle exceeds the included angle threshold value, belt deviation secondary judgment is conducted based on the current frame contour point cloud data, and a belt deviation detection result is obtained. By the adoption of the scheme, deviation detection of the underground long-distance belt conveyor can be achieved, and the deviation detection accuracy is high.

Description

Technical Field

[0001] The present disclosure relates to the technical field of coal mine monitoring, and in particular to a system and method for detecting deviation of an underground long-distance belt conveyor based on three-dimensional point cloud. Background Art

[0002] In the coal industry, especially in underground coal mining, belt conveyors are mostly used in the transportation process. Belt deviation is the most common phenomenon encountered during the operation of belt conveyors. If the problem is serious, the intense friction between the belt and the frame will cause damage to the production equipment, and the consequences are disastrous. Especially for long-distance belt conveyors underground, deviation is more common. Therefore, it is necessary to analyze and study the causes of belt conveyor deviation and take corresponding measures to improve the reliability of its safe operation, while reducing the loss of equipment and accessories, increasing the service life of the equipment, and increasing its transportation capacity. Summary of the invention

[0003] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0004] To this end, the first purpose of the present invention is to propose a method for detecting deviation of an underground long-distance belt conveyor based on three-dimensional point cloud, so as to realize deviation detection of an underground long-distance belt conveyor with high deviation detection accuracy.

[0005] The second objective of the present disclosure is to provide an underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud.

[0006] The third objective of the present disclosure is to provide an underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud.

[0007] A fourth objective of the present disclosure is to provide a computer-readable storage medium.

[0008] A fifth object of the present disclosure is to provide a computer program product.

[0009] To achieve the above-mentioned purpose, the first embodiment of the present disclosure proposes a method for detecting deviation of an underground long-distance belt conveyor based on a three-dimensional point cloud, comprising:

[0010] Acquire the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through the camera device;

[0011] Determining an angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section;

[0012] If the angle exceeds the angle threshold, a secondary belt deviation determination is performed based on the current frame contour point cloud data to obtain a belt deviation detection result.

[0013] Optionally, before determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section, the method further comprises:

[0014] The initial position of the belt cross section is determined, and the initial position is calibrated by a camera device to obtain the reference frame contour point cloud data of the belt cross section.

[0015] Optionally, before calibrating the initial position by a camera device to obtain reference frame contour point cloud data of the belt cross section, the method further includes:

[0016] The camera parameters of the camera device are set so that the reference frame contour point cloud data meets the belt deviation detection requirements, wherein the camera parameters include an area of ​​interest, brightness, exposure rate and frame rate.

[0017] Optionally, determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section includes:

[0018] Performing data preprocessing on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data to obtain preprocessed reference frame contour point cloud data and preprocessed current frame contour point cloud data;

[0019] An angle between the preprocessed reference frame contour point cloud data, the camera device, and the preprocessed current frame contour point cloud data is determined.

[0020] Optionally, the data preprocessing of the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data includes at least one of the following:

[0021] Performing through-filter processing on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data;

[0022] Statistical outlier removal processing is performed on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data.

[0023] Optionally, the performing through filtering on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data comprises:

[0024] Determine a filter coordinate axis and a filter data range corresponding to the filter coordinate axis;

[0025] According to the filtering data range corresponding to the filtering coordinate axis, the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data are subjected to through-filtering processing.

[0026] Optionally, the performing statistical outlier removal processing on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data comprises:

[0027] Determine an average distance between each data point in the point cloud data and a set of K-neighborhood points corresponding to the data point, wherein the point cloud data is the reference frame contour point cloud data of the belt cross section or the current frame contour point cloud data;

[0028] Determining the standard deviation between the average distances corresponding to all data points in the point cloud data;

[0029] A confidence range is determined according to the standard deviation and the average distance, and data points in the point cloud data whose confidences are outside the confidence range are removed.

[0030] Optionally, determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section includes:

[0031] Based on the Euclidean distance, determine the belt edge point on the left side of the current frame and the belt edge point on the right side of the current frame in the contour point cloud data of the current frame;

[0032] Based on the Euclidean distance, determining the belt edge point on the left side of the reference frame and the belt edge point on the right side of the reference frame in the reference frame contour point cloud data of the belt cross section;

[0033] Determine a first angle between the left belt edge point of the current frame, the camera device, and the left belt edge point of the reference frame;

[0034] A second angle between the right belt edge point of the current frame, the camera device, and the right belt edge point of the reference frame is determined.

[0035] Optionally, performing a secondary determination of belt deviation based on the current frame contour point cloud data to obtain a belt deviation detection result includes:

[0036] If the left frame contour point cloud data in the current frame contour point cloud data is located on the left side of the reference frame belt edge point, and the right frame contour point cloud data in the current frame contour point cloud data is located on the right side of the reference frame belt edge point, then the initial belt deviation detection result corresponding to the current frame contour point cloud data is determined to be that the belt has deviated;

[0037] If the initial belt deviation detection result is that the consecutive number of belt deviations is greater than the deviation number threshold, then the belt deviation detection result corresponding to the underground long-distance belt conveyor is determined to be that the belt has deviated.

[0038] To achieve the above-mentioned purpose, the second embodiment of the present disclosure proposes an underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud, comprising:

[0039] A data acquisition module, used for acquiring the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through a camera device;

[0040] A back-end processing unit, used to determine an angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section;

[0041] The back-end processing unit is further used to perform a secondary belt deviation determination based on the current frame contour point cloud data if the angle exceeds an angle threshold, so as to obtain a belt deviation detection result.

[0042] To achieve the above-mentioned purpose, the third aspect of the present disclosure provides an underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud, including: a processor, and a memory connected to the processor in communication;

[0043] Memory stores computer-executable instructions;

[0044] The processor executes the computer-executable instructions stored in the memory to implement the method shown in any one of the first aspects above.

[0045] To achieve the above-mentioned objectives, an embodiment of the fourth aspect of the present disclosure proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method shown in any one of the above-mentioned first aspects.

[0046] To achieve the above-mentioned objective, a fifth aspect of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the method shown in any one of the above-mentioned first aspects.

[0047] In summary, the method, system and storage medium provided by the present disclosure obtain the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through the camera device; determine the angle between the current frame contour point cloud data, the camera device and the reference frame contour point cloud data of the belt cross section; if the angle exceeds the angle threshold, perform a secondary belt deviation judgment based on the current frame contour point cloud data to obtain the belt deviation detection result. Therefore, the deviation detection of the underground long-distance belt conveyor can be realized, and the deviation detection accuracy is high.

[0048] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0050] Figure 1 A schematic diagram of a flow chart of a method for detecting deviation of an underground long-distance belt conveyor based on a three-dimensional point cloud provided in an embodiment of the present disclosure;

[0051] Figure 2 A working schematic diagram of a camera device provided by an embodiment of the present disclosure;

[0052] Figure 3 A schematic diagram of delineating a ROI area provided in an embodiment of the present disclosure;

[0053] Figure 4 A schematic diagram of imaging of a single frame of ordered three-dimensional point cloud data acquired based on a ROI region provided in an embodiment of the present disclosure;

[0054] Figure 5 A schematic diagram of an angle of belt deviation provided by an embodiment of the present disclosure;

[0055] Figure 6 A schematic diagram of a belt cross section under different coal carrying capacities provided by an embodiment of the present disclosure;

[0056] Figure 7 A schematic diagram of a belt-connected object and corresponding point cloud imaging provided by an embodiment of the present disclosure;

[0057] Figure 8 A schematic structural diagram of an underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0059] It should be noted that in the entire underground coal transportation system, according to the 6 belt conveyors with a belt length of 6,000 meters, it is estimated that 2,000 meters of belts need to be replaced every year due to deviation and wear, with a total cost of 800,000 to 1 million yuan. The deviation affects normal production, assuming an impact of 0.5 hours per day, 300 tons of coal per hour, and 300 yuan per ton. It is estimated that the income will be reduced by more than 13 million yuan in 300 working days a year. In other words, not deviating the belt is not only conducive to safe production, but also can achieve greater economic benefits.

[0060] Among the related technologies, the detection technologies for belt deviation mainly include:

[0061] Contact detection mainly uses contact sensors for detection, such as a vertical roller device, and the deflection amount of the device is used to detect whether the belt is deviating.

[0062] Infrared detection, a number of infrared generators and receivers are evenly arranged on both sides of the belt. When deviation occurs, the belt will block some infrared beams, and the displacement of the deviation is determined based on the light speed width information.

[0063] Machine vision detection, through the collection of two-dimensional images and the use of machine learning related algorithms, can detect the edge of the belt and then determine the deviation status of the belt.

[0064] However, due to the complex environment underground, traditional contact and infrared deviation detection methods cannot meet the safety production requirements of coal mines in terms of durability, sensitivity, and reliability. The detection effect based on image processing methods is greatly affected by light and has no depth information. The algorithm is not accurate enough, and the false detection rate and missed detection rate are high.

[0065] The present disclosure is described in detail below with reference to specific embodiments.

[0066] In the first embodiment, if Figure 1 As shown, Figure 1 This is a flow chart of a method for detecting deviation of a long-distance belt conveyor in a well based on a three-dimensional point cloud provided by an embodiment of the present disclosure. The method can be implemented by a computer program and can be run on a system for detecting deviation of a long-distance belt conveyor in a well based on a three-dimensional point cloud. The computer program can be integrated into an application or run as an independent tool application.

[0067] Among them, the underground long-distance belt conveyor deviation detection method based on three-dimensional point cloud can be executed by the underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud.

[0068] For example, the method for detecting deviation of an underground long-distance belt conveyor based on three-dimensional point cloud includes the following steps:

[0069] S101, obtaining the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through a camera device;

[0070] According to some embodiments, a camera device refers to a device for acquiring three-dimensional point cloud data.

[0071] In some embodiments, the current frame contour point cloud data refers to a single frame of ordered three-dimensional contour point cloud data acquired by the camera device at the current moment.

[0072] It should be noted that the three-dimensional point cloud data includes point cloud data of the coordinate axes (X, Y, Z). However, since the X-axis has the same value in a single-frame ordered three-dimensional point cloud data and is irrelevant to edge detection, the data in the X-axis direction can be discarded, and the remaining data is the single-frame ordered three-dimensional contour point cloud data.

[0073] S102, determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section;

[0074] According to some embodiments, the reference frame contour point cloud data refers to an initial single frame ordered three-dimensional point cloud data of the belt cross section.

[0075] S103: If the angle exceeds the angle threshold, a secondary belt deviation determination is performed based on the current frame contour point cloud data to obtain a belt deviation detection result.

[0076] According to some embodiments, the angle threshold does not specifically refer to a fixed threshold, and the angle threshold may be determined, for example, according to an actual application scenario.

[0077] In some embodiments, the belt deviation detection result is used to indicate whether the belt of the underground long-distance belt conveyor is deviating.

[0078] In summary, the method provided in this embodiment obtains the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through the camera device; determines the angle between the current frame contour point cloud data, the camera device and the reference frame contour point cloud data of the belt cross section; if the angle exceeds the angle threshold, a secondary belt deviation determination is performed based on the current frame contour point cloud data to obtain a belt deviation detection result. Therefore, the deviation detection of the underground long-distance belt conveyor can be realized, and the deviation detection accuracy is high.

[0079] This embodiment also provides another method for detecting deviation of an underground long-distance belt conveyor based on three-dimensional point cloud. The method can be executed by a system for detecting deviation of an underground long-distance belt conveyor based on three-dimensional point cloud.

[0080] For example, the method for detecting deviation of an underground long-distance belt conveyor based on a three-dimensional point cloud may include the following steps:

[0081] S201, determining an initial position of a belt cross section, and calibrating the initial position through a camera device to obtain reference frame contour point cloud data of the belt cross section;

[0082] According to some embodiments, the camera device may be, for example, a binocular line laser camera, which can still collect high-precision point cloud data in a dim environment and retain the three-dimensional shape information of the belt. Compared with image processing methods, it has better anti-interference and higher robustness, and is more suitable for the complex environment of coal mines. When the initial position is calibrated by the binocular line laser camera to obtain the reference frame contour point cloud data of the belt cross section, it can be obtained through the camera software development kit (SDK).

[0083] In some embodiments, the camera device may be provided with an explosion-proof housing to improve the safety of the camera device when in use.

[0084] In some embodiments, Figure 2 The following is a schematic diagram of the operation of a camera device provided by an embodiment of the present disclosure. Figure 2 As shown, the laser line profile point cloud data of the belt cross section is obtained by using the installed camera device. When performing deviation detection, the camera device can be installed above the belt, specifically, it can be installed on the gantry or fixed on the top of the lane above the belt, and the camera device shoots the belt vertically downward. The camera device can be installed at a range greater than 1 meter above the center line of the belt, and the camera is installed horizontally.

[0085] According to some embodiments, before calibrating the initial position through the camera device to obtain the reference frame contour point cloud data of the belt cross section, the camera parameters of the camera device can also be set so that the reference frame contour point cloud data meets the belt deviation detection requirements.

[0086] In some embodiments, camera parameters include, but are not limited to, region of interest (ROI), brightness, exposure rate, and frame rate.

[0087] Among them, since the belt material absorbs light and the overall scene has different levels of darkness, adjusting the brightness and exposure rate of the laser line can make the camera equipment adapt to various scenes and obtain accurate belt contour point cloud data.

[0088] The frame rate refers to how many point cloud laser lines are acquired in one second (each laser line has about 2,000 points, representing the entire monitoring area). The frame rate affects the distance between each line. For example, it can be preferably set to 500 frames during setting, so as to achieve a balance between timeliness and accuracy.

[0089] Among them, since the entire laser line area of ​​the camera device not only includes the entire belt area, but also includes other areas of no interest, therefore, by defining the ROI area, it can be ensured that the acquired point cloud data only contains part of the non-belt area point cloud, and the width of the retained part of the non-belt area only needs to ensure the range of the belt deviation movement area.

[0090] Take a scenario as an example. Figure 3 A schematic diagram of defining a ROI region provided in an embodiment of the present disclosure. Figure 3 As shown, the binocular line laser camera includes a left camera and a right camera. Figure 3 (a) in the figure shows the ROI area of ​​the left camera corresponding to the left camera in the ROI area. Figure 3 (b) in the figure shows the right camera ROI area corresponding to the right camera in the ROI area, which only retains the side rack rails on the left and right sides of the belt and the entire belt area as the ROI area.

[0091] It should be noted that when detecting the belt edge, the camera device can be installed at any position above the belt (vertically and tilted), but the complete ROI area can be obtained. The following situations should be considered: first, there is coal flow data inside the belt; second, the belt is unloaded. In addition, in practical applications, the camera device needs to be installed flexibly according to the actual scene. Due to the influence of the on-site environment, the installation of the camera device may not meet the standard requirements. Compared with the vertical downward installation method directly above the belt, there may be a certain offset and tilt. The position offset can be corrected by starting calibration and optimizing the algorithm.

[0092] S202, obtaining the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through a camera device;

[0093] It should be noted that after the reference frame contour point cloud data is acquired, the position of the camera device and the camera parameters remain fixed, and the acquisition of the current frame contour point cloud data continues.

[0094] Taking a scenario as an example, the initial position of the belt can be recorded by a binocular line laser camera, and the initial edge position of the belt can be calibrated as a reference frame. The binocular line laser camera can then be used to detect the three-dimensional coordinate information of the edge of the belt in real time.

[0095] S203, performing data preprocessing on the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross section to obtain preprocessed reference frame contour point cloud data and preprocessed current frame contour point cloud data;

[0096] It should be noted that due to the influence of lighting, reflective materials on the belt itself, etc., even if the camera equipment is adjusted by setting relevant exposure, brightness, etc., there will still be certain noise (outliers); or point cloud data with a large distance generated in the moving area after the belt deviates (herein referred to as noise). Therefore, data preprocessing, i.e. data noise removal, is required before processing the point cloud data to ensure the accuracy of belt deviation detection.

[0097] In some embodiments, when preprocessing the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross section, at least one of the following processing methods may be used:

[0098] Performing through-filter processing on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data;

[0099] Statistical outlier removal processing is performed on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data.

[0100] According to some embodiments, through filtering can filter the original point cloud data (including the baseline frame contour point cloud data and the current frame contour point cloud data) according to the specified coordinate axes (X, Y, Z) and the corresponding coordinate range, and select whether the retained point cloud is the point cloud within the set coordinate range or the point cloud data outside the coordinate range according to actual needs, thereby achieving filtering of the corresponding noise data.

[0101] In some embodiments, when the baseline frame contour point cloud data of the belt cross section and the current frame contour point cloud data are subjected to through-filtering processing, the filter coordinate axis and the filter data range corresponding to the filter coordinate axis can be determined; according to the filter data range corresponding to the filter coordinate axis, the baseline frame contour point cloud data of the belt cross section and the current frame contour point cloud data are subjected to through-filtering processing.

[0102] Taking a scenario as an example, when performing through-filtering processing on the baseline frame contour point cloud data and the current frame contour point cloud data of the belt cross section, the Z-axis data can be filtered to retain the point cloud data within the interval between the closest distance and the farthest distance between the installed camera and the belt. That is, the filtering coordinate axis is set to the Z-axis, and the Z-axis filtering data range is specified. The point cloud data within the Z-axis filtering data range is set to be retained, and the baseline frame contour point cloud data after through-filtering processing and the current frame contour point cloud data after through-filtering processing are output.

[0103] According to some embodiments, statistical outlier removal refers to determining whether a point is noise data by comparing the average distance and variance between a specified point and data points in a selected neighborhood, and the threshold for determining outliers changes dynamically.

[0104] In some embodiments, when performing statistical outlier removal processing on the reference frame contour point cloud data and the current frame contour point cloud data of the belt cross section, the following steps may be used:

[0105] S2031, determining an average distance between each data point in the point cloud data and a set of K-neighborhood points corresponding to the data point, wherein the point cloud data is a reference frame contour point cloud data or a current frame contour point cloud data of a belt cross section;

[0106] According to some embodiments, a K-Nearest Neighbor (KNN) classification algorithm may be used on point cloud data to determine a set of K-neighborhood points corresponding to each data point.

[0107] In some embodiments, the distance between each data point and each K neighboring point in the K neighboring point set corresponding to the data point is d ij (i∈[1,…,m], j∈[1,…,k]), where k is the number of K-neighborhood points in the K-neighborhood point set, m is the number of data points in the point cloud data, and d ij is the distance between the i-th data point in the point cloud data and its corresponding j-th K-neighborhood point.

[0108] In some embodiments, the average distance d between each data point and the set of K neighboring points corresponding to the data point is

[0109]

[0110] S2032, determining the standard deviation between the average distances corresponding to all data points in the point cloud data;

[0111] According to some embodiments, due to d ij It can be regarded as a Gaussian distribution with standard deviation σ and expected value μ. Therefore, the standard deviation between the average distances corresponding to all data points in the point cloud data can be determined according to the following formula:

[0112]

[0113] S2033, determining a confidence range according to the standard deviation and the average distance, and removing data points in the point cloud data whose confidences are outside the confidence range.

[0114] In some embodiments, the confidence range may be, for example, (dc*σ, d+c*σ), where c may be determined according to an actual application scenario, for example, c may be 3.

[0115] S204, determining an angle between the preprocessed reference frame contour point cloud data, the camera device, and the preprocessed current frame contour point cloud data;

[0116] It should be noted that Figure 4 The following is a schematic diagram of imaging of a single frame of ordered three-dimensional point cloud data obtained based on the ROI region provided by the embodiment of the present disclosure. Figure 4 As shown in the figure, the distance between the edge points P1 and P2 on both sides of the belt and the adjacent end points P0 and P3 of the side frame railings on both sides of the belt is much different from the distance between other adjacent points. Therefore, the belt edge detection can be performed based on the Euclidean distance between the two adjacent points. At the same time, in order to accurately find the edge points on both sides of the belt when there is coal flow in the belt, the belt edge detection can be performed on the M points before and after based on the single-frame ordered point cloud data, so that the P1 and P2 points in the single-frame ordered point cloud data can be obtained.

[0117] In some embodiments, the Euclidean distance between two adjacent points may be determined using the following formula:

[0118]

[0119] Among them, N is the number of point clouds in a single frame of ordered three-dimensional point cloud data; M is the number of point clouds selected for belt edge detection, and the value of M can be determined according to the actual application scenario. For example, M can be 200.

[0120] Among them, dis Pi is the i-th point adjacent to the left (y i , z i ) and the i-1th point (y i-1 , z i-1 ), dis Pj is the jth point adjacent to the right (y j , z j ) and the j+1th point (y j+1 , z j+1 ), where y represents the coordinate value of the point cloud on the Y axis and z represents the coordinate value of the point cloud on the Z axis.

[0121] It should be noted that when dis Pi When the distance is greater than the threshold value dis, the point is the left belt edge point P1; similarly, when dis Pj When the distance threshold value dis is greater than the distance threshold value, the point is the right belt edge point P2. The distance threshold value dis does not specifically refer to a fixed threshold value, and the distance threshold value dis can be adjusted according to the actual application scenario. For example, the distance threshold value dis can be 20 mm.

[0122] That is to say, the belt edge point P on the left side of the current frame in the contour point cloud data of the current frame can be determined based on the Euclidean distance. L 、The belt edge point P on the right side of the current frame R ; Based on the Euclidean distance, determine the belt edge point P on the left side of the reference frame in the reference frame contour point cloud data of the belt cross section L_0 , the right belt edge point P of the reference frame R_0 ; Determine the belt edge point P on the left side of the current frame L , camera device and the belt edge point P on the left side of the reference frame L_0 The first angle α1 between them; determine the belt edge point P on the right side of the current frame R , camera device and the right belt edge point P of the reference frame R_0 The second included angle α2 between them.

[0123] In some embodiments, Figure 5 Schematic diagram of the angle of belt deviation provided by the embodiment of the present disclosure. Figure 5 As shown, the position of the camera device can be set as the origin P src (0,0,0).

[0124] In some embodiments, the first angle α1 and the second angle α2 may be determined according to the following formula:

[0125]

[0126] Among them, P L_0y Indicates P L_0 Coordinate value on the Y axis; P L_0z Indicates P L_0 Coordinate value on the Z axis; P R_0y Indicates P R_0 Coordinate value on the Y axis; P R_0z Indicates P R_0 Coordinate value on the Z axis; P Ly Indicates P L Coordinate value on the Y axis; P Lz Indicates P L Coordinate value on the Z axis; P Ry Indicates P R Coordinate value on the Y axis; P Rz Indicates P R Coordinate value on the Z axis; P srcy Indicates P src Coordinate value on the Y axis; P srcz Indicates P src The coordinate value on the Z axis; |·| represents the distance between the corresponding two coordinate points.

[0127] S205: If the angle exceeds the angle threshold, a secondary belt deviation determination is performed based on the current frame contour point cloud data to obtain a belt deviation detection result.

[0128] According to some embodiments, a first angle threshold can be set for the first angle α1, and a second angle threshold can be set for the second angle α2. When the first angle α1 exceeds the first angle threshold and the second angle α2 exceeds the second angle threshold, a secondary belt deviation judgment can be made based on the current frame contour point cloud data. Alternatively, when the first angle α1 exceeds the first angle threshold and the second angle α2 exceeds the second angle threshold, a secondary belt deviation judgment can be made based on the current frame contour point cloud data.

[0129] The first angle threshold and the second angle threshold may be the same or different, and may be specifically determined according to actual application scenarios.

[0130] It should be noted that Figure 6 The cross-sectional diagram of a belt under different coal carrying capacities provided by the embodiment of the present disclosure is shown in FIG. Figure 6 As shown in the figure, the overall profile of the belt conveyor will change when the coal carrying amount is different. The position of the side frame railings on both sides of the belt will not change under different carrying conditions, and the belt as a whole will sink downward as the coal carrying amount increases. That is, the overall point cloud of the belt will increase the overall Z-axis value as the coal carrying amount increases, and the belt width will decrease as a whole.

[0131] in addition, Figure 7 This is a schematic diagram of a belt connection object and corresponding point cloud imaging provided by an embodiment of the present disclosure. Figure 7 As shown, under real working conditions, there may be reflective materials on the belt, causing the camera device to have related point cloud noise when imaging.

[0132] Therefore, in order to solve the influence of the above two situations on the belt deviation detection, when performing the secondary judgment of the belt deviation, first, for the changes in the cross-section of the belt with different coal carrying capacities, if the left frame contour point cloud data in the current frame contour point cloud data is located on the left side of the reference frame belt edge point, and the right frame contour point cloud data in the current frame contour point cloud data is located on the right side of the reference frame belt edge point, then the initial belt deviation detection result corresponding to the current frame contour point cloud data is determined to be the belt deviation; for the point cloud noise interference of reflective materials, if the initial belt deviation detection result is that the number of consecutive belt deviations is greater than the deviation number threshold, then the belt deviation detection result corresponding to the underground long-distance belt conveyor is determined to be the belt deviation.

[0133] The deviation number threshold does not specifically refer to a fixed threshold, and the deviation number threshold can be determined according to the actual application scenario. For example, deviation detection can be performed every 10 frames. When the initial belt deviation detection result is detected as the belt deviation for 8 consecutive times, it is determined that the belt deviation detection result corresponding to the underground long-distance belt conveyor is the belt deviation and a relevant alarm signal is output.

[0134] In summary, the method provided in this embodiment determines the initial position of the belt cross section, and calibrates the initial position through the camera device to obtain the reference frame contour point cloud data of the belt cross section; therefore, the accuracy of obtaining the reference frame contour point cloud data can be improved. Then, the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor is obtained through the camera device; the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data are preprocessed to obtain the preprocessed reference frame contour point cloud data and the preprocessed current frame contour point cloud data; the angle between the preprocessed reference frame contour point cloud data, the camera device and the preprocessed current frame contour point cloud data is determined; therefore, the accuracy of the angle determination can be improved. Then, if the angle exceeds the angle threshold, a secondary judgment of the belt deviation is performed based on the current frame contour point cloud data to obtain the belt deviation detection result; therefore, the deviation detection of the underground long-distance belt conveyor can be realized, and the deviation detection accuracy is high.

[0135] In order to implement the above embodiments, the present disclosure also proposes an underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud.

[0136] For example, the underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud includes:

[0137] A data acquisition module, used for acquiring the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through a camera device;

[0138] A back-end processing unit for determining an angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section;

[0139] The back-end processing unit is also used to perform a secondary determination of the belt deviation based on the current frame contour point cloud data if the angle exceeds the angle threshold, so as to obtain the belt deviation detection result.

[0140] It should be noted that the aforementioned explanation of the embodiment of the method for detecting deviation of an underground long-distance belt conveyor based on three-dimensional point cloud is also applicable to the system for detecting deviation of an underground long-distance belt conveyor based on three-dimensional point cloud in this embodiment, and will not be repeated here.

[0141] Among them, the camera device in the data acquisition module can collect data to the back-end processing unit at a fixed frequency, and the camera device is only responsible for the necessary camera parameter settings, Internet Protocol (IP), output frequency settings, etc.

[0142] Among them, the back-end processing unit can be composed of edge computing devices to complete algorithm processing and linkage control signal output. For example, the RK1126CPU can be used as the core, with auxiliary analog output input modules and direct current (DC) 24V input and output modules, and the relevant underground long-distance belt conveyor deviation detection algorithm based on three-dimensional point cloud and interactive interface can be transplanted internally to complete the three basic functions of camera equipment data collection, algorithm processing (for specific algorithm processing, see the underground long-distance belt conveyor deviation detection method based on three-dimensional point cloud), and algorithm result expression. The algorithm result expression includes the relay output and analog output it carries, and is compatible with common network communication outputs such as Modbustcp, Transmission Control Protocol (TCP), and databases.

[0143] Take a scenario as an example. Figure 8 The following is a schematic diagram of a system for detecting deviation of a long-distance belt conveyor in an underground mine based on a three-dimensional point cloud according to an embodiment of the present disclosure. Figure 8 As shown, the system includes a data acquisition module (including a camera device), a back-end processing unit, a control unit, an underground ring network, a ground server, a display, a power supply, a belt, and an alarm light.

[0144] Among them, after the camera equipment collects data, it can transmit the data to the back-end processing unit through the network cable. When the back-end processing unit detects that the deviation exceeds a given threshold (the threshold can be flexibly configured according to the on-site situation, such as setting an angle threshold. If the left / right angle exceeds 2°, it is considered as a deviation warning prompt. If the left / right deviation exceeds 4°, it is considered to be a serious deviation and intervention is required for shutdown processing), on the one hand, the alarm signal will be reported to the ground server platform to remind the monitoring room personnel to handle it in real time. At the same time, the alarm signal will be output through the IO board, the sound and light alarm lights will be controlled to alarm on the spot, and the control signal will be transmitted to the control unit through the modbustcp protocol for shutdown linkage processing.

[0145] The control unit may be, for example, a programmable logic controller (PLC), which may receive analog signals output by the back-end processing unit for linkage control or alarm output. The linkage control includes the start and stop control of the belt. For example, when the back-end processing unit detects that the belt deviation exceeds a given threshold, it may output an analog control signal to the PLC, and the PLC may linkage control the belt to stop.

[0146] Among them, the ground server can store historical belt speed, deviation data, etc., and can set the current shift, monthly, annual accumulation, etc. to form reports, and continuously optimize and adjust algorithm parameters and control strategies through data analysis.

[0147] Among them, the camera equipment, back-end processing unit, control unit and alarm light can be powered by 12v or 24v power supply.

[0148] In summary, the system provided by the embodiments of the present invention can achieve a deviation detection accuracy of millimeter level, and can effectively reduce the influence of some factors (such as uneven installation position, changes in belt shape, etc.) on deviation detection. The detection accuracy, sensitivity, and adaptability are higher, the deviation detection processing speed is faster, the system has good stability, high fault tolerance, and is not affected by the electromagnetic environment. Embedded algorithm transplantation supports edge computing, which can greatly reduce the computing pressure of the central server, and can still maintain normal operation when problems occur in the underground ring network. The system has high stability.

[0149] In order to implement the above-mentioned embodiments, the present disclosure also proposes an underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud, including: a processor, and a memory connected to the processor for communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided by the above-mentioned embodiments.

[0150] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0151] In order to implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0152] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure shall comply with the relevant laws and regulations and shall not violate public order and good morals.

[0153] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0154] The present disclosure anticipates providing implementation schemes for users to selectively block the use or access of personal information data. That is, the present disclosure anticipates providing hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, risks can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.

[0155] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0156] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0157] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0158] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0159] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0160] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0161] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0162] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A method for detecting deviation of underground long-distance belt conveyor based on three-dimensional point cloud, characterized in that: include: Acquire the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through the camera device; Determining an angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section; If the angle exceeds the angle threshold, a secondary belt deviation determination is performed based on the current frame contour point cloud data to obtain a belt deviation detection result.

2. The method according to claim 1, characterized in that Before determining the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section, the method further includes: The initial position of the belt cross section is determined, and the initial position is calibrated by a camera device to obtain the reference frame contour point cloud data of the belt cross section.

3. The method according to claim 2, characterized in that Before calibrating the initial position by a camera device to obtain the reference frame contour point cloud data of the belt cross section, the method further includes: The camera parameters of the camera device are set so that the reference frame contour point cloud data meets the belt deviation detection requirements, wherein the camera parameters include an area of ​​interest, brightness, exposure rate and frame rate.

4. The method according to claim 1, characterized in that: The determining of the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section comprises: Performing data preprocessing on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data to obtain preprocessed reference frame contour point cloud data and preprocessed current frame contour point cloud data; An angle between the preprocessed reference frame contour point cloud data, the camera device, and the preprocessed current frame contour point cloud data is determined.

5. The method according to claim 4, characterized in that The data preprocessing of the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data includes at least one of the following: Performing through-filter processing on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data; Statistical outlier removal processing is performed on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data.

6. The method according to claim 5, characterized in that The step of performing through filtering on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data comprises: Determine a filter coordinate axis and a filter data range corresponding to the filter coordinate axis; According to the filtering data range corresponding to the filtering coordinate axis, the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data are subjected to through-filtering processing.

7. The method according to claim 5, characterized in that The statistical outlier removal process is performed on the reference frame contour point cloud data of the belt cross section and the current frame contour point cloud data, including: Determine an average distance between each data point in the point cloud data and a set of K-neighborhood points corresponding to the data point, wherein the point cloud data is the reference frame contour point cloud data of the belt cross section or the current frame contour point cloud data; Determining the standard deviation between the average distances corresponding to all data points in the point cloud data; A confidence range is determined according to the standard deviation and the average distance, and data points in the point cloud data whose confidences are outside the confidence range are removed.

8. The method according to claim 1, characterized in that The determining of the angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section comprises: Based on the Euclidean distance, determine the belt edge point on the left side of the current frame and the belt edge point on the right side of the current frame in the contour point cloud data of the current frame; Based on the Euclidean distance, determining the belt edge point on the left side of the reference frame and the belt edge point on the right side of the reference frame in the reference frame contour point cloud data of the belt cross section; Determine a first angle between the left belt edge point of the current frame, the camera device, and the left belt edge point of the reference frame; A second angle between the right belt edge point of the current frame, the camera device, and the right belt edge point of the reference frame is determined.

9. The method according to claim 1, characterized in that: The secondary determination of belt deviation based on the current frame contour point cloud data to obtain the belt deviation detection result includes: If the left frame contour point cloud data in the current frame contour point cloud data is located on the left side of the reference frame belt edge point, and the right frame contour point cloud data in the current frame contour point cloud data is located on the right side of the reference frame belt edge point, then the initial belt deviation detection result corresponding to the current frame contour point cloud data is determined to be that the belt has deviated; If the initial belt deviation detection result is that the consecutive number of belt deviations is greater than the deviation number threshold, then the belt deviation detection result corresponding to the underground long-distance belt conveyor is determined to be that the belt has deviated.

10. An underground long-distance belt conveyor deviation detection system based on three-dimensional point cloud, characterized in that: include: A data acquisition module, used for acquiring the current frame contour point cloud data of the belt cross section of the underground long-distance belt conveyor through a camera device; A back-end processing unit, used to determine an angle between the current frame contour point cloud data, the camera device, and the reference frame contour point cloud data of the belt cross section; The back-end processing unit is further used to perform a secondary belt deviation determination based on the current frame contour point cloud data if the angle exceeds an angle threshold, so as to obtain a belt deviation detection result.

Images