Scraper conveyor chain breakage monitoring method and device based on machine vision

Abstract

The invention relates to the technical field of mining scraper conveyor equipment, in particular to a scraper conveyor chain breakage monitoring method and device based on machine vision. The method comprises the following steps: acquiring a real-time three-dimensional image when a chain moves through a stereo camera; obtaining real-time state data of the chain according to the real-time three-dimensional image; and comparing the real-time state data with standard data, and if the real-time state data does not meet a preset threshold value, determining that the chain has a chain breakage risk. According to the method, the real-time three-dimensional image of the chain during movement is obtained through the stereo camera, the real-time state data of the chain is obtained according to the real-time three-dimensional image, whether the chain breakage risk exists or not is determined by comparing the real-time state data with the standard data, and the chain state can be accurately monitored in real time.

Description

Technical Field

[0001] The present application relates to the technical field of mining scraper conveyor equipment, and in particular to a machine vision-based scraper conveyor chain break monitoring method and device. Background Art

[0002] Scraper conveyor is the main transportation equipment of the fully mechanized mining face in underground coal mines. During the operation of the scraper conveyor, various factors such as excessive coal loading, uneven working surface, and foreign matter jamming cause the chain to be repeatedly subjected to impact loads, resulting in severe chain wear and fatigue damage, and chain breakage failures often occur. Chain breakage failures not only affect production, but in severe cases also damage the scraper conveyor and cause the entire fully mechanized mining face to shut down, greatly reducing the company's production efficiency and causing huge economic losses. Therefore, chain breakage monitoring of the scraper chain of the scraper conveyor is particularly important under the requirements of today's smart mines.

[0003] At present, the prediction of scraper chain breakage is mainly achieved by inferring chain tension from monitoring parameters such as power, overhang, and cylinder pressure. For example, Chinese patent CN219193541U discloses a tension monitoring and chain break warning device for scraper conveyors, which indirectly infers the state of the chain through the change of tension. The existing scraper chain breakage monitoring scheme for scraper conveyors has the disadvantages of poor real-time performance, many objective influencing factors, and poor accuracy. Summary of the invention

[0004] In view of this, it is necessary to provide a scraper conveyor chain breakage monitoring method and device based on machine vision to solve the shortcomings of the prior art of predicting scraper chain breakage by inferring chain tension through side parameters, such as poor real-time performance, many objective influencing factors, and poor accuracy.

[0005] A chain break monitoring method for a scraper conveyor based on machine vision, comprising:

[0006] Acquire real-time 3D images of the chain in motion through a stereo camera;

[0007] According to the real-time three-dimensional image, real-time status data of the chain is obtained;

[0008] The real-time status data is compared with the standard data, and if a preset threshold is not met, it is determined that the chain is at risk of breaking.

[0009] Preferably, obtaining the real-time status data of the chain according to the real-time three-dimensional image information includes:

[0010] Calibrate the three-dimensional coordinate data of the chain links according to the real-time three-dimensional image;

[0011] According to the three-dimensional coordinate data, the real-time status data of the chain is obtained; wherein the real-time status data of the chain includes the real-time straight line data of the chain link, the pitch data of adjacent chain links and the arc angle data of adjacent chain links.

[0012] Preferably, obtaining the real-time status data of the chain according to the real-time three-dimensional image information includes:

[0013] According to the real-time three-dimensional image, real-time surface texture data of the chain is obtained;

[0014] The real-time surface texture data is compared with the preset texture data to obtain the real-time status data of the chain.

[0015] Preferably, the method further comprises: acquiring a real-time two-dimensional image of the chain through an industrial camera;

[0016] The real-time two-dimensional image and the real-time three-dimensional image are fused to obtain real-time fused surface texture image data of the chain;

[0017] The real-time fused surface texture image data is compared with the preset texture data to obtain the real-time status data of the chain.

[0018] Preferably, the surface texture features of the chain are extracted based on the real-time fused surface texture image data through a machine learning algorithm; defect identification is performed based on the surface texture features of the chain through a machine learning algorithm to obtain a defect identification result; the defect identification result is compared with the standard data, and if a preset threshold is not met, it is determined that the chain is at risk of breaking.

[0019] Preferably, the state data of the chain within a preset time period is analyzed to obtain the state change of the chain within the preset time period; and whether the chain is at risk of breaking is predicted based on the state change.

[0020] Preferably, the method further comprises: after determining that the chain is at risk of breaking, issuing an alarm.

[0021] Compared with the prior art, the beneficial effect of the present application is that a real-time three-dimensional image of the chain in motion is obtained through a stereo camera, and the real-time status data of the chain is obtained based on this, which is compared with the standard data to determine whether there is a risk of chain breakage. The chain status can be monitored in real time and accurately, thereby improving the real-time and accuracy of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic flow chart of a method for monitoring chain breakage of a scraper conveyor based on machine vision provided in an embodiment of the present application.

[0023] Figure 2 A schematic diagram of the monitoring range provided for an embodiment of the present application.

[0024] Figure 3 A schematic structural diagram of a chain break monitoring device 300 for a scraper conveyor based on machine vision provided in one embodiment of the present application.

[0025] Figure 4 This is an example diagram of a stereo camera. DETAILED DESCRIPTION

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] See also Figure 1 , Figure 1 1 is a flow chart of a method for monitoring chain breakage of a scraper conveyor based on machine vision provided in an embodiment of the present application. The method for monitoring chain breakage of a scraper conveyor based on machine vision comprises the following steps:

[0028] S101: Acquire a real-time three-dimensional image of the chain in motion through a stereo camera.

[0029] The stereo camera consists of a pair of lenses installed at a certain distance to capture images of the chain at different viewing angles. The stereo camera can be a binocular stereo camera, which can be set above the chain. When the chain is in motion, the stereo camera continuously captures a sequence of images of the chain. In order to ensure image quality, the stereo camera needs to be configured with appropriate lighting conditions. Line lasers can be used to assist positioning to enhance the contrast of the image and ensure that the surface features of the chain are clearly visible. In addition, the distance between the stereo camera and the scraper chain should be kept within a reasonable range to ensure the resolution and coverage of the image.

[0030] S102: Obtaining real-time status data of the chain according to the real-time three-dimensional image.

[0031] After obtaining the real-time three-dimensional image of the chain, the key feature information of the chain is extracted through the image processing algorithm. This information includes but is not limited to the geometric properties of the chain link, such as position, size, and shape. First, the algorithm pre-processes the three-dimensional image, including filtering and denoising, contrast enhancement, etc., to improve the accuracy of subsequent feature extraction. Then, the calibration algorithm is used to determine the position coordinates of the chain link in three-dimensional space, and then the actual distance and relative position between the chain links are calculated through the straight line and arc fitting algorithm. In addition, it is also necessary to calculate the pitch data between adjacent chain links and the arc angle between the chain links as part of the real-time status data of the chain.

[0032] S103: Compare the real-time status data with the standard data. If the preset threshold is not met, it is determined that the chain is at risk of breaking.

[0033] After obtaining the real-time status data of the chain, the system compares this data with the preset standard data. The standard data usually comes from the statistical data of the chain under normal working conditions, or the ideal state parameters provided by the manufacturer. If the real-time status data deviates from the standard data by more than the preset threshold, it indicates that the chain may be worn or damaged, and there is a risk of chain breaking. At this time, the system will generate an early warning message and notify the operator through the alarm or display to conduct inspection or maintenance to prevent the chain breakage from causing production interruption or other more serious consequences.

[0034] Furthermore, the real-time surface texture data of the chain is obtained according to the real-time three-dimensional image. The state information of the chain is obtained by analyzing the surface texture data in the real-time three-dimensional image. The surface texture data can help identify the wear and cracks on the surface of the chain, thereby more comprehensively evaluating the health of the chain. This method can effectively supplement the state evaluation based on geometric features and improve the comprehensiveness and accuracy of monitoring.

[0035] Furthermore, an industrial camera can be set at an appropriate position of the chain to obtain a two-dimensional image of the chain. The real-time two-dimensional image of the chain is obtained by the industrial camera, and the two-dimensional image data is introduced on the basis of the three-dimensional image, and analyzed together with the three-dimensional image, so as to obtain richer chain status data. The two-dimensional image can not only provide additional viewing angle information, but also combine the two-dimensional image with the three-dimensional image to enhance the recognition efficiency and quality of the chain surface texture, make up for the shortcomings of the three-dimensional image in some aspects, and improve the reliability and accuracy of the overall monitoring.

[0036] Furthermore, the state data of the chain within a preset time period is analyzed to obtain the state change of the chain within the preset time period. By analyzing the state data of the chain within a certain time period, the change trend of the chain state can be observed, which is helpful for early detection of potential chain break risks. By continuously monitoring and recording the state data of the chain, a state change curve can be drawn, especially in combination with real-time image analysis, which can improve the accuracy of real-time image recognition and provide an accurate, reliable and scientific basis for predictive maintenance and early warning.

[0037] The following is a further introduction to the chain break monitoring method of a scraper conveyor chain based on machine vision provided by the present application in conjunction with a specific embodiment:

[0038] For example, the maximum distance between two chains of a scraper is d = 330 mm; the maximum length of a chain link is P = 197 mm. The detection scenario is the chain motion state, and the length range to be monitored is Figure 2As shown, it is necessary to detect the two-point length of the scraper chain terminal position, the detection accuracy is required to be ±0.2mm, the chain movement speed is uneven, the chain appearance is the normal iron color, and the double chain of the scraper chain is detected.

[0039] In response to the scanning and measurement needs of the scraper chain in the scraper chain movement scene, a set of 320mm pupil distance (the distance between the two lenses) line laser binocular stereo cameras is used for on-site implementation. The camera is installed at a height of 450mm from the scraper chain. The direction of the laser is consistent with the direction of the scraper chain, and the binocular stereo camera is perpendicular to the direction of the scraper chain. When the chain is in motion, the stereo camera can generate three-dimensional data of the chain endpoints, and the back-end calibration algorithm, straight line and arc fitting algorithm, positioning algorithm, and measurement algorithm are used to measure the length of the scraper chain endpoints and compare them with the standard length to determine the stretching condition; and identify defects such as wear and cracks on the chain surface, so as to more comprehensively evaluate the health of the chain.

[0040] Specifically, the position coordinates of the chain link in three-dimensional space are determined by a calibration algorithm:

[0041] Use preset landmarks or objects of known geometric shapes (such as a chessboard) as calibration patterns and place them in the camera's field of view. Obtain landmark pattern images at multiple different angles, and use a camera calibration algorithm (such as the calibrateCamera function in OpenCV) to calculate the camera's internal parameter matrix, distortion coefficients, etc. Then perform feature point extraction: Identify and extract the key feature points of the chain link in the three-dimensional image, such as the center point of the chain link, the contact point, etc. These feature points usually have obvious edges or color changes, which can be achieved by edge detection algorithms (such as Canny edge detection), corner detection algorithms (such as Harris corner detection), etc. Finally, determine the three-dimensional coordinate mapping of the chain link: Use the internal and external parameters of the camera to map the two-dimensional feature points in the image to the three-dimensional space to obtain the position coordinates of the chain link in the three-dimensional space. Specifically, the backprojection method can be used to convert the two-dimensional image coordinates into three-dimensional world coordinates.

[0042] For example, select the endpoints of the chain link and extract the coordinates of each link endpoint from the three-dimensional coordinate data; the straight-line distance d between the two end points of the chain link can be calculated using the Euclidean distance formula:

[0043]

[0044] Among them, (x1, y1, z1) and (x2, y2, z2) are the three-dimensional coordinates of the two end points of the chain link. The calculated straight line distance is the real-time straight line data of the chain link.

[0045] The calculation of the pitch data of adjacent links requires extracting the coordinates of the contact points between them. Using the coordinates of the contact points, calculate the distance of the pitch between adjacent links. This can also be achieved using the Euclidean distance formula mentioned above. The pitch data is the distance between the contact points between adjacent links.

[0046] The straight line and arc fitting algorithm calculates the actual distance and relative position between the links:

[0047] Line fitting: For each segment between the links that is considered as a straight line, use the least squares method or other optimization algorithms to fit a straight line so that the line can best describe the direction of the link segment. Then calculate the line equation to obtain the direction vector of the line and the coordinates of a point.

[0048] Arc fitting: For the curved link part, an arc fitting algorithm is used to find an arc that can best match the actual direction of the link. Common arc fitting algorithms include Hough Transform and RANSAC (Random Sample Consensus).

[0049] Calculate the actual distance and relative position between links: Use the straight line and arc fitting results to calculate the distance between key points between adjacent links. Calculate the relative position relationship between two adjacent links, including but not limited to the angle between two straight lines, the curvature of the arc, etc.

[0050] For example, the calculation of the arc angle data of adjacent links: The arc angle data reflects the degree of bending between adjacent links. The coordinates of a point on the contact point and its extension line between adjacent links can be extracted, and then the angle between the line connecting these two points and the horizontal direction (or the set reference direction) can be calculated. Using the vector dot multiplication formula:

[0051]

[0052] in, and The angle θ can be obtained by respectively representing the vector of the connection line between adjacent links and the vector of the reference direction; the angle thus calculated is the arc angle data of the adjacent links.

[0053] Finally, the calculated link length, the actual distance between adjacent links, the angle representing the actual direction of the link, and the bending angle of the arc between adjacent links are compared with the standard data. If the preset threshold is not met, it is determined that the chain is at risk of breaking. The preset threshold can be set to 2.5%, that is, if the difference between the calculated data and the standard data reaches 2.5%, it is considered that the chain is at risk of breaking.

[0054] In addition, it is also possible to identify defects such as wear and cracks on the chain surface, and determine whether the chain is at risk of breaking based on these defects, which may include:

[0055] First, a line laser binocular stereo camera is used to obtain a real-time 3D image of the chain, and an industrial camera is used to obtain a 2D image of the chain at the same time. These images should include all angles and details of the chain for subsequent comprehensive analysis.

[0056] The acquired images are preprocessed, including but not limited to denoising to improve image quality and grayscale, histogram equalization and other contrast enhancement operations to improve image quality. Then, image processing algorithms are used to extract chain surface features, such as edges, textures, etc., in order to identify defects such as wear and cracks. Commonly used algorithms include but are not limited to Canny edge detection, Sobel operator, etc. The three-dimensional image is combined with the surface features in the two-dimensional image, and the surface information in the two-dimensional image is supplemented by the three-dimensional coordinate data, and vice versa, the texture features in the two-dimensional image are used to enhance the understanding of the three-dimensional image. The combination of the two can significantly enhance the recognition and extraction of chain surface texture and other features.

[0057] Through the results of feature extraction, methods such as machine learning or deep learning are used to identify defects such as wear and cracks on the chain surface. Neural network models can be trained to automatically identify these features. For example, through support vector machines (SVMs) and convolutional neural networks (CNNs), training is performed based on previously extracted features to establish a model that can distinguish between normal chains and defective chains. The image is segmented to separate areas where defects may exist from the background, and the extracted features are compared with a preset or known defect feature library to confirm which features match the known wear or crack features.

[0058] For identified defects, their severity is assessed. For example, the degree of wear can be quantified by comparing the area of ​​the worn area to the total area of ​​the chain, and crack length and width can also be assessed by counting pixels in the image.

[0059] Finally, a risk assessment is conducted: Based on the results of the assessment, if the defects reach a certain threshold, such as the wear area exceeds a predetermined ratio or the crack length exceeds a safe limit, the chain is considered to be at risk of breaking. These thresholds can be set based on experience and historical data.

[0060] When the chain is determined to be at risk of breaking, an alarm signal is immediately issued so that timely countermeasures can be taken. The alarm mechanism can ensure that the staff can respond quickly, prevent the occurrence of chain breakage, and ensure production safety.

[0061] Based on an inventive concept, such as Figure 3As shown, the present application also provides a chain break monitoring device 300 for a scraper conveyor based on machine vision, comprising:

[0062] The stereo camera 301 is used to obtain real-time three-dimensional images of the chain in motion. In some embodiments, a processing chip can be embedded in the camera end to directly output the scanned stereo data, and then send the stereo data to the calculation module 302.

[0063] Furthermore, the stereo camera 301 may be equipped with a line laser 304, and suitable lighting conditions and auxiliary positioning may be configured for the stereo camera 301 to filter out the influence of external light changes on the results, so as to enhance the contrast of the image. Figure 4 The stereo camera example diagram shown can integrate the stereo camera with a line laser, or use a line laser binocular stereo camera.

[0064] Furthermore, an industrial camera 305 may be provided to obtain two-dimensional images, and the two-dimensional images may be combined with three-dimensional images to enhance the recognition efficiency and quality of the chain surface texture, make up for the deficiencies of three-dimensional images in certain aspects, and improve the reliability and accuracy of overall monitoring.

[0065] The calculation module 302 obtains the real-time status data of the chain according to the real-time three-dimensional image. The calculation module 302 can be a central control processor, a single-chip microcomputer, etc. The calculation module 302 can be connected to the stereo camera 301 through a network port and a private communication protocol, and can perform subsequent data transmission through a TCP / IP communication protocol.

[0066] The comparison module 303 compares the real-time status data with the standard data, and determines that the chain is at risk of breaking if the preset threshold is not met. The comparison module 303 can be a central control processor, a single-chip microcomputer, etc., or can be integrated with the calculation module 302, and the functions of the calculation module 302 and the comparison module 303 can be realized by a PC or the like as needed.

[0067] Furthermore, the scraper conveyor chain breakage monitoring device 300 based on machine vision also includes an audible and visual alarm module 306, such as a buzzer and an LED light, which immediately sends out an alarm signal after determining that the chain is at risk of breaking.

[0068] Furthermore, the machine vision-based scraper conveyor chain breakage monitoring device 300 also includes a display module 307, such as a display, for displaying real-time monitoring images, monitoring results and alarm information.

[0069] In summary, the solution provided in this application can realize real-time monitoring of the scraper conveyor chain status, timely discover potential chain break risks, and effectively improve the accuracy of monitoring, providing a scientific basis for preventive maintenance, thereby improving the safety production level underground in coal mines.

Claims

1. A chain break monitoring method for a scraper conveyor based on machine vision, characterized in that: include: Acquire real-time 3D images of the chain in motion through a stereo camera; According to the real-time three-dimensional image, real-time status data of the chain is obtained; The real-time status data is compared with the standard data, and if a preset threshold is not met, it is determined that the chain is at risk of breaking.

2. The method according to claim 1, characterized in that According to the real-time three-dimensional image information, the real-time status data of the chain is obtained, including: Calibrate the three-dimensional coordinate data of the chain links according to the real-time three-dimensional image; According to the three-dimensional coordinate data, the real-time status data of the chain is obtained; wherein the real-time status data of the chain includes the real-time straight line data of the chain link, the pitch data of adjacent chain links and the arc angle data of adjacent chain links.

3. The method according to claim 1, characterized in that According to the real-time three-dimensional image information, the real-time status data of the chain is obtained, including: According to the real-time three-dimensional image, real-time surface texture data of the chain is obtained; The real-time surface texture data is compared with the preset texture data to obtain the real-time status data of the chain.

4. The method according to claim 3, characterized in that Also includes: Obtain real-time 2D images of the chain through industrial cameras; The real-time two-dimensional image and the real-time three-dimensional image are fused to obtain real-time fused surface texture data of the chain; The real-time fused surface texture image data is compared with the preset texture data to obtain the real-time status data of the chain.

5. The method according to claim 4, characterized in that include: Extracting chain surface texture features based on the real-time fused surface texture data through a machine learning algorithm; Defect recognition is performed based on the surface texture features of the chain using a machine learning algorithm to obtain a defect recognition result; The defect identification result is compared with the standard data, and if the preset threshold is not met, it is determined that the chain is at risk of breaking.

6. The method according to claim 1, characterized in that include: Analyzing the state data of the chain within a preset time period to obtain the state change of the chain within the preset time period; Predict whether the chain is at risk of breaking based on the state change.

7. The method according to claim 1, characterized in that Also includes: After determining that the chain is at risk of breaking, an alarm is issued.

8. A chain break monitoring device for a scraper conveyor based on machine vision, characterized in that: include: Stereo camera, used to obtain real-time three-dimensional images of the chain in motion; A calculation module also obtains real-time status data of the chain according to the real-time three-dimensional image; The comparison module compares the real-time status data with the standard data, and if a preset threshold is not met, it is determined that the chain is at risk of breaking.

9. The device according to claim 8, characterized in that include: Industrial camera for acquiring real-time 2D images of the chain.

10. The device according to claim 9, characterized in that Also includes: Line laser for irradiating the chain with laser light.

Images