Coal piling width and height joint detection device and method based on non-contact measurement

Abstract

The invention relates to the technical field of non-contact measurement, in particular to a coal piling width and height joint detection device and method based on non-contact measurement. The method comprises the following steps: S1, collecting a real-time coal falling point image by using monitoring equipment, and determining the position of a binocular camera; s2, carrying out calibration and calibration processing on the binocular camera to obtain standard original coal piling images of the left camera and the right camera, and processing a single picture to obtain the coal piling width; s3, processing the left and right coal piling images obtained in the S2 according to a PSSNet network to obtain an actual coal piling height; s4, whether coal piling occurs or not is jointly judged according to the width index and the height index of the piled coal, and the specific judgment rule is as follows: a first threshold value is set, whether the width exceeds a preset threshold value or not is judged, and if yes, a height coal piling strategy is executed; and if not, repeating the operation of obtaining the coal piling width in the step S2. Compared with the prior art, the novel coal piling detection device is provided, a monocular and binocular combined non-contact measurement method is adopted, and the problems that accurate coal piling detection is difficult to achieve through monocular measurement and the calculation amount of binocular measurement of the coal piling height is large are effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a device and method for jointly detecting the width and height of a coal pile based on non-contact measurement. Background Art

[0002] The underground transportation has complex terrain and long transportation distance. Belt conveyors are widely used in underground transportation because of their large transportation capacity, long transportation distance, high efficiency, and continuous transportation. Due to the long underground transportation distance, multi-stage belt conveyors are required for joint transportation. However, the coal falls too fast during the transfer process, and coal piles are easily generated before the unloading conveyor belt has time to transport it away. At present, the mining belt conveyor mainly uses coal pile sensors to detect coal piles. During the working process, the sensors are easily affected by harsh environments such as coal dust and moisture, and often cannot detect coal pile fault signals in time. The sensors are easily touched by falling materials, and the use effect and reliability are not ideal. Sensor protection can only be warned after coal pile occurs. After the warning, the belt conveyor is shut down, and the accumulated materials need to be cleaned manually, which is extremely time-consuming and labor-intensive. Therefore, early warning and early protection are very important.

[0003] With the development of industrial technology and the continuous progress of coal comprehensive automation, image detection technology based on machine vision has gradually become a new development direction. As a typical non-contact intelligent sensing technology, machine vision perception technology involves artificial intelligence, computer science, image processing, pattern recognition and other fields. By simulating human visual functions, as an extension of the human eye, it measures structured scenes from the images of objective objects, extracts information, processes and understands it, and finally realizes the detection and control of designated targets with its excellent monitoring and detection calculation speed, accuracy and efficient portability. The application of machine vision perception technology in coal stacking mainly uses monocular cameras.

[0004] There are two main ways to detect coal piles with a monocular camera: directly measuring the height of the coal pile and directly measuring the width of the coal pile. To detect the height of the coal pile, the camera is simply calibrated to derive the conversion parameters from the world coordinate system to the image coordinate system. However, monocular measurement technology has large errors in measuring irregular objects such as coal piles, and it is not easy to improve. To detect the width of the coal pile, the width of coal is a regular plane, and it is more accurate to measure the size with a monocular camera, but the simple width exceeding the threshold cannot be directly used as a judgment condition for coal piles, and can only be used as an early warning of the possibility of coal piles. Measuring the height of the coal pile to detect the coal pile is a good direction. Binocular vision has certain advantages in obtaining the size of irregular objects, but binocular detection requires a large amount of calculation. When using binocular height measurement, the existing stereo matching network either cannot capture the context information well to achieve accurate matching. In order to solve the above problems, the present invention also proposes a joint detection method for the width and height of coal piles based on non-contact measurement. Summary of the invention

[0005] The purpose of the present invention is to propose a device and method for jointly detecting the width and height of a coal pile based on non-contact measurement to solve the problems raised in the background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for joint detection of coal pile width and height based on non-contact measurement, comprising the following contents:

[0008] S1. Use monitoring equipment to collect real-time images of coal drop points and determine the position of the binocular camera;

[0009] S2, calibrating and calibrating the binocular camera to obtain the standard original coal pile images of the left and right cameras, and then obtaining the coal pile width by processing a single image;

[0010] S3, processing the left and right coal pile images obtained in S2 according to the PSSNet network to obtain the actual coal pile height;

[0011] S4. Combine the width and height indicators of the coal pile to jointly determine whether a coal pile exists. The specific judgment rules are as follows:

[0012] A first threshold is set to determine whether the width exceeds the preset threshold. If so, a high coal stacking strategy is executed; if not, the operation of obtaining the coal stacking width in S2 is repeated.

[0013] Preferably, the S1 specifically includes the following contents:

[0014] Use monitoring equipment to obtain the location of the coal drop point on the belt conveyor, and obtain a set x of multiple possible coal drop points;

[0015] The obtained coal drop point set x is trained and the best coal drop point is predicted as the target point;

[0016] A binocular camera is installed along the running direction of the belt at a position 10 to 30 cm away from the optimal coal dropping point.

[0017] Preferably, the conveyor belt of the belt conveyor comprises an upper belt and a lower belt, and the coal dropping point is located on the lower belt, indicating that the coal on the upper belt is transported to the dropping point of the lower belt.

[0018] Preferably, S2 specifically includes the following contents:

[0019] The standard original coal pile images of the left and right cameras are obtained through camera calibration and zero value calibration, and the YOLOv5 algorithm is used to perform width detection on the coal pile images obtained by the camera.

[0020] Preferably, the camera calibration and zero value calibration are implemented based on Zhang Zhengyou calibration method, and the specific operations are as follows:

[0021] The Camera Calibrator module in Matlab is used to calibrate the internal and external parameters of the binocular camera to obtain the focal length f and baseline B of the camera;

[0022] Use the calibrated binocular camera to obtain the image of the conveyor belt when it is empty, detect the distance from the bottom of the belt groove to the camera, and adjust the distance from the bottom of the belt to the camera to zero based on the actual distance using the relative distance method;

[0023] After the adjustment is completed, the original images of the left and right cameras when shooting coal stacking.

[0024] Preferably, the modified YOLOv5 algorithm is used to obtain the image width, and the specific operations are as follows:

[0025] The YOLOv5 algorithm is a target detection algorithm that uses a coal data set for training. During testing, it can detect coal piles and form a priori boxes. The width of the priori boxes is approximately equal to the width of the coal pile. The pixel coordinates of each corner of the rectangular priori box can be obtained through the YOLOv5 algorithm, from which the pixel width of the priori box can be obtained. The actual width of the priori box can be calculated based on the pinhole imaging principle. The formula is:

[0026] W=(D×P) / F

[0027] W is the actual width, D is the distance from the camera to the belt (known), P is the pixel width, and F is the focal length (known during camera calibration). The formula is written into the YOLOv5 algorithm to obtain the actual width of the coal pile.

[0028] Preferably, S3 specifically includes the following contents:

[0029] The original images obtained by the left and right cameras are matched using the stereo matching algorithm PSSNet to obtain a disparity map, and the depth value of each pixel is obtained through the principle of triangulation to obtain the depth value of the highest point;

[0030] The network structure of the stereo matching algorithm PSSNet is divided into:

[0031] Feature extraction module, feature extraction uses a pyramid squeezed attention structure;

[0032] The cost volume construction module uses the feature maps obtained by passing the left and right images through the feature extraction network to construct the cost volume;

[0033] The feature fusion module uses 3D convolution to extract the information between the left and right feature maps and different disparity levels to obtain the cost volume after feature fusion;

[0034] The disparity calculation module upsamples the cost volume after feature fusion to the original resolution and finds the disparity value with the smallest matching error.

[0035] Preferably, the pyramid squeezing attention structure of the feature extraction module is specifically composed of a new residual block PSABlock with a pyramid structure. The PSA module divides the input feature map into S groups from the channel, and each group uses convolutions with different convolution kernel sizes to construct a pyramid structure, and then obtains the channel attention value through the channel attention mechanism. In this way, PSA integrates contextual information of different scales and produces better pixel-level attention. The PSA module is used to replace the first and second convolution blocks in the residual block to form a new residual block structure PSABlock; the new feature extraction module is composed of four residual block structures PSABlock with four channel numbers of 32, 64, 128, and 128 respectively.

[0036] Preferably, the S4 specifically includes the following contents:

[0037] 80% of the maximum width of coal accumulation when no coal piling occurs during normal operation is set as the first threshold, and it is determined whether the obtained coal pile width data exceeds the first threshold. If it exceeds the first threshold, an early warning is issued to remind the staff and a high coal piling strategy is executed; if it does not exceed the first threshold, the operation of obtaining the coal pile width in S2 is repeated.

[0038] Preferably, the high coal stacking strategy specifically refers to: setting a coal stacking height threshold, called a second threshold, according to national standards and actual scenarios; comparing the coal stacking height obtained through the PSSNet network with the second threshold, and if it exceeds the threshold, sending a protection signal and shutting down the conveyor belt.

[0039] Further protection is provided for a device for joint detection of coal pile width and height based on non-contact measurement using the above method.

[0040] Compared with the prior art, the present invention provides a method for joint detection of coal pile width and height based on non-contact measurement, which has the following beneficial effects:

[0041] (1) The present invention measures the width and height of a coal pile based on a non-contact measurement method, and proposes a new method for jointly determining the coal pile situation by using monocular width recognition and binocular height recognition, thereby solving the problem that it is difficult to achieve accurate coal pile detection using a single camera. At the same time, a method for jointly recognizing the width is adopted, thereby reducing the problem of large computational complexity of binocular measurement and achieving coal pile detection.

[0042] (2) The present invention adopts a stereo matching algorithm to measure the height of the coal pile and proposes a stereo matching algorithm PSSNet based on a pyramid squeeze attention network.

[0043] (3) The PSSNet algorithm proposed in the present invention combines the pyramid structure with the residual structure, and uses context information to enrich the feature map to help estimate the correspondence relationship, which significantly improves the matching of improper areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of a method for joint detection of coal pile width and height based on non-contact measurement mentioned in Example 1 of the present invention;

[0045] Figure 2 It is a structural schematic diagram of the belt conveyor transport system and the coal pile width and height joint detection device based on non-contact measurement mentioned in Example 1 of the present invention;

[0046] Figure 3 This is a schematic diagram of the placement of the real-time coal drop point and binocular camera position mentioned in Example 1 of the present invention;

[0047] Figure 4 Schematic diagram of the network structure of the stereo matching algorithm PSSNet mentioned in Example 1 of the present invention;

[0048] Figure 5 Schematic diagram of the structure of the new residual block structure PSABlock mentioned in Embodiment 1 of the present invention;

[0049] Figure 6 This is a schematic diagram of the structure of the PSA module mentioned in Example 1 of the present invention;

[0050] Figure 7 This is a result diagram of the coal pile width measured by using the modified YOLO v5 algorithm mentioned in Example 1 of the present invention;

[0051] Figure 8 This is a result diagram of the height measured using the stereo matching algorithm PSSNet mentioned in Example 1 of the present invention. DETAILED DESCRIPTION

[0052] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0054] Embodiment 1:

[0055] See also Figure 1The present invention proposes a method for joint detection of coal pile width and height based on non-contact measurement, which includes the following contents:

[0056] Step 1: The monitoring equipment collects real-time images of the coal drop point and determines the position of the binocular camera. The specific method is as follows:

[0057] See also Figure 2-3 The monitoring facility is part of the conveyor belt transportation system and is installed above the conveyor belt to collect images in the entire area. Since there is a coal drop point during graded transportation on the belt conveyor, and coal is easily piled up at the drop point due to the change in the belt conveyor speed, it is necessary to monitor and find the coal drop point through monitoring equipment. Generally, the point vertically corresponding to the highest point of the pile is selected as the drop point. Since there is coal falling on the side of the upper belt near the drop point, the accuracy of the depth value of each point obtained by stereo matching is affected and it is not easy to identify the actual height of the coal pile. Generally, a binocular camera is set 20cm away from the best drop point along the running direction of the belt. The specific settings are as follows: Figure 2-3 shown.

[0058] The method of obtaining the coal dropping point image in step 1 of this embodiment is: first, obtaining the simulated coal dropping point information on the conveyor belt in the historical conveyor transportation system to determine the predicted dropping point data set x; then, training the predicted dropping point data set x to determine the optimal predicted dropping point position; finally, setting the camera according to the coal dropping point position on the conveyor belt.

[0059] Step 2: Calibrate the camera to obtain the standard left and right camera coal pile original images and obtain the coal pile width. The specific method is as follows:

[0060] Camera calibration and zero-value calibration are required to obtain a standard original image. Camera calibration is the basis of image distortion correction and stereo correction. The key to obtaining depth information through stereo matching lies in image distortion correction and stereo correction. Eliminating image distortion can allow the machine to better identify the feature information of the image, and stereo correction can make the imaging planes of the two images parallel. Zero-value calibration can convert the obtained depth information into height information. After camera calibration and zero-value calibration, the original images of the left and right cameras are obtained. Then the coal pile width is detected on the image obtained by the right camera. The width detection uses the YOLO v5 algorithm to detect coal, and the pinhole imaging principle is used to calculate the width of the inspection box instead of the coal pile width, such as Figure 7 Shown are the width detection results.

[0061] The camera calibration in step 2 of this example mainly includes distortion correction and stereo correction. Distortion correction and correction are to establish mathematical models of radial distortion and tangential distortion, and then combine them into the polynomial distortion mathematical model to complete the correction of the image through the calibration dot matrix. Stereo correction uses the Bouguet stereo correction algorithm to perform stereo correction on the left and right images obtained above, and obtain the camera focal length f and baseline length B after correction. Zero value calibration first takes an image of the conveyor belt when it is unloaded, and performs stereo matching to obtain the depth from the lowest point of the belt to the camera. The height of this point is set to zero and called the zero point. When taking pictures with coal piles, the depth of the highest point is alternately obtained with the depth of the zero point.

[0062] Step 3: Process the left and right coal pile images according to the PSSNet network to obtain the actual coal pile height, as follows:

[0063] First, the convolutional neural network is used to extract the features of the two images. After feature extraction, the features of the two images are cascaded to construct a 4D matching cost volume. The cost volume is regularized through three-dimensional convolution. The actual effectiveness of the regularization module will directly affect the matching accuracy. After normalization, the probability of the corresponding pixel disparity is calculated using the softmax operation, and the real disparity value is calculated and displayed using the disparity regression function. After obtaining the disparity value, the depth value is obtained using the triangulation principle. The formula is:

[0064] D=(f×B) / d

[0065] Where D is the depth value and d is the disparity value. The structure diagram of the PSSNet network is as follows Figure 4 As shown in Figure 1. The feature extraction module consists of three convolutions with a kernel size of 3*3 and a channel number of 32 and four residual blocks with channel numbers of 32, 64, 128, and 128 respectively. In the residual block, the PSA module is used to replace the first and second convolution blocks in the residual block to form a new residual block structure PSABlock, as shown in Figure 1. Figure 5 The PSA module is shown in Figure 6 As shown in the figure, the convolution results of convolution kernels of different sizes are spliced ​​to form a pyramid-shaped feature map, and then the attention mechanism is applied to this feature map to extract richer feature information.

[0066] Step 4: Combine the width and height of the coal pile to determine whether a coal pile exists, as follows:

[0067] Since the algorithm for determining the height of the coal pile is relatively complicated and requires a lot of calculations, and the coal pile width is relatively simple to obtain but the detection is inaccurate, it cannot be used as the only criterion for coal pile detection. Therefore, a joint judgment method of width and height is adopted. First, the threshold is set and the obtained coal pile width value is input to determine whether the coal pile width exceeds the threshold. The width result is as follows: Figure 7As shown, if it is an early warning, the staff will be reminded and the height of the coal pile will be detected. If not, continue to run. To detect the height of the coal pile, a height threshold must be set. The height value obtained by the stereo matching algorithm is input to determine whether the height of the coal pile exceeds the threshold. The height detection result is as follows Figure 8 As shown in the figure, if it is an alarm and the belt conveyor stops, if it is not, it continues to run. The specific identification steps are as follows Figure 1 shown.

[0068] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for joint detection of coal pile width and height based on non-contact measurement, characterized in that: Includes the following: S1. Use monitoring equipment to collect real-time images of coal drop points and determine the position of the binocular camera; S2, calibrating and calibrating the binocular camera to obtain the standard original coal pile images of the left and right cameras, and then obtaining the coal pile width by processing a single image; S3, processing the left and right coal pile images obtained in S2 according to the PSSNet network to obtain the actual coal pile height; S4. Combine the width and height indicators of the coal pile to jointly determine whether a coal pile exists. The specific judgment rules are as follows: A first threshold is set to determine whether the width exceeds the preset threshold. If so, a high coal stacking strategy is executed; if not, the operation of obtaining the coal stacking width in S2 is repeated.

2. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 1, characterized in that: The S1 specifically includes the following contents: Use monitoring equipment to obtain the location of the coal drop point on the belt conveyor, and obtain a set x of multiple possible coal drop points; The obtained coal drop point set x is trained and the best coal drop point is predicted as the target point; A binocular camera is installed along the running direction of the belt at a position 10 to 30 cm away from the optimal coal dropping point.

3. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 2 is characterized in that: The conveyor belt of the belt conveyor includes an upper belt and a lower belt, and the coal dropping point is located on the lower belt, indicating that the coal in the upper belt is transported to the dropping point of the lower belt.

4. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 1, characterized in that: The S2 specifically includes the following contents: The standard original coal pile images of the left and right cameras are obtained through camera calibration and zero value calibration, and the coal pile width is detected on the coal pile images obtained by the camera using the modified YOLOv5 algorithm.

5. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 4, characterized in that: The camera calibration and zero value calibration are implemented based on Zhang Zhengyou's calibration method, and the specific operations are as follows: The Camera Calibrator module in Matlab is used to calibrate the internal and external parameters of the binocular camera to obtain the focal length and baseline B of the camera; Use the calibrated binocular camera to obtain the image of the conveyor belt when it is empty, detect the distance from the bottom of the belt groove to the camera, and adjust the distance from the bottom of the belt to the camera to zero based on the actual distance using the relative distance method; After the adjustment is completed, the original images of the left and right cameras when shooting coal stacking.

6. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 4, characterized in that: The modified YOLOv5 algorithm is used to obtain the image width. The specific operations are as follows: The YOLOv5 algorithm is a target detection algorithm that uses a coal data set for training. During testing, the coal pile is detected and a priori box is formed. The width of the priori box is approximately equal to the width of the coal pile. The pixel coordinates of each corner of the rectangular priori box are obtained through the YOLOv5 algorithm, thereby obtaining the pixel width of the priori box. The actual width of the priori box is calculated based on the pinhole imaging principle. The specific calculation formula is: W=(D×P) / F Among them, W is the actual width, D is the distance from the camera to the belt, P is the pixel width, and F is the focal length; the formula is written into the YOLOv5 algorithm to obtain the actual width of the coal pile.

7. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 1, characterized in that: The S3 specifically includes the following contents: The original images obtained by the left and right cameras are matched using the stereo matching algorithm PSSNet to obtain a disparity map, and the depth value of each pixel is obtained through the principle of triangulation to obtain the depth value of the highest point; The network structure of the stereo matching algorithm PSSNet is divided into: Feature extraction module, feature extraction uses a pyramid squeezed attention structure; The cost volume construction module uses the feature maps obtained by passing the left and right images through the feature extraction network to construct CostVolume; The feature fusion module uses 3D convolution to extract the information between the left and right feature maps and different disparity levels to obtain the cost volume after feature fusion; The disparity calculation module upsamples the cost volume after feature fusion to the original resolution and finds the disparity value with the smallest matching error.

8. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 7, characterized in that: The pyramid squeezed attention structure of the feature extraction module is specifically composed of a new residual block PSABlock with a pyramid structure; PSABlock divides the input feature map into S groups from the channel, each group uses convolution with different convolution kernel sizes to construct a pyramid structure, and then obtains the channel attention value through the channel attention mechanism. In this way, PSABlock integrates context information of different scales and produces better pixel-level attention; PSABlock is used to replace the first and second convolution blocks in the residual block to form a new residual block structure; the PSABlock module is used to replace the residual structure and pyramid structure in PSMNet, and the new feature extraction module is composed of four residual block structures PSABlock with four channel numbers of 32, 64, 128, and 128 respectively.

9. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 1, characterized in that: The S4 specifically includes the following contents: 80% of the maximum width of coal accumulation when no coal piling occurs during normal operation is set as the first threshold, and it is determined whether the obtained coal pile width data exceeds the first threshold. If it exceeds the first threshold, an early warning is issued to remind the staff and a high coal piling strategy is executed; if it does not exceed the first threshold, the operation of detecting the coal pile width in S2 is repeated.

10. The method for joint detection of coal pile width and height based on non-contact measurement according to claim 9, characterized in that: The high coal stacking strategy specifically refers to: setting a coal stacking height threshold, called the second threshold, according to national standards and actual scenarios; comparing the coal stacking height obtained through the PSSNet network with the second threshold, and if it exceeds the threshold, sending a protection signal and shutting down the conveyor belt.

Images