Method for detecting deviation and foreign matter of conveyor belt of mining belt conveyor

Abstract

The invention discloses a method for detecting deviation and foreign matters of a conveyor belt of a mining belt conveyor, which comprises the following steps of: S1, acquiring image data including the conveyor belt, a carrier roller and the foreign matters, segmenting and marking the area of the conveyor belt, and marking bounding boxes of the carrier roller and the foreign matters; s2, training a multi-task detection model based on the image data, constructing an improved A-YOLOM network, and constructing an adaptive cross-scale feature fusion mechanism by introducing a progressive feature pyramid network, so as to significantly improve the spatial perception capability of foreign matters; meanwhile, the overall network parameter quantity is reduced by adopting GSConv; s3, inputting a to-be-detected image into the trained multi-task detection model, and obtaining a segmentation result, a carrier roller detection frame and a foreign matter detection frame of the conveyor belt; and S4, fault identification. The coal foreign matter detection precision is effectively improved, the parameter quantity of the model is reduced, foreign matter detection and conveying belt segmentation are carried out in one model at the same time, the conveying belt deviation detection time is shortened, and the detection efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of conveyor belt deviation and foreign object detection, and specifically relates to a detection method for the deviation and foreign objects of a mine belt conveyor. Background Art

[0002] Belt conveyors are widely used in the mining field and occupy an important position. This equipment is known for its large transportation volume and high load-bearing capacity, and is especially suitable for long-distance material transportation. However, in the coal transportation system, due to its long line and complex working environment, foreign objects such as metal rods and plastics often mix in. Moreover, due to its working characteristics, faults such as conveyor belt deviation and idler missing often occur during transportation, resulting in frequent safety accidents. Therefore, accurately detecting the deviation of the conveyor belt, identifying foreign objects and giving timely alarms are crucial for ensuring the safe operation of coal mines and preventing serious accidents.

[0003] At present, computer vision technology has developed rapidly, but most of the abnormal state detection methods for belt conveyors can only handle single detection tasks and it is difficult to simultaneously detect segmentation and detection tasks. To achieve multi-task detection, multiple models need to be used, which places high requirements on edge devices. Summary of the Invention

[0004] The purpose of the present invention is to address the above problems and provide a method for detecting coal foreign objects and conveyor belt deviation that can simultaneously perform segmentation and detection tasks, has a small number of parameters, and high accuracy.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:[[]]

[0006] A detection method for the deviation and foreign objects of a mine belt conveyor, including,[[]]

[0007] S1. Collect image data including the conveyor belt, idlers and foreign objects, segment and label the conveyor belt area, and label the bounding boxes of the idlers and foreign objects;[[]]

[0008] S2. Train a multi-task detection model based on the image data;[[]]

[0009] S3. Input the image to be detected into the trained multi-task detection model to obtain the segmentation result of the conveyor belt, the idler detection box and the foreign object detection box;[[]]

[0010] S4. Fault identification, specifically including:[[]]

[0011] Count the idler rollers on both sides of the conveyor belt according to the number of idler detection frames. If the number of idler rollers on both sides is not equal, a serious fault signal is generated. Otherwise, taking the line connecting the center points of the idler rollers along the conveying direction as the reference line, calculate the offset distance between the center point of the area between the reference lines on both sides of the conveyor belt and the center point of the conveyor belt area. If it exceeds the preset offset threshold, an overall offset fault signal is generated. Fit the side edges of the conveyor belt into a straight line and calculate the angle between the fitted edge line and the reference line. If it exceeds the preset angle threshold, a local torsion fault signal is generated. At the same time, detect the position coordinates of the foreign object detection frame. If it is located in the area between the reference lines on both sides, a foreign object presence signal is generated.

[0012] In a possible embodiment, the multi-task detection model in step S2 adopts an improved A-YOLOM (Adaptive YOLO for Real-Time and Generic Multi-Task) network model, including the following improvements:

[0013] Introduce AFPN (Adaptive Feature Pyramid Network) in the feature fusion stage to fuse adjacent-level features and gradually introduce high-level features;

[0014] Replace the standard convolution module with the GSConv (Groupwise Separable Convolution) module in the convolutional layers of the backbone network and the detection neck of the model to reduce the parameters and computational complexity of the model;

[0015] Adopt the fused sliding loss function (SlideLoss) as the loss function to dynamically adjust the weights of easy and difficult samples to improve the recognition ability of difficult samples;

[0016] Adopt the soft non-maximum suppression (Soft-NMS) post-processing algorithm to adjust the confidence of overlapping detection frames for attenuation to reduce missed detections.

[0017] In a possible embodiment, the expression of the fused sliding loss function (SlideLoss) in step S2 is:

[0018]

[0019] Among them, μ is the adaptive learning sample threshold parameter and the negative sample threshold parameter.

[0020] In a possible embodiment, adopting the soft non-maximum suppression (Soft-NMS) post-processing algorithm to attenuate the confidence of overlapping detection frames includes:

[0021] The confidence of overlapping low-confidence bounding boxes is attenuated according to the Gaussian distribution function. When the IOU value of the candidate bounding box is greater than the threshold, the soft non-maximum suppression post-processing algorithm adjusts the score of the candidate bounding box according to the following formula:

[0022]

[0023] In the formula: S i represents the original confidence score of the i th candidate bounding box; M is the candidate bounding box with the highest predicted score currently; b i is the candidate bounding box waiting to be processed, N is a preset IOU threshold.

[0024] In a possible embodiment, the implementation of the GSConv module includes the following steps:

[0025] Convert the input features into a primary feature map with c2 / 2 channels through a standard convolutional layer; perform depthwise separable convolution processing on the primary feature map to generate a secondary feature map with c2 / 2 channels; splice the primary feature map and the secondary feature map into a fused feature map with c2 channels; perform a channel shuffle operation on the fused feature map and output a c2-channel feature map with uniform channel distribution.

[0026] In a possible embodiment, the pixel distance is used to calculate the offset distance between the center point of the area between the reference lines on both sides of the conveyor belt and the center point of the conveyor belt area. If the offset distance satisfies:

[0027]

[0028] The system determines that the conveyor belt has a global offset fault. Among them, α is the deviation threshold, 0 < α < 1, and w is the distance between the reference line and the edge of the conveyor belt.

[0029] In a possible embodiment, the calculation of the center point of the conveyor belt area in step S4 includes:

[0030] Intercept the central area of the image based on a preset ratio to exclude edge interference;

[0031] Execute an edge detection algorithm on the central area of the image to extract the edge contour feature points of the conveyor belt;

[0032] Determine the midpoint coordinates of the conveyor belt area in the conveying direction according to the distribution of the feature points in the conveying direction.

[0033] In a possible embodiment, the preset ratio is 10%.

[0034] In a possible embodiment, in step S1, the conveyor belt area is segmented and labeled, and the idler and foreign objects are labeled with bounding boxes using the labelme software.

[0035] Advantages of the present invention: Through the improved A-YOLOM multi-task detection model, end-to-end synchronous processing of conveyor belt segmentation, idler detection, and foreign object detection is achieved, avoiding the computational redundancy of multiple independent model operations in traditional methods. The GSConv module is used to replace the standard convolution, and the progressive feature pyramid network is combined to optimize feature fusion. The model has a low number of parameters, is suitable for edge computing devices, and meets the requirements of mine scenarios for low-power and low-cost hardware. Based on the adaptive sliding loss function, the weights of easy and difficult samples are dynamically adjusted to improve the accuracy of conveyor belt edge segmentation under complex lighting or dust interference. Through soft non-maximum suppression, the confidence of overlapping foreign object bounding boxes is attenuated and adjusted, reducing the missed detection rate. By fusing multi-scale features, complex environmental interferences such as conveyor belt surface reflection and coal dust coverage are effectively addressed. Based on the three-level fault classification of the number of idlers, offset distance, and local torsion angle, the false alarm rate is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is the overall flowchart of the detection method for the offset and foreign objects of the mine belt conveyor in the embodiment of the present invention.

[0037] Figure 2 It is the detection model framework of the detection method for the offset and foreign objects of the mine belt conveyor in the embodiment of the present invention.

[0038] Figure 3 It is the fault identification flowchart of the detection method for the offset and foreign objects of the mine belt conveyor in the embodiment of the present invention.

[0039] Figure 4 It is the schematic diagram for judging the deviation of the conveyor belt in the embodiment of the present invention.

[0040] The text labels in the figure are represented as: 1. Reference line; 2. Conveyor belt edge line; 3. Center point of the conveyor belt area; 4. Center point of the area between the two reference lines. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings. The description of this part is only exemplary and explanatory, and should not have any limiting effect on the protection scope of the present invention.

[0042] Refer to Figure 1 , this embodiment provides a detection method for the offset and foreign objects of the conveyor belt of a mine belt conveyor, including the following steps:

[0043] S1. Obtain an image and annotate the image data using annotation software.

[0044] Collect image data containing a conveyor belt, idlers, and foreign objects. These image data are from a public dataset. Use the Labelimg image annotation software to annotate the idlers, conveyor belt, and foreign objects in the image. The foreign objects include metal rods, plastics, etc., which are used as model object detection labels. Secondly, use the Labelimg image annotation software to annotate the edge of the conveyor belt of the belt conveyor in the image as a model segmentation label.

[0045] S2. Train a multi-task detection model based on the image data.

[0046] Use the A-YOLOM multi-task detection model and improve it to be more suitable for this scenario. The specific improvements include: introducing a Progressive Feature Pyramid Network (AFPN) in the feature fusion stage to fuse adjacent hierarchical features and progressively introduce high-level features; using a lightweight convolution module GSConv module to replace the standard convolution module in the convolutional layers of the backbone network and detection neck of the model to reduce the parameters and computational amount of the model; adopting a fused sliding loss function as the loss function to dynamically adjust the weights of easy and hard samples to improve the recognition ability of hard samples; using a soft non-maximum suppression post-processing algorithm to adjust the confidence decay of overlapping detection boxes to reduce missed detections.

[0047] A-YOLOM is a multi-task real-time detection and segmentation model based on the YOLOv8 framework, which consists of a shared backbone network Backbone, multiple necks Neck, and a head network Head. Use one network to complete foreign object, idler detection, and conveyor belt segmentation detection at the same time, and integrate two necks and a head onto one backbone, saving a large amount of computational resources and inference time.

[0048] The Progressive Feature Pyramid Network (AFPN) mainly includes a Conv module, a C2f module, and an ASFF module. AFPN can effectively avoid significant semantic differences between non-adjacent levels and promote direct interaction between different levels by fusing adjacent low-level features and progressively introducing high-level features. AFPN solves the semantic gap and information degradation problems in traditional multi-scale fusion through a phased hierarchical feature fusion and dynamic weight allocation mechanism. Its core lies in gradually narrowing the hierarchical gap through a progressive architecture and using an adaptive strategy to optimize the fusion weights, ultimately achieving higher accuracy and robustness in multi-object detection tasks.

[0049] In some embodiments, the fused sliding loss function SlideLoss can adaptively learn the sample threshold parameter and the negative sample threshold parameter μ to guide the model to increase the attention to hard samples. The specific formula is:

[0050]

[0051] The model can dynamically adjust the attention to different samples according to the training data. For example, during the training process, if it is found that some samples (according to their corresponding x values) have a greater impact on the optimization of the model (possibly difficult samples), then the value of μ may be adjusted accordingly, thereby changing the output of the function f(x) for these samples, guiding the model to pay more attention to these difficult samples, and improving the model's processing ability for them.

[0052] For samples whose x values are in a specific range (such as close to μ or in other key intervals), the loss function can handle them in a different way from traditional loss functions. For example, giving a fixed large "penalty" (output is 1) to samples where x ≤ μ - 0.1 can prompt the model to pay more attention to the possible problems of such samples during training; while adjusting the attention to samples where x ≥ μ in an exponentially decaying manner can more finely handle those relatively easy but still requiring appropriate attention samples.

[0053] In some possible implementation manners, the Soft-NMS post-processing algorithm attenuates the confidence of overlapping detection boxes, including:

[0054] Attenuating the confidence of overlapping low-confidence boxes according to the Gaussian distribution function. When the IOU value of the candidate box is greater than the threshold, the Soft-NMS post-processing algorithm adjusts the score of the candidate box according to the following formula:

[0055]

[0056] In the formula: S i represents the original confidence score of the i-th candidate box; M is the candidate box with the highest current predicted score; b[[i]] is the candidate box to be processed, and N is a preset IOU threshold.

[0057] When IOU(M, b[[i]]) < N, that is, the intersection over union of the i-th candidate box and the candidate box M with the highest current score is less than the threshold N, then the confidence score S[[i]] of the candidate box b[[i]] remains unchanged and is still its original score S[[i]]. i ) < N, that is, the intersection over union of the i-th candidate box and the candidate box M with the highest current score is less than the threshold N, then the confidence score S[[i]] of the candidate box b[[i]] remains unchanged and is still its original score S[[i]]. i of the candidate box b[[i]] remains unchanged and is still its original score S[[i]]. i remains unchanged and is still its original score S[[i]]. i .

[0058] By attenuating the confidence of overlapping boxes instead of directly setting it to 0, Soft-NMS can, to a certain extent, retain the information of those boxes that although overlap with the high-score box but may still be valid detections, thereby improving the detection recall rate.

[0059] In some possible embodiments, refer to Figure 2, the GSConv module is used to replace the standard convolution module in the backbone network and the convolutional layer of the detection neck of the model, including: First, the input is converted into a feature map with c2 / 2 number of channels through standard convolution to enhance the feature expression ability. This feature map is further processed through depthwise separable convolution to generate another set of feature maps with c2 / 2 number of channels to reduce the computational complexity. These two sets of feature maps are merged through the Concat operation to form a concatenated feature map with c2 number of channels. Through channel shuffle, the uniform distribution of feature information is ensured, and at the same time, a feature map with the required c2 number of channels is output.

[0060] Reference Figure 2 , the implementation of the improved A-YOLOM algorithm for detecting coal foreign objects and conveyor belt deviation includes:

[0061] The image is input into the model after normalization, resizing, and data augmentation. First, the Backbone uses GSConv for lightweight feature extraction and combines with the C2f structure to improve the feature utilization rate. Subsequently, SPPF (SpatialPyramid Pooling Fast) enhances the receptive field through multi-scale pooling operations and provides richer semantic information for subsequent feature fusion. In the AFPN (Adaptive Feature Pyramid Network) stage, ASFF2 (AdaptiveSpatial Feature Fusion v2) is used for low-level and middle-level feature fusion to improve the detection ability for small targets, while ASFF3 (Adaptive Spatial Feature Fusion v3) further enhances the expression ability of deep features, optimizes multi-scale information interaction, and improves the accuracy of object segmentation.

[0062] Before entering the Segment (object segmentation) task, multiple A-Concat modules are interspersed to perform feature concatenation and fusion, enabling more sufficient interaction of features at different levels and further enhancing the perception ability of object edges and structures. In addition, the Upsample module is used to magnify the feature map to more precisely restore the detailed information of the object.

[0063] Finally, the decoupled detection head separately performs the Detect (object detection) and Segment (object segmentation) tasks.

[0064] Improve the Loss function - SlideLoss to handle the class imbalance problem; use Soft-NMS for post-processing, effectively reduce redundant bounding boxes, improve the stability of detection and segmentation results, and output the object category, bounding box, confidence, and segmentation mask.

[0065] The improved A-YOLOM model has approximately 2.429M trainable parameters in total; the mean Average Precision, the detection accuracy at an IoU (Intersection over Union) threshold of 0.5 is 89.9%. The mean Intersection over Union is 98.9%.

[0066] S3. Input the image to be detected into the trained multi-task detection model to obtain the segmentation result of the conveyor belt, the detection frame of the idler, and the detection frame of foreign objects.

[0067] Input the processed training set into the improved A-YOLOM network for training, and use the precision, recall, and mean Average Precision as the evaluation indicators for object detection tasks such as idlers and foreign objects on the belt conveyor. For the segmentation task, in this study's experiment, the mean Intersection over Union is used to evaluate the conveyor belt edge segmentation task.

[0068] S4. Fault identification.

[0069] Count the idlers on both sides of the conveyor belt according to the number of idler detection frames. If the number of idlers on both sides is not equal, a serious fault signal is generated. Otherwise, taking the line connecting the center points of the idlers along the conveying direction as the reference line, calculate the offset distance between the center point of the area between the reference lines on both sides of the conveyor belt and the center point of the conveyor belt area. If it exceeds the preset offset threshold, an overall offset fault signal is generated. Fit the side edges of the conveyor belt into a straight line to obtain the edge line and calculate its angle with the reference line. If it exceeds the preset angle threshold, a local torsion fault signal is generated. At the same time, detect the position coordinates of the foreign object detection frame. If the coordinates are within the area between the two reference lines, a foreign object presence signal is generated.

[0070] Reference Figure 3 , specifically including, first setting the region of interest, ignoring 10% of the four edge parts of the picture, then using the Canny algorithm to find the edge points and find the midpoint of the conveyor belt area. Secondly, count the identified idlers, divide the four idlers detected in the picture into left and right, and then connect the center points of the two idlers on the left and right along the conveying direction into two straight lines as the reference lines to locate the edge positions on both sides of the conveyor belt. If the number of idlers detected is not equal to the expected four (unequal), it may be a serious offset or the conveyor belt is missing, and no subsequent judgment is made and the detection stops. If the number of idlers detected on both sides is equal, the subsequent detection is performed.

[0071] Conveyor belt torsion fault identification: After dividing the edge points into left and right, use linear fitting to obtain a straight line for the edge points within the idler area, and calculate the angles between the edge lines of the left and right conveyor belts and the reference line. If it exceeds the preset angle threshold, a local torsion fault signal is generated.

[0072] Overall deviation fault identification: The distance between the center coordinate of the area between the midpoint coordinate of the conveyor belt area and the side reference line, if it exceeds the preset deviation threshold, an overall deviation fault signal is generated. Since the publicly available dataset is used and there are many scenarios, the specific distance cannot be known, so the picture pixels are used for judgment. The reference width is the distance w between the straight line of the idler and the edge of the conveyor belt, and the deviation threshold α (0 < α < 1) is determined. If the following condition is met:

[0073]

[0074] The system determines that the conveyor belt has an overall deviation fault.

[0075] Foreign object identification: For each detected foreign object, calculate its position coordinates and determine whether the position coordinates are completely within the idler area. If so, a foreign object presence signal is issued.

[0076] Reference Figure 4 , the reference lines on both sides of the conveyor belt are reference line 1, the edge lines of the conveyor belt on both edges are conveyor belt edge line 2, the point at the center position of the conveyor belt area is the center point 3 of the conveyor belt area, and the point at the midpoint between the reference lines is the center point of the area between the two reference lines.

[0077] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0078] Specific examples are applied in this article to elaborate on the principles and implementation manners of the present invention. The description of the above examples is only used to help understand the method and its core idea of the present invention. The above is only the preferred implementation manner of the present invention. It should be noted that due to the limited nature of written expression and objectively existing infinite specific structures, for those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements, refinements or changes can be made, or the above technical features can be combined in an appropriate manner; these improvements, refinements, changes or combinations, or directly applying the concept and technical solution of the invention to other occasions without improvement, should all be regarded as the protection scope of the present invention.

Claims

1. A detection method for the deviation and foreign objects of the conveyor belt of a mine belt conveyor, characterized in that, Including, S1. Collect image data including the conveyor belt, idlers, and foreign objects, segment and label the conveyor belt area, and perform bounding box labeling on the idlers and foreign objects; S2. Train a multi-task detection model based on the image data; S3. Input the image to be detected into the trained multi-task detection model to obtain the segmentation result of the conveyor belt, the idler detection box, and the foreign object detection box; S4. Fault identification, specifically including: Count the idlers on both sides of the conveyor belt according to the number of idler detection boxes. If the number of idlers on both sides is not equal, a serious fault signal is generated. Otherwise, taking the line connecting the center points of the idlers along the conveying direction as the reference line, calculate the offset distance between the center point of the area between the reference lines on both sides of the conveyor belt and the center point of the conveyor belt area. If it exceeds the preset offset threshold, an overall offset fault signal is generated. Fit the side edges of the conveyor belt into a straight line and calculate the angle between the edge line and the reference line. If it exceeds the preset angle threshold, a local torsion fault signal is generated. At the same time, detect the position coordinates of the foreign object detection box. If the coordinates are within the area between the reference lines on both sides, a foreign object presence signal is generated.

2. The detection method according to claim 1, wherein, The multi-task detection model in step S2 uses an improved A-YOLOM network model, including the following improvements: Introduce a progressive feature pyramid network in the feature fusion stage to fuse adjacent-level features and progressively introduce high-level features; Use the GSConv module to replace the standard convolution module in the convolutional layers of the backbone network and the detection neck of the model to reduce the parameters and computational complexity of the model; Adopt a fusion sliding loss function as the loss function to dynamically adjust the weights of easy and hard samples to improve the recognition ability of hard samples; Use a soft non-maximum suppression post-processing algorithm to perform confidence attenuation adjustment on overlapping detection boxes to reduce missed detections.

3. The detection method according to claim 2, wherein The expression of the fusion sliding loss function is: ； where μ is the adaptive learning sample threshold parameter and the negative sample threshold parameter.

4. The detection method according to claim 2, wherein Using the soft non-maximum suppression post-processing algorithm to perform confidence attenuation on overlapping detection boxes includes: Attenuate the confidence of overlapping low-confidence boxes according to the Gaussian distribution function. When the IOU value of the candidate box is greater than the threshold, the soft non-maximum suppression post-processing algorithm adjusts the score of the candidate box according to the following formula: ； In the formula: S i represents the original confidence score of the i th candidate box; M is the candidate box with the highest predicted score currently; b i is the candidate box waiting to be processed, N is a preset IOU threshold.

5. The detection method according to claim 2, characterized in that, The implementation of the GSConv module includes the following steps: Convert the input feature into a primary feature map with c2 / 2 channels through a standard convolutional layer; perform depthwise separable convolution processing on the primary feature map to generate a secondary feature map with c2 / 2 channels; splice the primary feature map and the secondary feature map into a fused feature map with c2 channels; perform a channel shuffle operation on the fused feature map and output a feature map with c2 channels with uniform channel distribution.

6. The detection method according to claim 1, characterized in that, The pixel distance is used to calculate the offset distance between the center point of the area between the reference lines on both sides of the conveyor belt and the center point of the conveyor belt area. If the offset distance satisfies: ； The system determines that the conveyor belt has an overall offset fault, where α is the deviation threshold, 0 < α < 1, and w is the distance between the reference line and the edge of the conveyor belt.

7. The detection method according to claim 1, wherein The calculation of the center point of the conveyor belt area in step S4 includes: Intercept the central area of the image based on a preset ratio to exclude edge interference; Perform an edge detection algorithm on the central area of the image to extract the edge contour feature points of the conveyor belt; Determine the midpoint coordinates of the conveyor belt area in the conveying direction according to the distribution of the characteristic points in the conveying direction.

8. The detection method according to claim 7, characterized in that The preset ratio is 10%.

Images