Multi-task monitoring method for running state of belt conveyor

Abstract

The invention discloses a multi-task monitoring method for the running state of a belt conveyor. The multi-task monitoring method comprises the steps that multi-task detection of the running state of the belt conveyor is achieved through a multi-task deep learning model; in the model, a backbone network is based on a backbone structure of YOLO11, and a Bottleneck residual block of an original C3K2 module is replaced by a MultiScaleEdgeInformationSelect module, so that a C3k2-Edge module is formed; in the neck network, foreign matter detection adopts an HS-PAN network, and belt deviation detection and coal flow detection adopt an HS-FPN network; in the head network, foreign matter detection, belt deviation detection and coal flow detection respectively adopt an LSCSBD detection head, a multi-scale feature fusion segmentation head and a progressive up-sampling segmentation head. The method meets the requirements of real-time performance, light weight and high precision of edge equipment in a monitoring task while adapting to multi-task parallel detection.

Description

Technical Field

[0001] The present invention relates to the technical field of task monitoring, and more particularly to a multi-task monitoring method for the running status of a belt conveyor. Background Art

[0002] In coal transportation systems for thermal power generation, belt conveyors, as connecting and transporting equipment, are widely used due to their high efficiency, low cost, and convenient operation. They have also become an indispensable core conveying equipment in bulk cargo port loading and unloading operations. However, under continuous high-load operation, belt conveyors are prone to failures such as belt deviation and foreign object obstruction, leading to frequent safety accidents, low production efficiency, and even casualties. Therefore, to ensure efficient and safe industrial production, it is particularly important to monitor the operating status of belt conveyors and prevent failures.

[0003] Currently, research on belt conveyor status monitoring in coal conveyor systems focuses primarily on belt deviation detection, belt tear detection, roller anomaly detection, foreign object detection, and coal flow detection. Traditional conveyor belt monitoring technologies are mostly based on machine vision. The paper "Belt Vision Localization Algorithm Based on Machine Vision and Belt Conveyor Deviation Detection" proposes an improved edge detection algorithm for belt positioning, addressing the difficulty in extracting belt edge lines and achieving good detection results. The paper "Research on Deviation Detection of Belt Conveyor Based on Inspection Robot and Deep Learning" utilizes Hough Linear Transformation and template matching algorithms to detect belt and roller edges. With the continuous development of computer vision and deep learning, many advanced computer vision technologies have been successfully applied to belt conveyor status monitoring, achieving significant results. Regarding belt deviation detection, the paper "A Deep Learning-Based Method for Deviation Status Detection in an Intelligent Conveyor Belt System" improves the object detection network YOLOv5 for belt edge detection and proposes a new belt deviation detection method. The paper "A novel combined method for conveyor belt deviation discrimination under complex operational scenarios" further improves the YOLOv5 model and introduces an improved semantic segmentation network, U-Net. These two methods work together to form a combined defuzzification method based on detection and segmentation, and are applied to belt deviation detection. Regarding foreign object detection, the paper "Proactive measures to prevent conveyor belt failures: Deep Learning-based faster foreign object detection" improves the backbone and core of the object detection network, YOLOv4, achieving a lightweight model and improving its feasibility for deployment on edge devices.The paper "Computer vision detection of offoreign objects in coal processing using attention CNN" introduces an attention mechanism into the classic convolutional neural network (CNN), achieving real-time and accurate segmentation of foreign objects from complex backgrounds. The paper "Improved foreign object tracking algorithm in coal for belt conveyor gangue selection robot with YOLOv7 and DeepSORT" significantly improves the detection and tracking performance of foreign objects in coal by leveraging the lightweight design of the YOLOv7 model and the feature matching mechanism of the DeepSORT algorithm. Regarding coal flow detection, more research is based on machine vision, such as the papers "Dual-field measurement system for real-time material flow on conveyor belt" and "Bulk material flow measurement based only on a smart camera fixed above a moving belt conveyor." Although these studies have achieved excellent results in their respective detection tasks, most can only complete single-task detection and fail to meet the multi-task requirements of real-time belt conveyor monitoring.

[0004] Current deep learning models for computer vision are limited to single-task monitoring and cannot perform multiple tasks simultaneously. This requires deploying multiple models and employing a multi-model parallel strategy, which requires significant computing resources and is not suitable for running on edge devices. Mask R-CNN, proposed by Facebook AI Research (FAIR) in 2016, has demonstrated excellent performance in instance segmentation, object detection, and individual keypoint detection. The You Only Look Once (YOLO) series of models, from YOLO V8 to the latest YOLO 11, supports tasks such as object detection and semantic segmentation. However, due to limitations in the model architecture, these models can only perform one of multiple tasks at a time and are unable to run multiple tasks simultaneously. Therefore, developing a multi-task model that can perform as many tasks as possible related to monitoring the operating status of belt conveyors is crucial.

[0005] In the field of autonomous driving, multi-task models are widely used. In autonomous driving systems (ADS), the functions of multi-task models are mainly divided into two types: target detection and segmentation. Among them, target detection is used for vehicle and pedestrian detection, while segmentation tasks are divided into lane line segmentation and drivable area segmentation. In addition, in autonomous driving tasks, the real-time, lightweight and high-precision performance of multi-task models are indispensable. Inspired by the multi-task models in autonomous driving technology, whether it is target detection or instance segmentation, the backbone network for feature extraction and the neck network for feature fusion are universal, without the need for the support of multiple feature extraction and fusion networks. To a certain extent, it can reduce the amount of model calculation and improve the real-time performance of the model. The only difference is in the detection head and segmentation head used for model output results, which provides us with the feasibility of building a multi-task model.

[0006] Therefore, how to adapt to the needs of multi-task parallel detection while meeting the real-time, lightweight and high-precision requirements of edge devices in the belt conveyor operation status monitoring task, and improve the practicality and feasibility of the belt conveyor intelligent status monitoring method, is an urgent problem that technical personnel in this field need to solve. Summary of the Invention

[0007] In view of the above problems, the present invention provides a multi-task monitoring method for the operating status of a belt conveyor to at least solve some of the technical problems mentioned in the above background technology.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] The present invention provides a multi-task monitoring method for the operating status of a belt conveyor, comprising the following steps:

[0010] The acquired target belt conveyor operating status monitoring data is input into the trained multi-task deep learning model, and the single target foreign body detection results, belt deviation detection results, and coal flow detection results are simultaneously output;

[0011] The multi-task deep learning model includes a backbone network, a neck network and a head network;

[0012] The backbone network is based on the backbone structure of the YOLO11 model, and the Bottleneck residual block of the original C3K2 module is replaced by the MultiScaleEdgeInformationSelect module to form a C3k2-Edge module;

[0013] In the neck network, the foreign body detection branch uses the HS-PAN network as the detection neck, and the belt deviation detection branch and the coal flow detection branch use the HS-FPN network as the segmentation neck;

[0014] In the head network, single-target foreign object detection adopts LSCSBD detection head, belt deviation detection adopts multi-scale feature fusion segmentation head, and coal flow detection adopts progressive upsampling segmentation head.

[0015] Furthermore, the MultiScaleEdgeInformationSelect module includes adaptive average pooling and EdgeEnhancer modules and DSM attention mechanism;

[0016] Pooling the input feature maps of the MultiScaleEdgeInformationSelect module into four specified output sizes through the adaptive average pooling, and extracting local information of the corresponding scales;

[0017] In the EdgeEnhancer module, average pooling is used to independently smooth the extracted local information of each scale to obtain a smoothed feature map; the input feature map is subtracted from the smoothed feature map to obtain high-frequency edge information, which is then convolved and added to the input feature map before being output to the DSM attention mechanism.

[0018] Furthermore, the DSM attention mechanism includes a space selection module and a frequency selection module;

[0019] In the space selection module:

[0020] The spatial attention map is generated by the parallel operation of average pooling and maximum pooling, which is expressed as:

[0021] F′=Conv3([AvgPool(F),MaxPool(F)])

[0022] Among them, F represents the input feature of the DSM attention mechanism; AvgPool represents average pooling; MaxPool represents maximum pooling; [,] represents the connection operation; Conv3 represents the convolution layer with a 3×3 kernel; F′ represents the spatial attention map;

[0023] The input feature F is separated into channels by depthwise separable convolution and modulated with the spatial attention map F′ to obtain the spatial selection feature F S Expressed as:

[0024]

[0025] Among them, DConvs5,7 represents the cascaded depth convolution layer with convolution size of 5×5 and 7×7; DConv3 represents the depth convolution using 3×3 convolution kernel; represents element-wise multiplication; T(F′,C) means copying the spatial attention map F′ along the channel dimension C times to match the dimension of the input feature F;

[0026] In the frequency selection module:

[0027] The spatial selection feature F is extracted by global average pooling S low-frequency features and select features F from the space s Subtract the low-frequency feature from the , and obtain the spatial selection feature F S The high-frequency characteristics of ; expressed as:

[0028]

[0029] Among them, Mean represents global average pooling; Mean(F S ) represents low-frequency features; Represents high-frequency features;

[0030] The high-frequency features and spatial selection feature F S Multiply element by element and add the multiplication result to the spatial selection feature F S Residual connection to obtain the output features of the frequency selection module Expressed as:

[0031]

[0032] The output characteristics of the frequency selection module That is the output feature of the DSM attention mechanism.

[0033] Furthermore, the HS-PAN network adds a bottom-up feature fusion path based on the top-down structure of HS-FPN to fuse deep semantic information and shallow location details.

[0034] Furthermore, the DySample dynamic upsampling operator is introduced into both the HS-PAN network and the HS-FPN network.

[0035] Furthermore, the LSCSBD detection head takes the multi-scale feature map output by the HS-PAN network as input; in the LSCSBD detection head:

[0036] Each scale feature map is passed through an independent 1×1 convolution layer to unify the number of channels, and then the context features are fused through two levels of 3×3 shared convolution layers to obtain the fused features.

[0037] After passing the fused features through multiple batch normalization layers, they are input into the bounding box regression branch and the classification branch to achieve target classification and target box positioning;

[0038] The Scale layer is introduced to dynamically scale the classified features and finally output the single-target foreign object detection results, including the target bounding box coordinates and category probability distribution.

[0039] Furthermore, both the bounding box regression branch and the classification branch adopt a shared convolution structure.

[0040] Furthermore, the loss function of the multi-task deep learning model includes a foreign body detection loss function, a roller segmentation loss function used in the belt deviation detection task, and a coal flow segmentation loss function used in the coal flow detection task.

[0041] Furthermore, the foreign object detection loss function L det Expressed as:

[0042] L det =λ BCE L BCE +λ DFL L DFL +λ CIoU L CIoU

[0043] Among them, L BCE represents the binary cross entropy loss function in the classification branch; L DFL represents the distribution focus loss function in the bounding box regression branch; L CIoU represents the completeIoU loss function in the bounding box regression branch; λ BCE Indicates L BCE The weight of λ DFL Indicates L DFL The weight of λ CIoU Indicates L CIoU The weight of

[0044] The roller segmentation loss function and the coal flow segmentation loss function are expressed in the same form, which is expressed as:

[0045] L seg =L Focal-Tversky =(1-TI) γ

[0046]

[0047] Among them, L seg represents the roller segmentation loss function or the coal flow segmentation loss function; L Focal-TverskyRepresents the FocalTversky Loss loss function; γ represents the focusing parameter, TI represents the Tversky index; TP represents the true positive example; FP represents the false positive example; FN represents the false negative example; α represents the control of the false positive penalty factor; β represents the control of the false negative penalty factor.

[0048] Furthermore, the belt deviation detection result output by the multi-task deep learning model is a binary mask image of the segmented roller area;

[0049] Based on the binary mask image of the segmented roller area, the exposed pixel areas of the left and right rollers are calculated, and the degree of deviation is quantitatively evaluated, which is expressed as:

[0050]

[0051] Among them, D represents the degree of deviation; S L Indicates the pixel area of ​​the left roller; S R is the pixel area of ​​the right roller.

[0052] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a multi-task monitoring method for the operation status of a belt conveyor, which has the following beneficial effects:

[0053] Based on the YOLO11 framework, this paper constructs a multi-task deep learning model for belt conveyor operation status monitoring for the first time, realizing parallel processing of foreign object detection, belt deviation detection and coal flow detection, significantly improving monitoring efficiency; and the multi-task deep learning model not only takes into account precision and accuracy, but also meets the real-time and lightweight requirements of edge device operation.

[0054] This paper introduces the C3k2-Edge module into the backbone network, enabling the model's backbone network to obtain high-resolution, low-level edge position information within low-resolution, high-level semantic information. This significantly enhances edge feature extraction capabilities, meeting the requirements for coal flow detection and belt deviation edge detection. By designing a multi-scale feature extraction module and introducing the DSM attention mechanism, the model's ability to capture belt edge and roller details is effectively enhanced, addressing the issue of traditional methods' insufficient edge feature extraction in complex industrial scenarios.

[0055] This paper introduces the DySample dynamic upsampling operator into the HS-PAN network to generate an efficient HS-PAN-DY feature extraction and fusion network. Furthermore, by innovatively combining the advantages of HS-FPN and PAN and introducing the dynamic upsampler DySample, this method significantly improves the efficiency of multi-scale feature fusion, reducing computational complexity while maintaining accuracy.

[0056] This paper proposes a lightweight detection head (LSCSBD), a multi-scale feature fusion segmentation head, and a progressive upsampling segmentation head. By sharing convolutional parameters and optimizing the batch normalization layer design, redundant segmentation head structures are removed, significantly reducing the number of model parameters (to only 1.79M) while maintaining detection accuracy, making the model more suitable for edge device deployment.

[0057] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0059] Figure 1 A schematic flow chart of a multi-task monitoring method for the operating status of a belt conveyor provided in an embodiment of the present invention.

[0060] Figure 2 Schematic diagram of the HD-multi model backbone network and neck network structure provided by an embodiment of the present invention.

[0061] Figure 3 A schematic diagram of the C3k2-Edge module structure provided in an embodiment of the present invention.

[0062] Figure 4 Schematic diagram of the DSM attention mechanism structure provided by an embodiment of the present invention.

[0063] Figure 5 A schematic diagram of the DySample dynamic upsampling operator structure provided in an embodiment of the present invention.

[0064] Figure 6 This is a schematic diagram of the LSCSBD detection head structure provided by an embodiment of the present invention.

[0065] Figure 7 A schematic diagram of the structure of a segmentation head provided in an embodiment of the present invention.

[0066] Figure 8 A schematic diagram of a belt conveyor provided in an embodiment of the present invention.

[0067] Figure 9 A schematic diagram of the visualization results of foreign body detection provided by an embodiment of the present invention.

[0068] Figure 10 This is a schematic diagram of the segmentation visualization results provided by an embodiment of the present invention.

[0069] Figure 11 A schematic diagram of the experimental platform structure provided by an embodiment of the present invention.

[0070] Figure 12 A schematic diagram of belt deviation measurement results provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] The embodiment of the present invention discloses a multi-task monitoring method for the running state of a belt conveyor. Figure 1 As shown, including:

[0073] The acquired target belt conveyor operating status monitoring data is input into the trained multi-task deep learning model, and the single target foreign body detection results, belt deviation detection results, and coal flow detection results are simultaneously output;

[0074] The multi-task deep learning model includes a backbone network, a neck network, and a head network;

[0075] The backbone network is based on the backbone structure of the YOLO11 model, replacing the Bottleneck residual block of the original C3K2 module with the MultiScaleEdgeInformationSelect module to form the C3k2-Edge module;

[0076] In the neck network, the foreign body detection branch uses the HS-PAN network as the detection neck, and the belt deviation detection branch and the coal flow detection branch use the HS-FPN network as the segmentation neck;

[0077] In the head network, the LSCSBD detection head is used for single-target foreign object detection, the multi-scale feature fusion segmentation head is used for belt deviation detection, and the progressive upsampling segmentation head is used for coal flow detection.

[0078] 1. Next, we will explain each part of the multi-task deep learning model (HD-Multi model) in detail:

[0079] like Figure 2As shown, the HD-Multi model is a single-stage model. First, the multi-task deep learning model of the present invention shares the multi-scale feature extraction structure (backbone) and feature selection structure (i.e., HS-PAN neck network) to reduce the model size and unnecessary computational overhead. In the foreign object detection task branch, the present invention uses a PAN structure that can handle multi-scale targets and precise positioning. In the detection head part, the present invention uses a single-target lightweight detection head LSCSBD specially designed for foreign object detection tasks. In the belt deviation detection task and the coal flow detection task, the present invention uses a simpler FPN (FeaturePyramidNetwork) structure; in the segmentation head part, the belt deviation task uses a multi-level feature fusion segmentation head suitable for small target segmentation, while the coal flow detection task uses a progressive upsampling segmentation head.

[0080] 1. Backbone network:

[0081] For the backbone design of the HD-multi model, the embodiment of the present invention refers to the backbone network of the YOLO11 model, which has been widely proven to be an excellent choice for feature extraction capabilities. The difference is that since the backbone network in YOLO11 does not do additional work for the extraction of edge information, the edge information extraction capability is insufficient. To solve this problem, the embodiment of the present invention proposes a MutilScaleEdgeInformationSelect module to replace the Bottleneck residual block in the backbone C3K2, forming a C3k2-Edge module, such as Figure 3 The main purpose of the C3k2-Edge module is to extract features from different scales, highlight edge information, and integrate these multi-scale features together. Then, it efficiently selects key features that are highly relevant to the target task, and finally outputs enhanced features through the convolutional layer. The edge feature extraction of the C3k2-Edge module can be divided into three steps: multi-scale feature extraction, edge information enhancement, and feature selection. Specifically:

[0082] (1) Multi-scale feature extraction:

[0083] AdaptiveAvgPool is used to pool the input feature maps of the MutilScaleEdgeInformationSelect module into four specified output sizes, extracting local information of different sizes and capturing multi-level features of the image.

[0084] (2) Edge information enhancement:

[0085] For edge information enhancement, the embodiment of the present invention designs an EdgeEnhancer module specifically for extracting edge information, thereby enhancing the network's sensitivity to edges, which is important for visual tasks (such as object detection and semantic segmentation). In the EdgeEnhancer module, AvgPool (average pooling) is used to independently smooth the local information of each scale extracted by the multi-scale feature extraction part, extract its low-frequency information, and record it as a smoothed feature map. The input feature map is subtracted from the smoothed feature map to obtain enhanced high-frequency edge information, and the enhanced edge information is further processed by a convolution operation. The processed edge information is added to the input feature map to form an enhanced output.

[0086] (3) Feature selection:

[0087] In order to enable the backbone network to extract effective edge information and reduce the interference of useless edge noise on the target task, the embodiment of the present invention introduces a Dual-Domain Selection Mechanism (DSM) that focuses on the ability of more important edge regions. Figure 4 As shown in Figure 2, the DSM attention mechanism consists of a spatial selection module (SpatialSelection Module, SSM) and a frequency selection module (Frequency Selection Module, FSM); where:

[0088] 1) In the space selection module:

[0089] First, a spatial attention map is generated by the parallel operation of average pooling and maximum pooling, which is expressed as:

[0090] F′=Conv3([AvgPool(F),MaxPool(F)])

[0091] Among them, F represents the input feature of the DSM attention mechanism, and F∈R H×W×C ; AvgPool represents average pooling; MaxPool represents maximum pooling; [,] represents a concatenation operation; Conv3 represents a convolutional layer with a 3×3 kernel; F′ represents a spatial attention map, and F′∈R H×W×C ; Based on this, the spatial attention map F′ contains the identification of the blurred area;

[0092] Secondly, the input feature F is channel-separated by depth-wise convolutions and modulated with the spatial attention map F′ to obtain the spatial selection feature F S Expressed as:

[0093]

[0094] Among them, DConvs5,7 represents the cascaded depth convolution layer with convolution size of 5×5 and 7×7; DConv3 represents the depth convolution using 3×3 convolution kernel; represents element-wise multiplication; T(F′,C) means copying the spatial attention map F′ along the channel dimension C times to match the dimension of the input feature F;

[0095] Finally, the obtained spatial selection feature F S Input into the frequency selection module for frequency selection;

[0096] 2) In the frequency selection module:

[0097] The main function of the frequency selection module is to enhance high-frequency information (such as edges and textures) in the frequency domain, because the main difference between blurred images and clear images is usually in the high-frequency part; specifically:

[0098] First, the spatial selection feature F is extracted by global average pooling (Mean Filter) S low-frequency features and select features F from the space s Subtract the low-frequency features from the mid-point to obtain the spatial selection feature F S The high-frequency characteristics of ; expressed as:

[0099]

[0100] Among them, Mean represents global average pooling; Mean(F S ) represents the global average pooling of the channel dimension, representing low-frequency features; Represents high-frequency features, including edge and texture information;

[0101] Secondly, the high-frequency features and spatial selection feature F S Multiply element by element and add the multiplication result to the spatial selection feature F S Residual connection to obtain the output features of the frequency selection module Expressed as:

[0102]

[0103] The output characteristics of the frequency selection module That is the output feature of the DSM attention mechanism; Figure 4 in and They represent element-wise multiplication, element-wise addition, and element-wise subtraction respectively; Figure 4 The Broadcast in represents a broadcast operation, which is used to expand the low-frequency signal after mean filtering to the same size as the input feature.

[0104] In summary, the MutilScaleEdgeInformationSelect module effectively enhances edge information from multi-scale features through three steps: multi-scale feature extraction, edge information enhancement, and feature selection. It also rationally selects the key features required for the target task through the DSM attention mechanism, effectively improving the model's ability to extract belt edges and roller details, enhancing the edge learning ability of the multi-task model network, and providing a reliable basis for deviation detection.

[0105] 2. Neck network:

[0106] In the multi-task detection model for the operating status of a belt conveyor, the embodiment of the present invention first introduced a lightweight neck network HS-FPN. However, due to the uncertainty of the position and the diversity of categories of foreign objects in the coal flow, high-resolution position information and low-resolution semantic information are difficult to learn, which is more difficult than the two tasks of belt deviation detection and coal flow detection. Therefore, the present invention uses a PAN structure with stronger feature fusion and precise positioning capabilities to improve and deepen the HS-FPN network in the foreign object detection neck to improve the model's foreign object detection performance. In the segmentation neck, an undeepened FPN structure is used to achieve the purpose of lightweighting the model while meeting the accuracy requirements.

[0107] 2.1、Neck detection:

[0108] First, in the embodiment of the present invention, the neck is detected using the HS-PAN network, such as Figure 2As shown. A common HS-FPN network is mainly divided into two modules: a feature selection module and a feature fusion module. The feature selection module extracts important information from feature maps of different scales through a channel attention mechanism (ChannelAttention_HS-FPN) and a dimension matching mechanism (DM), combined with global average pooling and maximum pooling operations. It also determines channel weights through a Sigmoid activation function, ultimately generating an optimized feature map. The feature fusion module, on the other hand, combines filtered low-level features with scaled high-level features through a selective feature fusion mechanism, thereby enhancing the model's ability to express small features. The HS-FPN network is a typical top-down feature pyramid structure (FPN). This structure lacks the ability to retain the position information of low-level features and precise positioning, and therefore does not meet the accuracy requirements for foreign object detection. Therefore, in the embodiment of the present invention, a bottom-up (downsampling) feature fusion path is added to the top-down infrastructure of the HS-FPN network, fusing deep semantic information (P5) with shallow position details (P3), improving the multi-scale target detection capability and forming an HS-PAN network with stronger feature fusion capability and more accurate position positioning capability.

[0109] It is worth mentioning that in order to solve the shortcomings of the traditional Nearest Neighbor Upsampling (nearest neighbor upsampling) of insufficient upsampling capability and large amount of calculation, the embodiment of the present invention uses the ultra-lightweight and efficient DySample (dynamic upsampling) in the neck to generate an efficient HS-PAN-DY feature extraction and fusion network to reduce information loss in the upsampling process (especially important for small targets) and achieve the purpose of lightweight. DySample is an upsampling operator based on dynamic sampling. Its core idea is to resample continuous feature maps through content-aware sampling points, thereby achieving efficient feature upsampling. The specific implementation process of DySample is as follows: Figure 5 As shown; Figure 5 In the figure, sH represents the height of the feature map after upsampling; sW represents the width of the feature map after upsampling; s represents the upsampling ratio; H represents the original height of the input feature map; W represents the original width of the input feature map; 2g represents the number of channels of the dynamic offset; "2" represents the two-dimensional coordinate offset of each sampling point; g represents the number of groups; Pixel Shuffle represents pixel shuffling, which is used to reorganize the information in the channel dimension into the spatial dimension to achieve efficient upsampling; "0.25" represents the static scaling factor; 0.5σ represents the dynamic scaling factor;

[0110] Figure 5Here, χ, χ′, O, and G represent input features, upsampled features, generated offsets, and original grids, respectively. The sampling set is generated by the sampling point generator, which resamples the input features using the grid sample function. In the generator, the sampling set is the sum of the generated offsets and the original grid positions. The upper box shows a version with a “static scope factor”, where the offsets are generated via a linear layer. The lower box describes a version with a “dynamic scope factor”, where a scope factor is first generated and then used to modulate the offset, and “σ” represents the sigmoid function.

[0111] 2.2. Split the neck:

[0112] In terms of the design choice of the segmentation neck structure, experimental results show that the PAN architecture based on the bidirectional fusion path (such as YOLO11 and HS-PAN models) has better segmentation accuracy in belt deviation detection and coal flow detection tasks than the HS-FPN model with the traditional FPN structure. However, considering the design requirements of the lightweight model, this improvement in segmentation accuracy is accompanied by a significant increase in the number of parameters, which is difficult to accept in engineering applications. Therefore, the embodiment of the present invention ultimately adopts a lighter HS-FPN network as the segmentation neck structure, and introduces the key lightweight DySample upsampling operator to prevent the loss of feature information. The design generates a P3-scale feature map through a top-down feature selection module and a feature fusion module and outputs it to the segmentation head. Experimental verification shows that for detection tasks with relatively stable position information, this lightweight design can meet the accuracy requirements while maintaining high computational efficiency.

[0113] 3. Head network:

[0114] Regarding the design of the head, the embodiments of the present invention use different heads for different tasks. Experimental research has found that, for both the more accurate YOLO11 model and the more lightweight HS-FPN model, the head parameters account for a quarter of the total model parameters. This is due to the sudden increase in model parameters caused by the decoupling head design. Therefore, the present invention addresses this issue to achieve a more lightweight model.

[0115] 3.1、Detection head design:

[0116] In the study of single-target foreign body detection tasks, we found that the YOLO series detectors usually assign independent detection heads to different feature levels. This design will lead to reduced model parameter utilization. In theory, objects detected by different feature levels should have similar feature distributions at relative scales. This view is supported by mainstream detectors (such as RetinaNet and FCOS) - they all use parameter-sharing detection head designs. Based on this, the detection head in the embodiment of the present invention adopts a shared convolution structure to significantly reduce the number of parameters. However, in order to avoid the performance of the batch normalization (BN) layer being affected by the running mean cumulative error when sharing parameters, the embodiment of the present invention only shares parameters in the convolution layer, while keeping the BN layer independent. This makes the model lighter while also ensuring the performance of the detection head.

[0117] The structure of LSCSBD detection head is as follows Figure 6 As shown. The structure takes the multi-scale feature maps (P3, P4, P5) output by the HS-PAN network as input; first, the number of channels of each scale feature map is unified through an independent 1×1 convolution layer to reduce computational redundancy and improve efficiency; then, two levels of 3×3 shared convolution layers (parameter sharing) are used to extract local spatial features, enhance semantic expression capabilities, perform context feature fusion, and obtain fusion features; at the same time, for low-resolution feature maps such as P4 / P5, its large receptive field characteristics can further enhance the detection capability of large targets; it is worth noting that although the convolution layer parameters are shared, each level of convolution is followed by an independent A batch normalization (BN) layer is used to avoid sliding statistic bias caused by differences in feature distributions at different scales. At the output stage of the detection head, both the bounding box regression branch and the classification branch adopt a shared convolutional structure to improve parameter utilization and achieve target classification and target box positioning. To address the problem of target size differences in multi-scale detection, a learnable Scale layer is introduced to dynamically scale features and ultimately output single-target foreign object detection results, including target bounding box coordinates and category probability distribution. The LSCSBD detection head optimizes computational complexity and efficiency while maintaining detection accuracy, making it suitable for real-time single-target foreign object detection tasks.

[0118] 3.2, Split head design:

[0119] Split head design Figure 7 As shown. For belt deviation detection and coal flow detection tasks, traditional methods mainly rely on high-level features (P4 / P5). Although they are rich in semantic information, they have two significant drawbacks: first, the large number of parameters is not conducive to model lightweight; second, low-resolution features lead to the loss of detailed information of small targets (such as rollers). To this end, the embodiment of the present invention proposes a multi-scale adaptive segmentation head:

[0120] (1) Small target segmentation (belt deviation roller segmentation):

[0121] Introducing the low-level features of the backbone network (P1 / P2), P1 and P2 correspond to Figure 2 The first convolution (Conv) and the second convolution (Conv) of the backbone enhance the ability to preserve spatial details through a multi-level feature fusion mechanism. Specifically, a cross-scale feature aggregation module is designed to fuse the high-resolution detail information of P1 / P2 with the semantic information of P3 to form a multi-scale feature fusion segmentation head, thereby improving the accuracy of small object segmentation. Figure 7 As shown in (a).

[0122] (2) Large target segmentation (coal flow segmentation):

[0123] For large-scale target detection tasks such as coal flow, in order to balance accuracy and efficiency, a progressive upsampling segmentation head is used, which only fuses P3-P5 features, and then gradually restores the spatial resolution through cascaded deconvolution (upsampling). While ensuring the accuracy of coal flow region segmentation, it reduces the computational overhead brought by P1 and P2 branches. Figure 7 (b) shown.

[0124] 4. Loss function:

[0125] The embodiment of the present invention adopts an end-to-end training method and a multi-task loss function. Specifically, the loss function L of the present invention consists of three parts: a detection and two segmentation parts, including a foreign object detection loss function, a roller segmentation loss function used in the belt deviation detection task, and a coal flow segmentation loss function used in the coal flow detection task. The formula is as follows:

[0126] L=L det +L seg1 +L seg2

[0127] Among them, L det represents the foreign body detection loss function; L seg1 represents the roller segmentation loss function used in the belt deviation detection task, L seg2 represents the coal flow segmentation loss function used in the coal flow detection task;

[0128] (1) Detection tasks:

[0129] For the detection task, the loss function is divided into two main branches: classification branch and bounding box regression branch; the classification branch includes binary cross entropy loss; the bounding box branch includes distribution focus loss and completeIoU loss; based on this, the foreign object detection loss function L det Expressed as:

[0130] Ldet =λ BCE L BCE +λ DFL L DFL +λ CIoU L CIoU

[0131] Among them, L BCE represents the binary cross entropy loss function in the classification branch; L DFL represents the distribution focus loss function in the bounding box regression branch; L CIoU represents the completeIoU loss function in the bounding box regression branch; λ BCE Indicates L BCE The weight of λ DFL Indicates L DFL The weight of λ CIoU Indicates L CIoU The weight of

[0132] (2) Segmentation task:

[0133] For both segmentation tasks, the present invention uses the same loss function, Focal Tversky Loss. FocalTversky Loss is a combination of Tversky Loss and Focal Loss. By introducing a dynamic modulation factor, it simultaneously solves the problems of class imbalance and difficult sample learning. Its design goal is to enhance the gradient contribution to difficult samples (such as small objects and fuzzy boundaries) while retaining Tversky Loss's flexible control over false positives (FP) and false negatives (FN). Specifically, the loss function for roller segmentation or coal flow segmentation is expressed as:

[0134] L seg =L Focal-Tversky =(1-TI) γ

[0135]

[0136] Among them, L seg represents the roller segmentation loss function or the coal flow segmentation loss function; L Focal-Tversky Represents the FocalTversky Loss loss function; γ represents the focusing parameter, TI represents the Tversky index; TP represents the true positive example; FP represents the false positive example; FN represents the false negative example; α represents the control of the false positive penalty factor; β represents the control of the false negative penalty factor.

[0137] 2. Next, based on the above multi-task deep learning model, the multi-task detection in this application is specifically explained.

[0138] 1. Single target foreign body detection:

[0139] For the foreign body detection task, the present invention uses target detection in conventional computer vision to achieve this goal, specifically based on the above-mentioned multi-task deep learning model HD-Multi. Since conveyor belts may carry various foreign objects such as steel bars, iron cables, iron blocks, wood, plastic pipes, etc. when transporting coal, these foreign objects will affect the normal operation of crushers, ball mills and conveyor belts, reducing production efficiency. Therefore, when studying foreign body detection, the present invention classifies all objects other than coal that appear on the conveyor belt as "foreign bodies" and performs single target detection, eliminating the tedious classification steps, which is conducive to the efficient operation of foreign body detection tasks.

[0140] 2. Belt deviation detection:

[0141] In order to solve the problems of insufficient real-time performance and strong hardware dependence of existing belt deviation detection methods, the embodiment of the present invention proposes an engineering-friendly belt deviation discrimination method based on semantic segmentation to simplify the detection process and improve efficiency. This method is based on the symmetrical structural characteristics of the belt conveyor, such as Figure 8 As shown in the figure. The conveying process is supported by rollers and drives the belt. When the belt deviates, the bearing area of ​​the left and right rollers will change accordingly. Specifically, when the belt deviates to the left, the contact area between the left roller and the belt increases, resulting in a decrease in its exposed area; conversely, when the belt deviates to the right, the exposed area of ​​the right roller decreases. Based on this rule, this paper obtains a binary mask image of the roller area through the segmentation task, calculates the exposed pixel area of ​​the left and right rollers, and quantitatively evaluates the degree of deviation, expressed as:

[0142]

[0143] Among them, D represents the degree of deviation; S L Indicates the pixel area of ​​the left roller; S R is the pixel area of ​​the right roller; the embodiment of the present invention is based on the pixel area analysis of the roller segmentation mask, and the new deviation calculation formula proposed has achieved a measurement error of only 5% in actual application, providing reliable technical support for engineering practice.

[0144] 3. Coal flow detection:

[0145] For the coal flow detection task, the embodiment of the present invention also uses the segmentation task to segment the coal flow on the conveyor belt. The purpose of doing so is to facilitate subsequent coal flow calculation and coal flow spillage detection.

[0146] 3. Next, the above content is demonstrated through relevant experiments and results.

[0147] 1. Dataset preparation:

[0148] Due to the lack of public data sets for belt conveyor operation status monitoring, the embodiment of the present invention also developed a new data set. The data set in this article comes from the actual operation images of the belt conveyor in a power plant. In order to prevent overfitting of training and improve the generalization ability of the model under extreme conditions (occlusion, fog, rain, dimness), the present invention uses a data enhancement algorithm to expand the original 400 data sets to 1200, and uses labelme software to label foreign objects, left and right rollers, and coal flow, and record them as four categories of labels: "foreign, left, right, coal". Finally, the data set is divided into training set, validation set, and test set in a ratio of 8:1:1.

[0149] 2. Operating environment configuration and training parameters:

[0150] The computer operating system is Windows 10, and the hardware configuration used is an AMD Ryzen 97950X 16-Core Processor and an NVIDIA GeForce RTX 4080 graphics card with 16GB of video memory. Pytorch 2.0.0, CUDA 11.8, and Python 3.9 are used for programming. The training parameters are shown in Table 1:

[0151] Table 1 Training parameter configuration

[0152] Training parameters Numerical Enter the image size 640×640 Initial learning rate 0.01 Optimizer SGD Number of training batches 4 Number of iteration cycles 300 Pre-trained weights YOLO11n.pt

[0153] 3. Model evaluation indicators:

[0154] (1) For the single-class foreign body detection task, the present invention uses recall (R), AP50 (Average Precision at IoU = 0.5) and mAP50 (mean Average Precision at IoU = 0.5) as evaluation indicators of model accuracy. These indicators are widely recognized and accepted in detection tasks. Recall rate indicates the proportion of positive samples (true targets) correctly detected by the model to all true positive samples. The formula is:

[0155]

[0156] Among them, TP represents true positive examples; FP represents false positive examples;

[0157] (2) AP50 is a commonly used evaluation metric in object detection, representing the average precision (AP) when the IoU (intersection over union) threshold is 0.5, that is, the area under the precision (P)-recall (R) curve. mAP50 is the average AP50 of all detection categories, and the formula is:

[0158]

[0159] Where n represents the number of all inspection categories; AP i represents the average precision of the i-th inspection category;

[0160] (3) For the segmentation task, the embodiment of the present invention uses the mean intersection over union (mIoU) to measure the accuracy of the model segmentation task. The specific calculation formula is as follows:

[0161]

[0162] Among them, n cs Represents the segmentation category; IoU represents the degree of overlap between the true value and the predicted value;

[0163] (4) To comprehensively evaluate the engineering applicability of the model for deployment on edge devices with limited computing resources, the embodiment of the present invention uses the model parameter quantity (unit: M), floating-point operation quantity (unit: Gb), model weight file size (unit: MB) and average frame rate (FPS) to systematically verify the lightweight degree of the model and the feasibility of edge deployment;

[0164] The above evaluation indicators correspond to: model complexity, computational efficiency, deployment cost, and real-time performance of end-to-end inference.

[0165] 4. Experimental results and analysis:

[0166] 4.1. Lightweight and real-time comparison experiment:

[0167] To verify the improved performance (lightweight and real-time) of the HD-Multi model, this embodiment of the present invention selected its basic framework YOLO11 as the benchmark model and conducted quantitative comparisons along the following four dimensions. The comparison results are shown in Table 2. All frame rate tests were performed on the same device with a batch size of 1.

[0168] Table 2 Baseline comparison results

[0169] Model Parameters Gflops (Gb) Size(Mb) FPS (BatchSize = 1) YOLO11n(det) 2582347 6.3 5.5 212.1 YOLO11n(seg) 2835348 10.4 5.7 228.5 HD-Multi 1792214 20 4.5 157.1

[0170] Quantitative evaluation results show that the HD-Multi model significantly reduces model complexity while maintaining efficient reasoning capabilities by sharing the backbone network and neck feature extraction structure. Specifically, in terms of parameter efficiency, the total number of model parameters is reduced to 1.79M, which is 0.79M (30.6%) and 1.04M (36.7%) less than the single-task baseline models YOLOv11n(det) and YOLOv11n(seg), respectively. In terms of computational efficiency, although the floating-point operations are about twice that of YOLOv11n(seg), it should be noted that HD-Multi simultaneously completes one target detection and two semantic segmentation tasks, and its multi-task processing performance is significantly better than the simple superposition of single-task models. In terms of deployment cost, the HD-Multi model weight file size is reduced to 4.5Mb, which also proves that the HD-Multi model has a lower deployment cost than the YOLO11 model. In terms of inference performance, while completing multi-task detection of conveyor belts, it still maintains an inference speed of 157.1FPS, which is only about 70FPS lower than the single-task model.

[0171] 4.2 Ablation Experiment

[0172] Based on the YOLO11 framework, this paper proposes a multi-task lightweight model HD-Multi for belt conveyor operation status detection. In response to the dual requirements of model accuracy and efficiency in industrial scenarios, the embodiment of the present invention innovatively designed the C3k2-Edge, HS-PAN-DY neck, and LSCSBD detection heads. To verify the contribution of each module, the embodiment of the present invention designed a systematic ablation experiment, gradually introducing each improved module on the basic multi-task model. The experimental results are shown in Table 3. After adding the C3k2-Edge module to the basic model, the model significantly improves the detection accuracy while maintaining its lightweight, increasing both mAP50 and recall by 1.2 percentage points, reducing the number of parameters by 0.42M (a decrease of 16%), and increasing the inference speed by 11.6FPS (+14%). This result verifies the effectiveness of the edge feature enhancement mechanism in belt conveyor edge detection. Based on the C3k2-Edge module, the HS-PAN-DY neck was introduced into the model. Although this resulted in a 1.4% decrease in mAP50 and 1.6% in recall, respectively, it also significantly improved model efficiency, including a further reduction of 0.67M parameters (a 25.8% decrease) and a 20% increase in FPS (+15.4%). This trade-off between accuracy and efficiency is acceptable in industrial inspection, especially for scenarios with strict real-time requirements. Adding our LSCSBD detection head, specifically designed for single-target foreign object detection, to the above foundation, the model achieved improved accuracy (mAP50 +0.7%, recall +0.6%), a further 0.13M parameter reduction (a cumulative total reduction of 32%), and a 7% increase in FPS (a cumulative +39 FPS), while maintaining its lightweight advantage.

[0173] Table 3 Ablation experiments

[0174]

[0175] In summary, experimental results demonstrate that through inter-module collaborative optimization, HD-Multi achieves an optimal balance between accuracy (mAP5082.5%), efficiency (157.1 FPS), and parameter size (1.79M). Compared to the original basic multi-task model, it reduces the number of parameters by 32% while maintaining comparable detection performance. This design provides a reliable solution for embedded deployment of industrial inspection equipment.

[0176] 4.3 Accuracy comparison experiment:

[0177] To further illustrate the performance superiority of the multi-task model HD-Multi proposed in this embodiment of the present invention, we compared the HD-Multi model with the mainstream single-task object detection models, Faster R-CNN (two-stage) and YOLO11 (one-stage), and the single-task segmentation models, Mask R-CNN (two-stage) and YOLO11-seg (one-stage). The comparative analysis covers average precision and average intersection-over-union (IoU) ratio.

[0178] The comparison results of the foreign body detection task indicators are shown in Table 4. From the quantitative results, it can be seen that our multi-task model shows the best performance in the mAP50 indicator, which shows that our model has excellent accuracy in the foreign body detection task.

[0179] Table 4 Foreign matter detection results

[0180] Model mAP50 (%) YOLO11n(det) 0.766 Faster R-CNN 0.724 HD-Multi 0.814

[0181] Table 5 shows the performance comparison of roller segmentation in the belt deviation detection task, and Table 6 shows the performance comparison of coal flow segmentation metrics in the coal dust detection task. Experimental results show that the proposed HD-Multi multi-task model achieves slightly lower accuracy in roller and coal flow segmentation than the single-task models YOLO11n (seg) and Mask R-CNN. This performance difference stems from our lightweight design in the model's neck and head networks, which involves making appropriate trade-offs within an acceptable accuracy range to ensure that the model fully meets lightweight requirements.

[0182] Table 5 Roller segmentation results

[0183] Model mIoU YOLO11n(seg) 0.90 MaskR-CNN 0.892 HD-Multi 0.864

[0184] Table 6 Coal flow segmentation results

[0185] Model mIoU YOLO11n(seg) 0.991 MaskR-CNN 0.99 HD-Multi 0.956

[0186] 5. Visualization:

[0187] The visualization results of the HD-Multi multi-task model proposed in this embodiment of the present invention are as follows: Figure 9 、 Figure 10 As shown. Among them, Figure 9 Figures (a), (b), and (c) are the visualization results of foreign body detection by YOLO11(det), Faster R-CNN, and HD-Multi, respectively. The visualization results show that the model proposed in the embodiment of the present invention exhibits high detection accuracy and excellent generalization ability in complex environments, which is crucial for foreign body detection during the operation of belt conveyors in complex environments. Figure 9As shown in the figure, in samples with severe noise and interference, compared with YOLO11(det), this model not only has higher prediction confidence, but also generates more accurate object detection boxes. In addition, although the confidence of this model is slightly lower than that of Faster R-CNN in some samples, Faster R-CNN has false detections in complex environments, indicating that the HD-Multi multi-task model has stronger model generalization and environmental adaptability than the Faster R-CNN model.

[0188] Figure 10 Figures (a), (b), and (c) in the middle are the visualization results of YOLO11n(seg), Mask R-CNN, HD-Multi roller and coal flow segmentation, respectively. The visualization analysis results show that although the model proposed in the embodiment of the present invention is slightly lower than the YOLO11n(seg) and Mask R-CNN baseline models in the generation accuracy of the segmentation mask, it shows significant advantages in processing geometric features. Specifically, this model can generate smoother and more continuous segmentation boundaries in the sharp corners and edge areas of the target. In addition, in terms of model generalization performance, compared with the false detection phenomenon of Mask R-CNN in complex scenarios, this model shows better robustness and environmental adaptability.

[0189] 6. Belt deviation test:

[0190] In order to verify the belt deviation detection method based on roller segmentation mask pixel area proposed in the embodiment of the present invention, we built a belt conveyor experimental platform, such as Figure 11 The belt used in the experiment was an NN-300(L) nylon cord rubber belt with a tensile strength of 1200 N / mm, a total length of 253 m, a width of 50 cm, and a thickness of 8 mm. The cover rubber was 1.5 mm thick and had a tensile strength of 15 MPa. The maximum operating speed of the belt was 3 m / s. A 1330M-A-I1 camera was used for image acquisition.

[0191] In the experiment, the embodiment of the present invention sets the following Figure 12 The degree of deviation is shown. Figure 12 The measured data of the belt conveyor roller position shown in Figure (a) shows that the belt is deviating to the right, with an actual deviation of 2 cm and a deviation degree of 40%. Figure 12Figure (b) shows the masked pixel areas of the left and right idlers obtained using the multi-task model HD-Multi. To verify the accuracy of the proposed belt deviation method, we used the proposed visual inspection method to process and analyze the captured images. The final detection result was "right deviation" with a deviation of 42%. Experimental data showed that the algorithm's measurement results were highly consistent with the actual deviation (with a relative error of only 5%), fully verifying the measurement accuracy of the method provided by this embodiment of the present invention.

[0192] In summary, the embodiment of the present invention provides a multi-task monitoring method for the operating status of a belt conveyor, in which the multi-task deep learning model, with its excellent real-time performance and lightweight characteristics, helps to realize intelligent real-time monitoring of the operating status of the conveyor, and shows significant application value in the belt conveyor status monitoring of the coal transportation system of a thermal power plant, providing an effective technical solution for the intelligent upgrade of the conveyor system in industrial scenarios.

[0193] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0194] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-task monitoring method for the operation status of a belt conveyor, characterized in that: The steps include: The acquired target belt conveyor operating status monitoring data is input into the trained multi-task deep learning model, and the single target foreign body detection results, belt deviation detection results, and coal flow detection results are simultaneously output; The multi-task deep learning model includes a backbone network, a neck network and a head network; The backbone network is based on the backbone structure of the YOLO11 model, and the Bottleneck residual block of the original C3K2 module is replaced by the MultiScaleEdgeInformationSelect module to form a C3k2-Edge module; In the neck network, the foreign body detection branch uses the HS-PAN network as the detection neck, and the belt deviation detection branch and the coal flow detection branch use the HS-FPN network as the segmentation neck; In the head network, single-target foreign object detection adopts LSCSBD detection head, belt deviation detection adopts multi-scale feature fusion segmentation head, and coal flow detection adopts progressive upsampling segmentation head.

2. A multi-task monitoring method for the operation status of a belt conveyor according to claim 1, characterized in that: The MultiScaleEdgeInformationSelect module includes adaptive average pooling and EdgeEnhancer modules and DSM attention mechanism; Pooling the input feature maps of the MultiScaleEdgeInformationSelect module into four specified output sizes through the adaptive average pooling, and extracting local information of the corresponding scales; In the EdgeEnhancer module, average pooling is used to independently smooth the extracted local information of each scale to obtain a smoothed feature map; the input feature map is subtracted from the smoothed feature map to obtain high-frequency edge information, which is then convolved and added to the input feature map before being output to the DSM attention mechanism.

3. A multi-task monitoring method for the operation status of a belt conveyor according to claim 2, characterized in that: The DSM attention mechanism includes a spatial selection module and a frequency selection module; In the space selection module: The spatial attention map is generated by the parallel operation of average pooling and maximum pooling, which is expressed as: F′=Conv3([AvgPool(F),MaxPool(F)]) Among them, F represents the input feature of the DSM attention mechanism; AvgPool represents average pooling; MaxPool represents maximum pooling; [,] represents the connection operation; Conv3 represents the convolution layer with a 3×3 kernel; F′ represents the spatial attention map; The input feature F is separated into channels by depthwise separable convolution and modulated with the spatial attention map F′ to obtain the spatial selection feature F S Expressed as: Among them, DConvs5,7 represents the cascaded depth convolution layer with convolution size of 5×5 and 7×7; DConv3 represents the depth convolution using 3×3 convolution kernel; represents element-wise multiplication; T(F′,C) means copying the spatial attention map F′ along the channel dimension C times to match the dimension of the input feature F; In the frequency selection module: The spatial selection feature F is extracted by global average pooling S low-frequency features and select features F from the space s Subtract the low-frequency feature from the , and obtain the spatial selection feature F S The high-frequency characteristics of ; expressed as: Among them, Mean represents global average pooling; Mean(F S ) represents low-frequency features; Represents high-frequency features; The high-frequency features and spatial selection feature F S Multiply element by element and add the multiplication result to the spatial selection feature F S Residual connection to obtain the output features of the frequency selection module Expressed as: The output characteristics of the frequency selection module That is the output feature of the DSM attention mechanism.

4. A multi-task monitoring method for the operation status of a belt conveyor according to claim 1, characterized in that: The HS-PAN network adds a bottom-up feature fusion path based on the top-down structure of HS-FPN to fuse deep semantic information and shallow location details.

5. A multi-task monitoring method for the operation status of a belt conveyor according to claim 1, characterized in that: The DySample dynamic upsampling operator is introduced into both the HS-PAN network and the HS-FPN network.

6. A multi-task monitoring method for the operation status of a belt conveyor according to claim 1, characterized in that: The LSCSBD detection head takes the multi-scale feature map output by the HS-PAN network as input; in the LSCSBD detection head: Each scale feature map is passed through an independent 1×1 convolution layer to unify the number of channels, and then the context features are fused through two levels of 3×3 shared convolution layers to obtain the fused features. After passing the fused features through multiple batch normalization layers, they are input into the bounding box regression branch and the classification branch to achieve target classification and target box positioning; The Scale layer is introduced to dynamically scale the classified features and finally output the single-target foreign object detection results, including the target bounding box coordinates and category probability distribution.

7. A multi-task monitoring method for the operation status of a belt conveyor according to claim 6, characterized in that: Both the bounding box regression branch and the classification branch adopt a shared convolution structure.

8. A multi-task monitoring method for the operation status of a belt conveyor according to claim 6, characterized in that: The loss functions of the multi-task deep learning model include a foreign body detection loss function, a roller segmentation loss function used in the belt deviation detection task, and a coal flow segmentation loss function used in the coal flow detection task.

9. A multi-task monitoring method for the operation status of a belt conveyor according to claim 8, characterized in that: The foreign body detection loss function L det Expressed as: L det =λ BCE L BCE +λ DFL L DFL +λ CIoU L CIoU Among them, L BCE represents the binary cross entropy loss function in the classification branch; L DFL represents the distribution focus loss function in the bounding box regression branch; L CIoU represents the completeIoU loss function in the bounding box regression branch; λ BCE Indicates L BCE The weight of λ DFL Indicates L DFL The weight of λ CIoU Indicates L CIoU The weight of The roller segmentation loss function and the coal flow segmentation loss function are expressed in the same form, which is expressed as: THE seg =L Focal-Tversky =(1-TI) γ Among them, L seg represents the roller segmentation loss function or the coal flow segmentation loss function; L Focal-Tversky Represents the Focal TverskyLoss loss function; γ represents the focusing parameter, TI represents the Tversky index; TP represents the true positive example; FP represents the false positive example; FN represents the false negative example; α represents the control of the false positive penalty factor; β represents the control of the false negative penalty factor.

10. A multi-task monitoring method for the operation status of a belt conveyor according to claim 6, characterized in that: The belt deviation detection result output by the multi-task deep learning model is a binary mask image of the segmented roller area; Based on the binary mask image of the segmented roller area, the exposed pixel areas of the left and right rollers are calculated, and the degree of deviation is quantitatively evaluated, which is expressed as: Among them, D represents the degree of deviation; S L Indicates the pixel area of ​​the left roller; S R is the pixel area of ​​the right roller.

Images