Dust detection method and system based on multi-task learning model

Abstract

The invention relates to a dust detection method and system based on a multi-task learning model, the system comprises a sensing measurement module, a model training module, a model reasoning module and a visualization and over-limit early warning module which are mutually matched, the sensing measurement module is used for collecting image data and dust concentration data in a mine environment; the model training module is used for constructing and training a multi-task learning model and comprises functions of data loading, model definition, loss function setting and model optimization; the model reasoning module is used for running a trained model in a deployment environment, performing dust detection and concentration prediction on an image acquired in real time, and optimizing the reasoning speed of the model; the visualization and overrun early warning module is used for visually displaying a detection result and a concentration value, and when the dust concentration exceeds a preset threshold value, an alarm mechanism is triggered to remind related personnel, so that real-time and accurate dust concentration monitoring can be realized, and important technical support and decision basis can be provided for mine dust prevention and control.

Description

Technical Field

[0001] The present invention relates to the field of mine dust detection, and specifically to a dust detection method based on a multi-task learning model and its usage method. Background Art

[0002] The dust concentration in the mine environment is directly related to the safe production of mines and the health of workers. High-concentration dust can not only cause workers to suffer from occupational diseases such as pneumoconiosis, but also trigger major safety accidents such as explosions. Therefore, effective monitoring and control of mine dust are key links in ensuring the safe production of mines.

[0003] Traditional dust detection methods mainly include the filter membrane sampling method, the light scattering dust meter, and the β-ray method. With the rapid development of computer vision and deep learning technologies, image-based dust detection methods have gradually attracted attention. The above methods can meet engineering requirements to a certain extent, but there are still many limitations: (1) Traditional detection methods usually require regular manual sampling and off-line analysis, and cannot achieve continuous detection and real-time data update. (2) Many detection methods can only detect the dust concentration at fixed points or local areas, and cannot achieve full coverage of the entire mine space. This limitation may lead to the failure to timely detect and handle high-concentration dust in some key areas. (3) Most image detection methods can only identify the presence or absence of dust, and cannot accurately quantify the dust concentration in the detection area, making it difficult to meet the requirements of refined dust concentration detection.

[0004] Therefore, there is an urgent need for a new dust detection method to solve the above problems. Summary of the Invention

[0005] The present invention proposes a dust detection method and system based on a multi-task learning model.

[0006] The dust detection method based on a multi-task learning model includes the following steps:

[0007] (1) Install and debug the data acquisition equipment, specifically including: arranging camera equipment and dust concentration sensors in the mine roadway; debugging and calibrating the camera equipment and sensors;

[0008] (2) Conduct data acquisition and dataset construction, specifically including:

[0009] (2.1) Use the installed camera equipment to continuously collect image data in the mine roadway, covering the dust distribution under different time periods, different regions, and different working conditions; synchronously record the data of the dust concentration sensors corresponding to the regions of the collected images, and obtain the dust concentration values at the corresponding times and positions;

[0010] (2.2) Use an image annotation tool to manually annotate the images collected in step (2.1), perform manual semantic segmentation on the dust areas in the images, and generate a pixel-level annotation mask; divide the roadway images into two categories: background and dust, and each pixel point is labeled as "Background" or "Dust" to form a semantic segmentation label; for each annotated image, associate the dust concentration value corresponding to the acquisition time of the image with its dust area label, where Background represents the background and Dust represents the dust;

[0011] (2.3) Construct a dataset, specifically including: perform random augmentation operations on the images in step (2.2) to increase data diversity and improve the robustness of the model, organize and convert the annotated image data into a format that the model can read, construct a multi-task learning dataset containing images, semantic segmentation labels, and dust concentration values, and preprocess the dataset;

[0012] (3) Construct and train a multi-task learning model for dust detection, specifically including:

[0013] (3.1) Construct a multi-task learning model for dust detection, that is, construct a feature extraction network, namely Backbone, a multi-scale feature fusion module, a segmentation head, and a regression head. Backbone is responsible for extracting general features of the image; the multi-scale feature fusion module is used to fuse and enhance features of different scales in the image; the segmentation head is used for pixel-level classification to locate the dust area; the regression head is used to predict the concentration value of the dust segmentation area. In the multi-task learning model, the features after multi-scale feature fusion can be shared by multiple subsequent task branches;

[0014] (3.2) Train and optimize the model constructed in step (3.1), specifically including: randomly divide the preprocessed image dataset in step (2) into a 70% training set, a 20% validation set, and a 10% test set; the training set is used for training the model, and train the multi-task learning model for dust detection constructed in step (3.1); during the training process, the model simultaneously optimizes the segmentation task and the regression task, and by combining the segmentation loss and the regression loss, obtain the optimal performance of the model in these two tasks; the validation set is used to evaluate the performance of the model during the training process and perform hyperparameter tuning; the test set is used to evaluate the final performance of the model after training to ensure the generalization ability of the model;

[0015] (4) Use the optimized model for dust concentration detection, specifically including: using the camera device installed in step (1), obtaining the original image data of the mine roadway in the manner of step (2), constructing an image dataset with annotations, importing the constructed image dataset into the optimized dust detection multi-task learning model established in step (3) to obtain a multi-scale feature map, the segmentation head performs upsampling on the learned different-scale feature maps, outputs a segmentation probability map, sets the threshold to 0.5, and generates a binary mask according to the set threshold. The prediction head extracts the dust area features based on the generated binary mask, pools the features of the dust area to generate a feature vector; and inputs the feature vector into the regression head module for dust concentration prediction to achieve real-time identification of image dust and concentration detection;

[0016] (5) Visualize and process the over-limit warning for the dust detection results detected in step (4).

[0017] Preferably, the image annotation tool in step (2.2) is LabelImg.

[0018] Preferably, the random augmentation operations in step (2.3) include image rotation, translation, scaling, and color jitter.

[0019] Preferably, the operations for preprocessing the dataset in step (2.3) include resizing the image and color normalization.

[0020] Preferably, constructing the feature extraction network in step (3.1) specifically includes:

[0021] Use ResNet50 for feature extraction. After feature extraction, it includes: an input branch with 2 layers, namely 1 7×7 convolutional layer and 1 3×3 max pooling layer; 48 residual blocks in four stages, namely:

[0022] Stage1: contains 3 residual blocks;

[0023] Stage2: contains 4 residual blocks, and the output feature map is denoted as C 3 ;

[0024] Stage3: contains 6 residual blocks, and the output feature map is denoted as C 4 ;

[0025] Stage4: contains 3 residual blocks, and the output feature map is denoted as C 5 ,

[0026] where each residual block consists of 3 convolutional layers; C 3 has 512 channels; C 4 has 1024 channels; C 5 has 2048 channels.

[0027] Preferably, in step (3.1), a multi-scale feature fusion module is constructed, including:

[0028] The BiFPN architecture is used for multi-scale feature fusion, where BiFPN enhances multi-scale features through top-down and bottom-up bidirectional feature fusion. BiFPN includes:

[0029] (a1) Generate the initial feature pyramid layer

[0030] Generate the initial feature pyramid layer. From bottom to top according to the feature pyramid levels, the feature layers are P 3 , P 4 , P 5 , P 6 , P 7 ;

[0031] P 3 , P 4 , P 5 are obtained by 1×1 convolution of the feature layers C 3 , C 4 , C 5 . The specific calculation formula is: P i = Conv 1x1 (C i ), i = 3, 4, 5, and Conv 1x1 is a 1×1 convolutional layer;

[0032] Add convolution and downsampling operations to P5 to generate the feature layers P 6 and P 7 . The specific calculation formula is:

[0033] P 6 = Conv 3x3,s＝2 (P 5 )

[0034] P 7 = ReLU(Conv 3x3,s＝2 (P 6 ))

[0035] Among them, Conv 3x3，s＝2 is a 3x3 convolutional kernel with a stride s of 2; ReLU is the activation function, ReLU = max(0, x);

[0036] (a2) Top-down path fusion

[0037] Perform feature fusion from high-level features to low-level features. In the top-down path, the calculation formula for feature fusion is:

[0038]

[0039]

[0040] Among them, is the feature map of the i-th layer in the top-down path; P i is the feature layer of the initial input; is the top-down feature map of the previous layer, i.e., the (i + 1)-th layer; is a learnable non-negative weight, and its non-negativity is ensured by passing through the ReLU activation function; ∈ is a small constant to prevent division by zero, usually taking ∈ = 10 -4 ; Upsample is an upsampling operation that doubles the size of the feature map; Conv 3x3 is a 3×3 convolutional layer used for smoothing the fused features;

[0041] (a3) Feature fusion in the bottom-up path

[0042] Feature fusion is performed from low-level features to high-level features. In the bottom-up path, the calculation formula for feature fusion is:

[0043]

[0044] Among them, is the feature map of the i-th layer in the bottom-up path; is the feature map of the i-th layer in the top-down path; is the bottom-up feature map of the next layer, i.e., the (i - 1)-th layer; is a learnable non-negative weight, and its non-negativity is ensured by passing through the ReLU activation function; ∈ is a small constant to prevent division by zero, usually taking ∈ = 10 -4 ; Downsample is a downsampling operation that halves the size of the feature map; Conv 3x3 is a 3×3 convolutional layer used for smoothing the fused features;

[0045] (a4) Learnable fusion weights

[0046] At each fusion node in steps (b) and (c), learnable weights are introduced to adjust the contributions of different feature layers; set the initial weights During the training process, the weight parameters are automatically learned through the backpropagation algorithm; enable the BiFPN multi-scale feature fusion module to output feature maps of different scales

[0047] Preferably, constructing the segmentation head in step (3.1) includes: receiving the feature map from the multi-scale feature fusion module, upsampling the low-resolution feature map, and fusing between feature maps of different scales, and finally generating a feature map with the same size as the input image, and classifying each pixel of the feature map to determine whether it belongs to "Dust" or "Background", which specifically includes the following steps:

[0048] (b1) Perform feature layer adjustment, which specifically includes:

[0049] For the multi-scale feature layers of BiFPN from step (d1): Align the number of channels, perform 1×1 convolution on each feature layer, and obtain to

[0050] wherein, is the adjusted feature layer;

[0051] (b2) Perform multi-scale feature fusion and upsampling, which specifically includes:

[0052] Adopt a bottom-up upsampling strategy to gradually upsample the low-resolution feature map and fuse it with the high-resolution feature map. Starting from , gradually upsample and fuse with higher-resolution feature layers to obtain fused features F 7 , F 6 , F 5 , F 4 , F 3 ;

[0053] For i = 7, perform initialization sampling,

[0054] For i = 6, 5, 4, 3, perform the following steps:

[0055] (b21) Upsample the previous fused feature F i+1 to the size of the current adjusted feature layer . The upsampling operation uses bilinear interpolation or transposed convolution: U i = Upsample(F i+1 ), where U i is the upsampled feature;

[0056] (b22) Fuse the upsampled feature U i with the current adjusted using concatenation fusion:

[0057]

[0058] Among them, conv3x3 is a 3×3 convolutional layer with the number of output channels being C;

[0059] Obtain the feature layer F with the highest resolution 3 , with the size of

[0060] Among them, H is the height of the input image; W is the width of the input image;

[0061] (b22) Gradually upsample F 3 to the size H×W of the input image. After each upsampling, extract features through a convolutional layer; Upsample to F up1 = Upsample(F 3 ), and then perform convolutional processing: F up1 = Conv3x3(F up1 ), Upsample to F up2 = Upsample(F up1 ), and then perform convolutional processing: F up2 = Conv3x3(F up2 ), Upsample to H×W: F up3 = Upsample(F up2 ), and then perform convolutional processing: F up3 = Conv3x3(F up3 ), F up3 is an image feature map with the same size as the original image; F up1 is the feature layer F with the highest resolution 3 is the upsampling operation parameter of the feature layer obtained by upsampling the feature layer F by one layer; F up2 is the feature layer F with the highest resolution 3 is the upsampling operation parameter of the feature layer obtained by upsampling the feature layer F by two layers; F up3 is the feature layer F with the highest resolution 3 is the upsampling operation parameter of the feature layer obtained by upsampling the feature layer F by three layers;

[0062] (b3) Split the output layer, specifically including:

[0063] Perform pixel-level classification on the features after upsampling in step (b22), and output the segmentation result and the fused feature map, specifically including:

[0064] (b31) Use a 1×1 convolutional layer to transform the upsampled feature F up3 to adjust the number of channels of the feature map and output the adapted feature map O:

[0065] O = Conv 1x1 (F up3 )

[0066] (b32) Use the Sigmoid activation function to map the output to the range (0, 1), representing the probability that the pixel belongs to "dust":

[0067] S = σ(O)

[0068] where σ is the Sigmoid activation function,

[0069] (b33) Generate a binary segmentation mask according to the probability in step (b32):

[0070]

[0071] where τ is the threshold, set to 0.5; S(x, y) represents the probability value that any pixel (x, y) on the feature map O belongs to the "dust" class. Generate a binary segmentation mask according to the threshold to obtain the final segmentation result.

[0072] Preferably, constructing the regression head in step (3.1) specifically includes:

[0073] (c1) Perform dust area feature extraction:

[0074] Call the feature map that has been fused and upsampled to high resolution in the segmentation head, and focus on the area related to dust in the image, filtering out the background and other irrelevant information, which specifically includes the following steps:

[0075] (c1) Obtain the feature map O from the segmentation head and denote it as F seg , with a size of H×W×C;

[0076] (c12) Apply the segmentation mask M(x, y) obtained in step (b32) to the feature map F seg , perform a masking operation on each pixel position, only retaining the features of the dust area, and setting the features of the background area to zero. The specific calculation formula is:

[0077] F masked (x, y) = F seg (x, y) × M(x, y)

[0078] where F seg (x, y) is the feature map O after one-dimensional convolution in the segmentation head, and F masked (x, y) is the feature map after the masking operation, only retaining the features of the "dust" area and setting the background area to zero;

[0079] (c2) Perform feature aggregation and global representation:

[0080] Convert the feature map of the dust area into a feature vector of fixed length to provide input for subsequent regression calculations, and calculate the feature vector f of the dust area:

[0081]

[0082] where N dust is the total number of pixels in the dust area, and the calculation method is:

[0083]

[0084] where the dimension of f is C, which is the number of channels of the feature map. If N dust = 0, it means that no dust area is detected;

[0085] (c3) Regression calculation layer and dust concentration output calculation:

[0086] Use a fully connected layer to perform regression calculations on the feature vector and output the dust concentration value D:

[0087] D = W 2 h + b 2 , where W 2 is the weight matrix; b 2 is the bias scalar; the calculation method of h is as follows:

[0088] h = ReLU(W 1 f + b 1 )

[0089] where h is the output of the hidden layer; W 1 is the weight matrix; b 1 is the bias scalar;

[0090] Preferably, step (3.2) specifically includes:

[0091] (d1) Define the total loss function L:

[0092] L = λ reg L reg + λ seg L reg

[0093] where L reg is the regression loss, which is used to measure the error of dust concentration prediction; L seg is the segmentation loss, which is used to measure the loss of dust image segmentation; λ reg and λ seg are loss weight coefficients, which control the contributions of the segmentation task and the regression task to the total loss;

[0094] (d2) Perform model training, which specifically includes:

[0095] (d21)Perform forward propagation

[0096] Input the images and dust concentration data of the training set divided in step (2) into the model constructed in step (3.1) to perform joint inference of the segmentation head and the regression head, and obtain the dust segmentation result and the concentration prediction value;

[0097] (d22)Perform loss calculation

[0098] Based on the segmentation result predicted in step (d21) and the true segmentation label, calculate the segmentation loss L seg , and based on the predicted dust concentration value and the true concentration value, calculate the regression loss L reg , and combine the two loss functions to obtain the total loss L;

[0099] (d23)Perform backpropagation

[0100] Based on the total loss L obtained in step (d22), perform backpropagation to calculate the gradients of each layer of the model;

[0101] (d23)Perform parameter update

[0102] Use the optimizer to update the model parameters according to the gradients to minimize the total loss. During training, adaptively adjust the learning rate according to the performance of the model on the validation set to avoid the model falling into local optima;

[0103] (d24)Perform hyperparameter tuning, specifically including:

[0104] According to the convergence situation of the model, use the learning rate scheduler to gradually adjust the learning rate, adjust the batch size according to the hardware resource situation to balance the training speed and the model stability, and dynamically adjust λ reg and λ seg weights to ensure that the losses of the segmentation task and the regression task have a reasonable proportion in the total loss;

[0105] (d3)Perform model validation, specifically including:

[0106] After the model training is completed, evaluate the model performance through the validation set, calculate the segmentation accuracy and the concentration prediction error to ensure that the model can adapt to the dust detection task in the actual mine environment. Specifically, the following steps are included:

[0107] (d31)Validation set preparation and preprocessing, specifically including:

[0108] Preprocess the image data of the validation set divided in step (2) to ensure that the data input format is the same as that of the training set;

[0109] (d32)Calculate performance metrics, specifically including:

[0110] (d321) Calculate the evaluation metrics for segmentation performance, specifically including:

[0111] Calculated accuracy, i.e., Accuracy: It represents the proportion of correct predictions among all predictions and is used to measure the overall accuracy in the dust segmentation task.

[0112] For the dust segmentation task, the formula for accuracy is:

[0113]

[0114] Where: TP is the number of pixels with the true class being "dust" and being correctly predicted as "dust"; TN is the number of pixels with the true class being "background" and being correctly predicted as "background"; FP is the number of pixels with the true class being "background" but being wrongly predicted as "dust"; FN is the number of pixels with the true class being "dust" but being wrongly predicted as "background".

[0115] Intersection over Union, i.e., IoU: It is used to measure the overlap degree between the prediction result and the true label. For the segmentation task, calculate the overlap degree between the predicted dust area and the true dust area. The higher the IoU, the higher the segmentation accuracy of the model.

[0116]

[0117] Among them, A represents the predicted segmentation area; B represents the true segmentation area; |A∩B| represents the number of intersection pixels between the predicted area and the true area; |AUB| represents the number of union pixels between the predicted area and the true area.

[0118] Calculate the F1 value, i.e., F1 Score: Comprehensively consider the precision and recall of the segmentation to evaluate the segmentation performance of the model.

[0119]

[0120]

[0121]

[0122] (d322) Calculate the evaluation metrics for concentration prediction error, specifically including:

[0123] Mean Squared Error, i.e., MSE: It is used to measure the difference between the predicted value and the true value of the dust concentration.

[0124]

[0125] Where: y i is the i-th true value; is the i-th predicted value; n is the total number of samples.

[0126] The mean absolute error, i.e., MAE: It is used to measure the average of the absolute errors between the predicted values and the true values.

[0127]

[0128] (d33) Save the optimal model, specifically including:

[0129] According to the performance metrics calculated in step (d32), check the fluctuations of the model in the segmentation and regression tasks on the validation set to ensure that the prediction results of the model on different samples are stable without obvious overfitting or underfitting phenomena; the model parameters that achieve the best performance on the validation set will be saved for dust detection and concentration prediction in practical applications.

[0130] (d4) Conduct model testing. Use the test set in step (2) to test the validated model and test the model performance.

[0131] The present invention also provides a system based on the above method, which is characterized in that it includes a perception measurement module, a model training module, a model inference module, a visualization and overlimit warning module that cooperate with each other. The perception measurement module is used to collect image data and dust concentration data in the mine environment; the model training module is used to construct and train a multi-task learning model, including functions of data loading, model definition, loss function setting, and model optimization; the model inference module is used to run the trained model in the deployment environment to perform dust detection and concentration prediction on the real-time collected images and optimize the inference speed of the model; the visualization and overlimit warning module is used to visually display the detection results and concentration values, and when the dust concentration exceeds the preset threshold, trigger an alarm mechanism to remind relevant personnel. The perception measurement module is responsible for collecting environmental data and transmitting the environmental data to the model training module. The model training module receives the environmental data, conducts multi-task learning model training, and outputs the trained multi-task learning model to the model inference module. The model inference module performs model inference on the on-site collected data through the multi-task learning model and outputs the dust concentration prediction result to the visualization and overlimit warning module. The visualization and overlimit warning module visualizes the prediction results and gives an early warning broadcast according to the set dust concentration threshold.

[0132] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0133] (1) The present invention can perform on-site sampling and online analysis on the area to be detected, and can achieve continuous detection and real-time data update.

[0134] (2) The dust concentration detection method of the present invention can achieve full coverage of the entire mine space, so as to avoid the high-concentration dust in some key areas from not being discovered and processed in time.

[0135] (3) The present invention uses a multi-task learning model to integrate the semantic segmentation task and the regression prediction task into a unified deep learning framework. By using shared feature representations, it can achieve accurate segmentation of the dust dispersion area and accurate detection of the concentration, making up for the deficiencies of traditional detection methods and being of great significance for the real-time, comprehensive, and accurate monitoring of mine dust.

[0136] (4) The present invention does not require pre-set monitoring points and can directly capture the entire view of the roadway through an intelligent camera, providing higher detection accuracy and timeliness, and at the same time achieving a higher degree of detection automation.

[0137] (5) The present invention detects the dust environment through image recognition technology, which is more adaptable to the complex environment of underground roadways. Description of the Drawings

[0138] Figure 1 is a schematic flow chart of the dust detection method according to Embodiment 1 of the present invention;

[0139] Figure 2 is a schematic diagram of the multi-task learning model architecture for dust detection according to Embodiment 1 of the present invention;

[0140] Figure 3 is a schematic structural diagram of the dust detection system according to Embodiment 2 of the present invention. Specific Embodiment Method

[0142] In order to make the purpose, technical solution, and optimization points of this embodiment clearer and more definite, the present invention will be further described in detail below with reference to the drawings and embodiments. Embodiment 1

[0143] As Figures 1-3 shown, this embodiment provides a dust detection method and system based on a multi-task learning model.

[0144] The dust detection method based on a multi-task learning model includes the following steps:

[0145] (1) Install and debug the data acquisition equipment, specifically including: arranging camera equipment and dust concentration sensors correspondingly in the mine roadway; debugging and calibrating the camera equipment and sensors;

[0146] (2) Conduct data acquisition and dataset construction, specifically including:

[0147] (2.1) Use the installed camera equipment to continuously collect image data in the mine roadway, covering the dust distribution under different time periods, different regions, and different working conditions; synchronously record the data of the dust concentration sensors in the corresponding areas of the collected images to obtain the dust concentration values at the corresponding times and positions;

[0148] (2.2) Use an image annotation tool to perform manual annotation on the images collected in step (2.1). The image annotation tool is LabelImg. Perform manual semantic segmentation on the dust areas in the images to generate a pixel-level annotation mask. Divide the roadway images into two categories: background and dust. Each pixel is labeled as "Background" or "Dust" to form a semantic segmentation label. For each annotated image, associate the dust concentration value corresponding to the acquisition time of the image with its dust area label. Background represents the background, and Dust represents the dust.

[0149] (2.3) Construct a dataset, which specifically includes: perform random augmentation operations on the images in step (2.2) to increase data diversity and improve the robustness of the model. The random augmentation operations include image rotation, translation, scaling, and color jitter. Organize and convert the annotated image data into a format that the model can read. Construct a multi-task learning dataset containing images, semantic segmentation labels, and dust concentration values, and perform preprocessing on the dataset. The preprocessing operations include resizing the images and normalizing the colors.

[0150] (3) Construct and train a multi-task learning model for dust detection, which specifically includes:

[0151] (3.1) Construct a multi-task learning model for dust detection, that is, construct a feature extraction network, namely Backbone, a multi-scale feature fusion module, a segmentation head, and a regression head. Backbone is responsible for extracting general features of the images; the multi-scale feature fusion module is used to fuse and enhance features of different scales in the images; the segmentation head is used for pixel-level classification to locate the dust areas; the regression head is used to predict the concentration values of the dust segmentation areas. In the multi-task learning model, the features after multi-scale feature fusion can be shared by multiple subsequent task branches. The construction of the feature extraction network specifically includes:

[0152] Use ResNet50 for feature extraction. After feature extraction, it includes: an input branch with 2 layers, namely a 7×7 convolutional layer and a 3×3 max pooling layer; 48 residual blocks in four stages, namely:

[0153] Stage1: contains 3 residual blocks;

[0154] Stage2: contains 4 residual blocks, and the output feature map is denoted as C 3 ;

[0155] Stage3: contains 6 residual blocks, and the output feature map is denoted as C 4 ;

[0156] Stage4: contains 3 residual blocks, and the output feature map is denoted as C 5 ,

[0157] Each residual block is composed of 3 convolutional layers; C 3 The number of channels is 512; C 4 The number of channels is 1024; C 5 The number of channels is 2048;

[0158] Construct a multi-scale feature fusion module, including:

[0159] Adopt the BiFPN architecture for multi-scale feature fusion. Among them, BiFPN realizes the enhancement of multi-scale features through top-down and bottom-up bidirectional feature fusion. BiFPN includes:

[0160] (a1) Generate the initial feature pyramid layer

[0161] Generate the initial feature pyramid layer. The feature layers from bottom to top in the feature pyramid hierarchy are the feature layers P 3 , P 4 , P 5 , P 6 , P 7 ;

[0162] P 3 , P 4 , P 5 are obtained by 1×1 convolution of the feature layers C 3 , C 4 , C 5 . The specific calculation formula is: P i = Conv 1x1 (C i ), i = 3, 4, 5, Conv 1x1 is a 1×1 convolutional layer;

[0163] Add convolution and downsampling operations to P5 to generate the feature layers P 6 and P 7 . The specific calculation formula is:

[0164] P 6 = Conv 3x3,s＝2 (P 5 )

[0165] P 7 = ReLU(Conv 3x3,s＝2 (P 6 ))

[0166] Among them, Conv 3x3，s＝2 is a 3x3 convolutional kernel with a stride s of 2; ReLU is the activation function, ReLU = max(0, x).

[0167] (a2) Top-down path fusion

[0168] Feature fusion is performed from high-level features to low-level features. In the top-down path, the calculation formula for feature fusion is as follows:

[0169]

[0170]

[0171] Among them, is the feature map of the i-th layer in the top-down path; P i is the initial input feature layer; is the top-down feature map of the previous layer, i.e., the (i + 1)-th layer; is a learnable non-negative weight, and its non-negativity is ensured by passing through the ReLU activation function; ∈ is a small constant to prevent division by zero, usually taken as ∈ = 10 -4 ; Upsample is the upsampling operation, which doubles the size of the feature map; Conv 3x3 is a 3×3 convolutional layer used for smoothing the fused features; The BiFPN architecture is a neural network architecture well-known to those skilled in the art for object detection tasks;

[0172] (a3) Bottom-up path fusion

[0173] Feature fusion is performed from low-level features to high-level features. In the bottom-up path, the calculation formula for feature fusion is as follows:

[0174]

[0175] Among them, is the feature map of the i-th layer in the bottom-up path; is the feature map of the i-th layer in the top-down path; is the bottom-up feature map of the next layer, i.e., the (i - 1)-th layer; is a learnable non-negative weight, and its non-negativity is ensured by passing through the ReLU activation function; ∈ is a small constant to prevent division by zero, usually taken as ∈ = 10 -4 ; Downsample is the downsampling operation, which halves the size of the feature map; Conv 3x3 is a 3×3 convolutional layer used for smoothing the fused features;

[0176] (a4) Learnable fusion weights

[0177] At each fusion node in steps (b) and (c), learnable weights are introduced to adjust the contributions of different feature layers; Set the initial weights During the training process, the weight parameters are automatically learned through the backpropagation algorithm; Make the BiFPN multi-scale feature fusion module output feature maps of different scales

[0178] The construction segmentation head includes: receiving the feature map from the multi-scale feature fusion module, upsampling the low-resolution feature map, and fusing between feature maps of different scales, and finally generating a feature map with the same size as the input image, and classifying each pixel of the feature map to determine whether it belongs to "Dust" or "Background", specifically including the following steps:

[0179] (b1) Perform feature layer adjustment, specifically including:

[0180] For the multi-scale feature layers of BiFPN from step (d1): Align the number of channels, perform 1×1 convolution on each feature layer to obtain to

[0181] where is the adjusted feature layer;

[0182] (b2) Perform multi-scale feature fusion and upsampling, specifically including:

[0183] Adopt a bottom-up upsampling strategy to gradually upsample the low-resolution feature map and fuse it with the high-resolution feature map. Starting from , gradually upsample and fuse with higher-resolution feature layers to obtain fused features F 7 , F 6 , F 5 , F 4 , F 3 ;

[0184] For i = 7, perform initialization sampling,

[0185] For i = 6, 5, 4, 3, execute the following steps:

[0186] (b21) Upsample the previous layer's fused feature F i+1 to the size of the current adjusted feature layer , and the upsampling operation uses bilinear interpolation or transposed convolution: U i = Upsample(F i+1 ), where U i is the upsampled feature;

[0187] (b22) Fuse the upsampled feature U i with the current adjusted , and the fusion method uses concatenation fusion:

[0188]

[0189] Among them, Conv3x3 is a 3×3 convolutional layer with the number of output channels being C;

[0190] Obtain the feature layer F with the highest resolution 3 , with the size of

[0191] where H is the height of the input image; W is the width of the input image.

[0192] (b22) Gradually upsample F 3 to the size H×W of the input image. After each upsampling, extract features through a convolutional layer; when upsampling to F up1 = Upsample(F 3 ), then perform convolutional processing: F up1 = Conv3x3(F up1 ), when upsampling to F up2 = Upsample(F up1 ), then perform convolutional processing: F up2 = Conv3x3(F up2 ), when upsampling to H×W: F up3 = Upsample(F up2 ), then perform convolutional processing: F up3 = Conv3x3(F up3 ), F up3 is an image feature map with the same size as the original image; F up1 is the feature layer F with the highest resolution 3 is the upsampling operation parameter of the feature layer obtained by upsampling the feature layer F with the highest resolution by one layer; F up2 is the feature layer F with the highest resolution 3 is the upsampling operation parameter of the feature layer obtained by upsampling the feature layer F with the highest resolution by two layers; F up3 is the feature layer F with the highest resolution 3 is the upsampling operation parameter of the feature layer obtained by upsampling the feature layer F with the highest resolution by three layers;

[0193] (b3) Split the output layer, specifically including:

[0194] Perform pixel-level classification on the features after upsampling in step (b22) to output the segmentation result and the fused feature map, specifically including:

[0195] (b31) Use a 1×1 convolutional layer to process the upsampled feature F up3 to adjust the number of channels of the feature map and output the adapted feature map O:

[0196] O = Conv 1x1 (F up3 )

[0197] (b32) Use the Sigmoid activation function to map the output to the range (0, 1), representing the probability that a pixel belongs to "dust":

[0198] S = σ(O)

[0199] where σ is the Sigmoid activation function,

[0200] (b33) Generate a binary segmentation mask based on the probability in step (b32):

[0201]

[0202] where F seg (x, y) is the feature map O after one-dimensional convolution in the segmentation head, and F masked (x, y) is the feature map after masking operation, only retaining the features of the "dust" area and setting the background area to zero; τ is the threshold, set to 0.5; S(x, y) represents the probability value that any pixel (x, y) on the feature map O belongs to the "dust" class, and a binary segmentation mask is generated according to the threshold to obtain the final segmentation result;

[0203] Constructing the regression head specifically includes:

[0204] (c1) Perform dust area feature extraction:

[0205] Call the feature map that has been fused and upsampled to high resolution in the segmentation head, and focus on the areas related to dust in the image, filtering out the background and other irrelevant information, which specifically includes the following steps:

[0206] (c11) Obtain the feature map O from the segmentation head and denote it as F seg , with a size of H × W × C;

[0207] (c12) Apply the segmentation mask M(x, y) obtained in step (b32) to the feature map F seg , perform a masking operation on each pixel position, only retaining the features of the dust area and setting the features of the background area to zero. The specific calculation formula is:

[0208] F masked (x, y) = F seg (x, y) × M(x, y)

[0209] where F seg (x, y) is the feature map O after one-dimensional convolution in the segmentation head, and F masked (x, y) is the feature map after masking operation, only retaining the features of the "dust" area and setting the background area to zero;

[0210] (c2) Feature aggregation and global representation:

[0211] Convert the feature map of the dust area into a feature vector of fixed length to provide input for subsequent regression calculations, and calculate the feature vector f of the dust area:

[0212]

[0213] where N dust is the total number of pixels in the dust area, and the calculation method is:

[0214]

[0215] where the dimension of f is C, which is the number of channels of the feature map. If N dus t = 0, it means that no dust area is detected;

[0216] (c3) Regression calculation layer and dust concentration output calculation:

[0217] Use a fully connected layer to perform regression calculations on the feature vector and output the dust concentration value D:

[0218] D = W 2 h + b 2

[0219] y = W 2 h + b 2

[0220] where W 2 is the weight matrix; b 2 is the bias scalar; the calculation method of h is as follows:

[0221] h = ReLU(W 1 f + b 1 )

[0222] where h is the output of the hidden layer; W 1 is the weight matrix; b 1 is the bias scalar;

[0223] (3.2) Train and optimize the model constructed in step (3.1), specifically including: randomly divide the preprocessed image dataset in step (2) into 70% training set, 20% validation set, and 10% test set. The training set is used for model training. Train the multi-task learning model for dust detection constructed in step (3.1). During the training process, the model simultaneously optimizes the segmentation task and the regression task. By combining the segmentation loss and the regression loss, the optimal performance of the model in these two tasks is obtained; the validation set is used to evaluate the performance of the model during training and perform hyperparameter tuning, and the test set is used to evaluate the final performance of the model after training to ensure the generalization ability of the model;

[0224] Specifically, it includes:

[0225] (d1) Define the total loss function L:

[0226] L = λ reg L reg + λ reg L reg

[0227] Among them, L reg is the regression loss, which is used to measure the error of dust concentration prediction; L seg is the segmentation loss, which is used to measure the loss of dust image segmentation; λ reg and λ reg are loss weight coefficients, which control the contributions of the segmentation task and the regression task to the total loss;

[0228] (d2) Perform model training, specifically including:

[0229] (d21) Perform forward propagation

[0230] Input the images of the training set divided in step (2) into the model constructed in step (3.1), perform joint inference of the segmentation head and the regression head, and obtain the dust segmentation result and the concentration prediction value;

[0231] (d22) Calculate the loss

[0232] Based on the segmentation result predicted in step (d21) and the true segmentation label, calculate the segmentation loss L seg , based on the predicted dust concentration value and the true concentration value, calculate the regression loss L reg , combine the two loss functions to obtain the total loss L;

[0233] (d23) Perform backpropagation

[0234] Based on the total loss L obtained in step (d22), perform backpropagation to calculate the gradients of each layer of the model;

[0235] (d23) Perform parameter update

[0236] Use the optimizer to update the model parameters according to the gradients to minimize the total loss. During the training process, adaptively adjust the learning rate according to the performance of the model on the validation set to avoid the model falling into local optima; The optimizer in this embodiment uses the stochastic gradient descent method to calculate the gradients and update the model parameters according to each data sample during the training process, iteratively approaching the minimum value of the objective function; The update rule is:

[0237]

[0238] Among them, θt is a parameter; α is the learning rate; is to calculate the gradient of the loss function based on the current sample (x i , y i ); (d24) perform hyperparameter tuning, specifically including:

[0239] According to the model convergence situation, use a learning rate scheduler to gradually adjust the learning rate, adjust the batch size according to the hardware resource situation, balance the training speed and model stability, and dynamically adjust λ according to the experimental feedback reg and λ seg weights to ensure that the losses of the segmentation task and the regression task have a reasonable proportion in the total loss; the learning rate scheduler in this embodiment uses StepLR; StepLR multiplies the current learning rate by a predetermined decay factor after every certain number of training iterations, so as to gradually reduce the learning rate to optimize the model performance. The formula for learning rate adjustment is:

[0240]

[0241] where lr t : the current learning rate; lr 0 : the initial learning rate; γ: the learning rate decay factor (usually set to 0.1 or 0.5); epoch: the current training epoch; step_size: the step size of learning rate decay, indicating how many epochs to reduce the learning rate after each; round down;

[0242] (d3) Perform model verification, specifically including:

[0243] After the model training is completed, evaluate the model performance through the validation set, calculate the segmentation accuracy and the concentration prediction error, and ensure that the model can adapt to the dust detection task in the actual mine environment, specifically including the following steps:

[0244] (d31) Validation set preparation and preprocessing, specifically including:

[0245] Preprocess the image data of the validation set divided in step (2) to ensure that the data input format is the same as that of the training set;

[0246] (d32) Model evaluation performance metrics, specifically including:

[0247] (d321) Calculate the segmentation performance evaluation metrics, specifically including:

[0248] Calculate the accuracy, that is, Accuracy: representing the proportion of correct predictions in all predictions, used to measure the overall accuracy in the dust segmentation task;

[0249] For the dust segmentation task, the formula for accuracy is:

[0250]

[0251] Where: TP is the number of pixels with the true class being "dust" and being correctly predicted as "dust"; TN is the number of pixels with the true class being "background" and being correctly predicted as "background"; FP is the number of pixels with the true class being "background" but being wrongly predicted as "dust"; FN is the number of pixels with the true class being "dust" but being wrongly predicted as "background".

[0252] The Intersection over Union, i.e., IoU: It is used to measure the overlap degree between the prediction result and the true label. For the segmentation task, it calculates the overlap degree between the predicted dust area and the true dust area. The higher the IoU, the higher the segmentation accuracy of the model.

[0253]

[0254] Where, A represents the predicted segmentation area; B represents the true segmentation area; |A∩B| represents the number of intersection pixels between the predicted area and the true area; |AUB| represents the number of union pixels between the predicted area and the true area.

[0255] Calculating the F1 value, i.e., F1 Score: It comprehensively considers the precision and recall of the segmentation to evaluate the segmentation performance of the model.

[0256]

[0257]

[0258]

[0259] (d322) Calculate the evaluation indicators for the concentration prediction error, specifically including:

[0260] Mean Squared Error (MSE): It is used to measure the difference between the predicted value and the true value of the dust concentration.

[0261]

[0262] Where: y i is the i-th true value; is the i-th predicted value; n is the total number of samples;

[0263] Mean Absolute Error, i.e., MAE: It is used to measure the average of the absolute errors between the predicted value and the true value.

[0264]

[0265] (d33) Save the optimal model, specifically including:

[0266] According to the performance metrics calculated in step (d32), check the fluctuations of the model in the segmentation and regression tasks on the validation set to ensure that the prediction results of the model on different samples are stable, without obvious overfitting or underfitting phenomena; the model parameters that achieve the best performance on the validation set will be saved for dust detection and concentration prediction in actual applications.

[0267] (d4) Conduct model testing, use the test set in step (2) to test the validated model, and test the model performance.

[0268] (4) Use the optimized model for dust concentration detection, specifically including: using the camera device installed in step (1), obtaining the original image data of the mine roadway in the way of step (2), constructing an image dataset with annotations, importing the constructed image dataset into the optimized dust detection multi-task learning model established in step (3) to obtain multi-scale feature maps, the segmentation head performs upsampling on the learned different-scale feature maps and outputs a segmentation probability map, sets the threshold to 0.5, and generates a binary mask according to the set threshold. The prediction head extracts the dust area features based on the generated binary mask, pools the features of the dust area to generate a feature vector. And input the feature vector into the regression head module for dust concentration prediction to realize real-time identification and concentration detection of image dust.

[0269] (5) Visualize and process the over-limit warning for the dust detection results detected in step (4), and display the currently detected dust concentration value in real time on the external display screen, with the unit of mg / m 3 , set different levels of warning thresholds according to national or industry standards, such as 2mg / m 3 , the system continuously monitors the dust concentration, and once it exceeds the threshold, immediately triggers a warning.

[0270] This embodiment also provides a system based on the above method, which is characterized in that it includes a perception measurement module, a model training module, a model inference module, and a visualization and over-limit warning module that cooperate with each other.

[0271] Perception measurement module: This module is responsible for collecting image data and dust concentration data in real time in the mine roadway environment, providing data support for the subsequent dust detection multi-task learning model.

[0272] Specifically, the perception measurement module includes: a self-cleaning mine camera for collecting mine roadway image data; a dust concentration sensor for obtaining the dust concentration data inside the roadway; and an industrial-grade Ethernet cable for transmitting high-definition images.

[0273] Model Training Module: This module is responsible for constructing and training a multi-task learning model for dust detection, and outputs a trained multi-task learning model for dust detection.

[0274] Specifically, the model training module includes: model construction, using ResNet-50 as the backbone network and combining with a bidirectional feature pyramid network, namely BiFPN, to achieve multi-scale feature fusion. The segmentation head is used for pixel-level segmentation of the dust area, and the regression head is used for the prediction of dust concentration; model training and optimization, using the labeled dataset to train the model, adopting a multi-task loss function, combining the segmentation loss and the regression loss, and adjusting the hyperparameters to optimize the model performance; model verification, evaluating the model performance on the validation set, calculating the segmentation accuracy and the concentration prediction error to ensure the reliability of the model in practical applications.

[0275] Model Inference Module: This module receives the real-time image transmitted by the image acquisition module, uses the trained multi-task learning model to perform image segmentation and concentration detection on the dust in the image, and transmits the calculation results to the visualization and over-limit warning module.

[0276] Visualization and Over-limit Warning Module: This module receives the output of the model inference module, performs visual display of the detection results, and provides a warning when the dust concentration exceeds the limit.

[0277] Specifically, the visualization and over-limit warning module includes: a mine external display, which displays the dust concentration value on the external display screen; an over-limit warning device, which triggers a warning prompt when the dust concentration exceeds the safety threshold.

[0278] In this embodiment, the deployment of the detection system takes the mine roadway as an example, but the application of this embodiment is not limited to underground mines. This system is also applicable to the dust concentration detection in dust-producing places such as flour mills and construction sites.

[0279] The equipment deployment of this system is simple, the maintenance cost is low, it is convenient to be quickly deployed and used in the mine environment, and it greatly improves the efficiency and safety of underground inspections. This embodiment is based on the dust detection technology of the multi-task learning model, realizing the ability to obtain the position of the dust area in the roadway and the dust concentration from end to end, greatly simplifying the work process, and improving the detection efficiency and accuracy. Therefore, this embodiment can not only achieve real-time and accurate dust concentration monitoring, but also provide important technical support and decision-making basis for mine dust prevention and control.

[0280] The above content is only a preferred embodiment of the present invention, and does not impose any form of limitation on the present invention. Any simple modifications and equivalent changes made to the above embodiments based on the technical essence of the present invention all fall within the protection scope of the present invention.

Claims

1. A dust detection method based on a multi-task learning model, characterized in that: The following steps are involved: (1) Install and debug data acquisition equipment, including: arranging video equipment and dust concentration sensors in the mine tunnels; debugging and calibrating video equipment and sensors; (2) Data collection and data set construction, including: (2.1) Using the installed camera equipment, continuously collect image data in the mine tunnel, covering the dust distribution in different time periods, different areas and different working conditions; synchronously record the data of the dust concentration sensor in the area corresponding to the collected image, and obtain the dust concentration value at the corresponding time and location; (2.2) Use image annotation tools to manually annotate the images collected in step (2.1), perform manual semantic segmentation on the dust areas in the images, and generate pixel-level annotation masks; divide the tunnel images into two categories: background and dust, and each pixel is marked as "Background" or "Dust" to form a semantic segmentation label; for each annotated image, associate the dust concentration value of the image at the corresponding acquisition time with its dust area label, where Background represents background and Dust represents dust; (2.3) constructing a data set, specifically including: performing random enhancement operations on the images in step (2.2) to increase data diversity and improve the robustness of the model, organizing and converting the annotated image data into a format that the model can read, constructing a multi-task learning data set containing images, semantic segmentation labels and dust concentration values, and preprocessing the data set; (3) Build and train a multi-task learning model for dust detection, including: (3.1) Constructing a multi-task learning model for dust detection, i.e., constructing a feature extraction network, namely, Backbone, a multi-scale feature fusion module, a segmentation head, and a regression head. Backbone is responsible for extracting common features of the image; the multi-scale feature fusion module is used to fuse and enhance features of different scales in the image; the segmentation head is used to perform pixel-level classification and locate dust areas; the regression head is used to predict the concentration value of the dust segmentation area. In the multi-task learning model, the features after multi-scale feature fusion can be shared by multiple subsequent task branches; (3.2) Training and optimizing the model constructed in step (3.1), specifically including: randomly dividing the preprocessed image data set in step (2) into 70% training set, 20% validation set and 10% test set, the training set is used for model training, and the dust detection multi-task learning model constructed in step (3.1) is trained. During the training process, the model optimizes the segmentation task and the regression task at the same time, and obtains the optimal performance of the model in these two tasks by combining the segmentation loss and the regression loss; the validation set is used to evaluate the performance of the model and perform hyperparameter tuning during the training process; the test set is used to evaluate the final performance of the model after the training is completed; (4) using the optimized model to detect dust concentration, specifically including: using the camera installed in step (1), obtaining the original image data of the mine tunnel through the method of step (2), and constructing a labeled image data set, importing the constructed image data set into the optimized dust detection multi-task learning model established in step (3), and obtaining a multi-scale feature map; the segmentation head upsamples the learned feature maps of different scales, outputs a segmentation probability map, sets the threshold to 0.5, and generates a binary mask according to the set threshold; the prediction head extracts dust area features based on the generated binary mask, pools the features of the dust area, generates a feature vector, and inputs the feature vector into the regression head module to predict dust concentration, thereby realizing real-time recognition of image dust and concentration detection; (5) Visualize and process the dust detection results detected in step (4) to issue an over-limit warning.

2. The dust detection method based on the multi-task learning model according to claim 1 is characterized in that: The image annotation tool in step (2.2) is LabelImg.

3. The dust detection method based on the multi-task learning model according to claim 1 is characterized in that: The random enhancement operations in step (2.3) include image rotation, translation, scaling, and color jittering.

4. The dust detection method based on the multi-task learning model according to claim 1, characterized in that: The operations of preprocessing the data set in step (2.3) include resizing the image and normalizing the color.

5. The dust detection method based on the multi-task learning model according to claim 1 is characterized in that: The construction of the feature extraction network in step (3.1) specifically includes: ResNet50 is used for feature extraction, which includes: 2 layers of input branches, namely 1 7×7 convolution layer and 1 3×3 maximum pooling layer; 48 layers of four-stage residual blocks, namely: Stage 1: contains 3 residual blocks; Stage 2: contains 4 residual blocks, and the output feature map is denoted as C3; Stage 3: contains 6 residual blocks, and the output feature map is denoted as C4; Stage 4: contains 3 residual blocks, and the output feature map is denoted as C5. Each residual block consists of 3 convolutional layers; the number of channels in C3 is 512; the number of channels in C4 is 1024; and the number of channels in C5 is 2048.

6. The dust detection method based on the multi-task learning model according to claim 5 is characterized in that: In step (3.1), a multi-scale feature fusion module is constructed, including: The BiFPN architecture is used for multi-scale feature fusion. BiFPN enhances multi-scale features through top-down and bottom-up bidirectional feature fusion. BiFPN includes: (a1) Generate the initial feature pyramid layer Generate the initial feature pyramid layer, which is feature layer P3, P4, P5, P6, and P7 from bottom to top according to the feature pyramid level; P3, P4, and P5 are obtained by 1×1 convolution of feature layers C3, C4, and C5. The specific calculation formula is: i =Conv 1x1 (C i ),i=3,4,5,Conv 1x1 It is a 1×1 convolutional layer; Add convolution and downsampling operations on P5 to generate feature layers P6 and P7. The specific calculation formula is: P6=Conv 3x3,s＝2 (P5) P7=ReLU(Conv 3x3,s＝2 (P6)) Among them, Conv 3x3，s＝2 The convolution kernel is 3x3 with a step size of 2. ReLU is the activation function, ReLU(x)=max(0,x). (a2) Top-down path fusion Feature fusion is performed from high-level features to low-level features. In the top-down path, the calculation formula for feature fusion is: in, is the feature map of the i-th layer in the top-down path; P i is the feature layer of the initial input; It is the top-down feature map of the previous layer, i.e. the i+1th layer; is a learnable non-negative weight, which is non-negative after the ReLU activation function; ε is a small constant to prevent division by zero, usually ε = 10 -4 ; Upsample is an upsampling operation, which increases the size of the feature map by 2 times; Conv 3x3 It is a 3×3 convolutional layer, which is used for smoothing the fused features; (a3) Bottom-up path fusion From low-level features to high-level features, feature fusion is performed. In the bottom-up path, the calculation formula for feature fusion is: in, is the feature map of the i-th layer in the bottom-up path; is the feature map of the i-th layer in the top-down path; It is the bottom-up feature map of the next layer, i.e. the i-1th layer; is a learnable non-negative weight, which is non-negative after the ReLU activation function; ∈ is a small constant to prevent division by zero, usually ε=10 -4 ; Downsample is a downsampling operation that reduces the feature map size by 2 times; Conv 3x3 It is a 3×3 convolutional layer, which is used for smoothing the fused features; (a4) Learnable fusion weights In each fusion node of steps (b) and (c), learnable weights are introduced to adjust the contribution of different feature layers; initial weights are set During the training process, the weight parameters are automatically learned through the back propagation algorithm, so that the BiFPN multi-scale feature fusion module can output feature maps of different scales.

7. The dust detection method based on the multi-task learning model according to claim 6 is characterized in that: Step (3.1) constructs the segmentation head, including: receiving the feature map from the multi-scale feature fusion module, upsampling the low-resolution feature map, and fusing the feature maps of different scales to finally generate a feature map of the same size as the input image, classifying each pixel of the feature map to determine whether it belongs to "Dust" or "Background", specifically including the following steps: (b1) Perform feature layer adjustments, including: For the multi-scale feature layer of BiFPN from step (d1): Align the number of channels and perform 1×1 convolution on each feature layer to obtain to in, is the adjusted feature layer; (b2) Multi-scale feature fusion and upsampling, including: A bottom-up upsampling strategy is adopted to gradually upsample the low-resolution feature map and fuse it with the high-resolution feature map. At the beginning, the samples are gradually upsampled and fused with the feature layers with higher resolution to obtain the fused features F7, F6, F5, F4, and F3 in turn; For i=7, initial sampling is performed. For i=6,5,4,3, perform the following steps: (b21) The previous layer fusion feature F i+1 Upsample to the current adjusted feature layer The upsampling operation uses bilinear interpolation or deconvolution: U i =Upsample(F i+1 ), U i is the feature after upsampling; (b22) The upsampled feature U i Compared with the current adjusted Fusion is performed using splicing fusion: Among them, Conv3x3 is a 3×3 convolutional layer with C output channels; Get the highest resolution feature layer F3, size is Among them, H is the input image height; W is the input image width. (b22) F3 is gradually upsampled to the size of the input image H×W, and features are extracted through the convolutional layer after each upsampling; F up1 = Upsample(F3), then convolution processing: F up1 =Conv3x3(F up1 ), upsampled to F up2 =Upsample(F up1 ), followed by convolution: F up2 =Conv3x3(F up2 ), upsampled to H×W:F up3 =Upsample(F up2 ), followed by convolution: F up3 =Conv3x3(F up3 ), F up3 is an image feature map with the same size as the original image; F up1 The feature layer upsampling operation parameter for upsampling one layer of the highest resolution feature layer F3; F up2 The feature layer upsampling operation parameter for upsampling 2 layers of the highest resolution feature layer F3; F up3 The feature layer upsampling operation parameters for upsampling 3 layers of the highest resolution feature layer F3; (b3) Segmentation output layer, specifically including: Perform pixel-level classification on the features after upsampling in step (b22), and output the segmentation results and fusion feature maps, specifically including: (b31) Use the sampled feature F at the 1×1 convolution layer up3 Transform to adjust the number of channels of the feature map and output the adapted feature map O: O=Conv 1x1 (F up3 ) (b32) Use the Sigmoid activation function to map the output to the (0,1) range, indicating the probability that the pixel belongs to "dust": S=σ(O) Among them, σ is the Sigmoid activation function, (b33) Generate a binary segmentation mask based on the probability of step (b32): Among them, τ is the threshold, which is set to 0.5; S(x, y) represents the probability value of any pixel (x, y) on the feature map O belonging to the "dust" class. A binary segmentation mask is generated according to the threshold to obtain the final segmentation result.

8. The dust detection method based on the multi-task learning model according to claim 7 is characterized in that: Step (3.1) to build the regression header specifically includes: (c1) Extract dust area features: Call the feature map that has been fused and upsampled to high resolution in the segmentation head, focus on the dust-related areas in the image, and filter out the background and other irrelevant information. The specific steps include: (c1) Get the feature map O from the segmentation head and record it as F seg , size is H×W×C; (c12) Apply the segmentation mask M(x,y) obtained in step (b32) to the feature map F seg , perform mask operation on each pixel position, retain only the features of the dust area, and set the features of the background area to zero. The specific calculation formula is: F masked (x,y)=F seg (x,y)×M(x,y) Among them, F seg (x, y) is the feature map O after one-dimensional convolution in the segmentation head, F masked (x, y) is the feature map after the mask operation, which only retains the features of the "dust" area and sets the background area to zero; (c2) Feature aggregation and global representation: The feature map of the dust area is converted into a feature vector of fixed length to provide input for subsequent regression calculations and calculate the feature vector f of the dust area: Among them, N dust is the total number of pixels in the dust area, calculated as: Among them, the dimension of f is C, which is the number of channels of the feature map. If N dust =0, indicating that no dust area is detected; (c3) Regression calculation layer and dust concentration output calculation: Use the fully connected layer to regress the feature vector and output the dust concentration value D: D=W2h+b2 Among them, W2 is the weight matrix; b2 is the bias scalar; h is calculated as follows: h=ReLU(W1f+b1) Among them, h is the hidden layer output; W1 is the weight matrix; b1 is the bias scalar.

9. The method for using the dust detection method based on the multi-task learning model according to claim 1, characterized in that: Step (3.2) specifically includes: (d1) Define the total loss function L: L=λ reg L reg +λ seg L seg Among them, L reg is the regression loss, which is used to measure the error of dust concentration prediction; L seg is the segmentation loss, which is used to measure the loss of dust image segmentation; reg and λ seg is the loss weight coefficient, which controls the contribution of the segmentation task and regression task in the total loss; (d2) Perform model training, including: (d21) Perform forward propagation Input the images of the training set divided in step (2) into the model constructed in step (3.1), perform joint reasoning of the segmentation head and the regression head, and obtain the dust segmentation results and concentration prediction values; (d22) Calculate loss Based on the segmentation results predicted in step (d21) and the actual segmentation labels, calculate the segmentation loss L seg , based on the predicted dust concentration value and the actual concentration value, calculate the regression loss L seg , combining the two loss functions, we get the total loss L; (d23) Back propagation Perform back propagation based on the total loss L obtained in step (d22) to calculate the gradient of each layer of the model; (d23) Update parameters Use the optimizer to update the model parameters according to the gradient to minimize the total loss. During the training process, the learning rate is adaptively adjusted according to the performance of the model on the validation set to prevent the model from falling into the local optimum. (d24) Perform hyperparameter adjustment, including: According to the convergence of the model, the learning rate scheduler is used to gradually adjust the learning rate, the batch size is adjusted according to the hardware resources, the training speed and model stability are balanced, and λ is dynamically adjusted according to the experimental feedback. reg and λ seg The weights ensure that the losses of the segmentation task and the regression task have a reasonable proportion in the total loss; (d3) Conduct model validation, including: After the model training is completed, the model performance is evaluated through the validation set, and the segmentation accuracy and concentration prediction error are calculated to ensure that the model can adapt to the dust detection task in the actual mine environment. The specific steps include the following: (d31) Preparation and preprocessing of validation set, including: Preprocess the image data of the validation set divided in step (2) to ensure that the data input format is consistent with the training set; (d32) Calculate performance indicators, including: (d321) Calculate the segmentation performance evaluation indicators, including: Calculation accuracy is Accuracy: It indicates the proportion of correct predictions among all predictions, which is used to measure the overall accuracy in the dust segmentation task; For the dust segmentation task, the accuracy formula is: Among them: TP is the number of pixels whose true category is "dust" and is correctly predicted as "dust"; TN is the number of pixels whose true category is "background" and is correctly predicted as "background"; FP is the number of pixels whose true category is "background" but is incorrectly predicted as "dust"; FN is the number of pixels whose true category is "dust" but is incorrectly predicted as "background"; Intersection over Union (IoU): It is used to measure the overlap between the predicted result and the real label. For the segmentation task, the overlap between the predicted dust area and the real dust area is calculated. The higher the IoU, the higher the segmentation accuracy of the model. Where A represents the predicted segmentation area; B represents the real segmentation area; |A∩B| represents the number of intersection pixels between the predicted area and the real area; |AUB| represents the number of union pixels between the predicted area and the real area; Calculate the F1 value, or F1 Score: Comprehensively consider the segmentation precision, or Precision, and recall, or Recall, to evaluate the segmentation performance of the model; (d322) Calculate the concentration prediction error evaluation index, including: Mean square error or MSE: used to measure the difference between the predicted value and the true value of dust concentration; Where: y i is the i-th true value; is the i-th predicted value; n is the total number of samples; Mean absolute error or MAE: It is used to measure the average value of the absolute error between the predicted value and the true value; (d33) Save the optimal model, including: According to the performance indicators calculated in step (d32), check the fluctuation of the segmentation and regression tasks of the model on the validation set to ensure that the prediction results of the model on different samples are stable without obvious overfitting or underfitting; the model parameters that achieve the best performance on the validation set will be saved for dust detection and concentration prediction in actual applications; (d4) Perform model testing and use the test set in step (2) to test the verified model and test the model performance.

10. A system based on the dust detection method based on the multi-task learning model according to claim 1, characterized in that: It includes a perception measurement module, a model training module, a model reasoning module, and a visualization and over-limit warning module that cooperate with each other. The perception measurement module is used to collect image data and dust concentration data in the mine environment; the model training module is used to build and train a multi-task learning model, including data loading, model definition, loss function setting and model optimization functions; the model reasoning module is used to run the trained model in the deployment environment, perform dust detection and concentration prediction on the real-time collected images, and optimize the reasoning speed of the model; the visualization and over-limit warning module is used to visualize the detection results and concentration values. When the dust concentration exceeds the preset threshold, the alarm mechanism is triggered to remind relevant personnel; the perception measurement module is responsible for collecting environmental data and transmitting the environmental data to the model training module; The model training module receives environmental data, performs multi-task learning model training, and outputs the trained multi-task learning model to the model inference module; The model inference module uses the multi-task learning model to infer the data collected on site, and outputs the dust concentration prediction results to the visualization and over-limit warning module; The visualization and over-limit warning module visualizes the prediction results and issues warnings based on the set dust concentration threshold.

Images