Coal mine underground miner safety helmet wearing detection method

Abstract

The invention discloses a coal mine underground miner safety helmet wearing detection method, and relates to the field of coal mine underground safety operation detection.The method comprises the steps that an underground image in a coal mine underground area needing to be monitored is obtained, a trained miner safety helmet wearing detection model is used for detecting the current image, whether a miner wears a safety helmet or not is determined, and if yes, the miner wears the safety helmet, and if yes, the miner wears the safety helmet. And the obtained detection result is displayed on the image. Due to the fact that the underground environment of a coal mine is complex, the quality of collected images is low, and the detection precision is reduced. K-Means + + is adopted to re-cluster a priori frame according to research conditions in the invention; the SPD-Conv is adopted to replace the common convolution with the step length of 2 in the reference model; an SIoU loss function is adopted as a bounding box loss function; and a final improved detection model is obtained. The application can make a contribution to the intelligent recognition of the behavior that miners violate regulations in an underground coal mine.

Description

Technical Field

[0001] The invention relates to a method for detecting the wearing of a safety helmet of a miner in an underground coal mine. Background Art

[0002] Deep learning technology is changing with each passing day, and new research results are emerging in an endless stream. It has been successfully applied to many scenarios in the field of industrial production. The excellent performance of convolutional neural networks in the field of computer vision makes it widely used in target detection, image segmentation, image classification and other fields, and has achieved excellent results. This has also further promoted the in-depth application of deep learning in various industrial fields. The traditional method of detecting the wearing of safety helmets in coal mines is to first use a feature extractor to extract head features, and then use a classifier such as a support vector machine for classification. The disadvantage of this method is that it is not adaptable to complex environments and the workload is also large.

[0003] The training basis of convolutional neural networks is a large number of labeled data sets. As technology gradually develops to the point where large-scale data sets can be collected and hardware equipment develops, researchers have gradually begun to apply convolutional neural networks to the field of target detection. Target detection algorithms include one-stage methods and two-stage methods. The two-stage research methods mainly include R-CNN, Fast R-CNN, Faster R-CNN, etc. The common feature of these methods is that they need to locate the image first and then classify the located image. The advantage of these methods is high detection accuracy, but at the same time, the detection speed is slow. One-stage target detection algorithms have developed rapidly in recent years, mainly including SSD, YOLO series algorithms, etc. These methods directly regress the bounding box, with low detection accuracy, but fast detection speed. Since real-time detection is required in industrial application scenarios, this paper selects the relatively lightweight YOLOv5s algorithm in the YOLO series of algorithms as the basic algorithm of this paper, which can have high detection speed and high detection accuracy. The experimental results show that the recognition accuracy of the selected and improved model reaches a high level, which can be applied to the recognition of safety helmets worn by miners in coal mines. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting whether miners in coal mines are wearing safety helmets. The present invention uses an image enhancement method to process the collected images, inputs the processed images into a detection model for detection, outputs the detection results and displays them on the image, so as to facilitate timely viewing of the wearing status of miners' safety helmets and timely discover the occurrence of not wearing safety helmets, so as to ensure the personnel safety of underground miners.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A method for detecting the wearing of safety helmets by coal mine workers in underground mines, wherein the method constructs a network model for safety helmet detection, and the detection result of the network model is displayed in real time, and specifically comprises the following steps:

[0007] 1) Create a data set for detecting whether miners in coal mines wear helmets;

[0008] 2) Use the deep learning PyTorch framework to configure the network environment and complete the model construction in this environment;

[0009] 3) Before the data set is input into the network model, K-Means++ is used to re-cluster the prior boxes;

[0010] 4) Based on the YOLOv5s model, SPD-Conv is used to replace the conventional convolution with a step size of 2 in the original model;

[0011] 5) Based on YOLOv5s, SIoU loss function is used as the bounding box regression loss function;

[0012] 6) Using the improved YOLOv5s model, the preprocessed data set is input into the model for training, and the network model is tested using video images of miners wearing and not wearing safety helmets in underground coal mines.

[0013] The method for detecting the wearing of safety helmets by coal mine workers in underground mines comprises the following steps: the system performs preprocessing operations on the image data set of the input model; for the underground coal mine environment, the image data of the underground miners wearing safety helmets and miners not wearing safety helmets are collected; the collected images are labeled using the labelimg tool, and the label format is output in the XML format; during network training, the data needs to be divided into a training set and a test set, and the ratio of the training set to the test set is 7:3.

[0014] The invention discloses a method for detecting the wearing of safety helmets by coal mine workers in underground mines. The method comprises preprocessing an image data set to complete the construction of a safety helmet wearing detection model. The model is mainly divided into four parts, namely an input end, a backbone feature extraction network, a Neck network layer, and a prediction end. The input end processes the input image and adopts Masaic data enhancement and other methods to improve the performance of network training. The backbone feature extraction network adopts a CSP structure and an SPPF pyramid structure to better extract features. The Neck part adopts an FPN+PAN structure to further extract features. The prediction end adopts an SIoU loss function as a bounding box regression loss function to predict image features. SGD is used as an optimizer during model training, with an initial learning rate of 0.003, a cyclic learning rate of 0.1, a learning rate momentum of 0.937, and a weight attenuation coefficient of 0.0005.

[0015] The method for detecting the wearing of safety helmets by coal mine workers in underground mines, the model for detecting the wearing of safety helmets is constructed. In order to further improve the accuracy of the model, K-Means++ is first used to re-cluster the prior frame for the data set of this article, and the model training is started based on the re-clustered prior frame; secondly, SPD-Conv is used to replace the conventional convolution with a step size of 2 in YOLOv5s for low-resolution images underground and small targets caused by long-distance photography; finally, SIoU loss function is used as the bounding box regression loss function to further improve the reasoning ability of the model. Compared with the original YOLOv5s model, the improved YOLOv5s model has a detection accuracy improved by 1.3%, which can improve the detection accuracy of the model.

[0016] The method for detecting the wearing of safety helmets by coal mine workers in underground mines, wherein the model is optimized, and the model training parameters such as batch size, epoch, etc. are set before starting the model training. After the model training is completed, it is checked whether the model training results meet the requirements, and the image data is input for detection to check whether the output results meet the requirements.

[0017] The method for detecting the wearing of safety helmets by miners in underground coal mines is described. The model training is completed and tested. After the safety helmet wearing detection model completes the detection of whether the miners appearing in the image are wearing safety helmets, the detection result is output and displayed in the image. The displayed result is the number of miners wearing safety helmets and the number of miners not wearing safety helmets, which is convenient for timely and convenient viewing of whether the miners in the monitoring area are wearing safety helmets in accordance with the underground safety production requirements.

[0018] The advantages and effects of the present invention are:

[0019] The present invention adopts the improved YOLOv5s model as the helmet wearing detection model. After completing the detection of whether the miners appearing in the image are wearing helmets, the detection result is output and displayed in real time in the image. The display result is the number of miners wearing helmets and the number of miners not wearing helmets, which is convenient for timely and convenient viewing whether the miners in the monitoring area are wearing helmets in accordance with the underground safety production requirements.

[0020] The present invention adopts target detection technology to construct a helmet wearing detection model. By detecting the wearing of helmets by miners in coal mines, the real-time detection of the wearing of helmets by miners is obtained. Through real-time analysis of the wearing of helmets, miners who are not wearing helmets can be found and reminded in time. In order to further improve the detection accuracy, K-Means++ is first used to re-cluster the prior frame for the data set in this paper, and the model training is started based on the re-clustered prior frame; secondly, SPD-Conv is used to replace the conventional convolution with a step size of 2 in YOLOv5s for low-resolution images underground and small targets caused by long-distance photography; finally, the SIoU loss function is used as the bounding box regression loss function to further improve the reasoning ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram of the safety helmet detection process of the present invention;

[0022] Figure 2 This is the structure diagram of YOLOv5s of the present invention;

[0023] Figure 3 This is a schematic diagram of the SPD-Conv principle of the present invention;

[0024] Figure 4 Schematic diagram of the Angle cost principle of SIoU of the present invention;

[0025] Figure 5 Schematic diagram of the distance cost principle of SIoU of the present invention;

[0026] Figure 6 This is a flow chart of model training of the present invention;

[0027] Figure 7 It is the mAP graph of the present invention;

[0028] Figure 8 The Loss curve diagram for training the model of the present invention;

[0029] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0030] The present invention is based on the method in the field of target detection and applied to the wearing recognition of safety helmets. Whether a safety helmet is worn is used as the detection category to construct a safety helmet wearing detection model.

[0031] Safety helmet inspection process diagram Figure 1 shown.

[0032] Before inputting the detection model, the input image needs to be preprocessed. For the underground coal mine environment, collect the image data of miners wearing helmets and miners without helmets; use the labelimg tool to annotate the collected images, and output the label format in XML format; when training the network, the data needs to be divided into training set and test set, and the ratio of training set to test set is 7:3.

[0033] After completing the preprocessing of the data set, you need to build a helmet wearing detection model and configure the operating environment. Use Pycharm as the project's IDE and import the PyTorch environment into the project. After the environment configuration is completed, you need to build the detection model and configure the environment. The YOLOv5s model structure diagram is shown below: Figure 2 As shown in the figure, the model is mainly divided into four parts: input end, backbone feature extraction network, Neck network layer, and prediction end. The input end processes the input image and uses Masaic data enhancement and other methods to improve the performance of network training. The backbone feature extraction network adopts the CSP structure and the SPPF pyramid structure to better extract features. The Neck part adopts the FPN+PAN structure to further extract features. The prediction end uses the CIoU loss function as the bounding box regression loss function to predict image features.

[0034] After the YOLOv5s model is built, K-Means++ is first used to re-cluster the prior frames for the dataset in this article. The model training is started based on the re-clustered prior frames. The K-Means++ algorithm is improved compared to the K-Means algorithm used by the original YOLOv5s when clustering the prior frames before training. The essential difference lies in the initialization of the cluster center. Secondly, SPD-Conv is used to replace the conventional convolution with a step size of 2 in YOLOv5s for low-resolution images underground and small targets caused by long-distance photography. The principle diagram is shown in the figure. Figure 3 As shown; Finally, the SIoU loss function is used instead of the original CIoU loss function as the bounding box regression loss function at the prediction end to further improve the reasoning ability of the model. The relevant principle diagram is as follows Figure 4 and Figure 5 shown.

[0035] After the model is improved, the preprocessed image dataset needs to be input into the improved YOLOv5s model for training. Before officially starting the model training, set the paths of the training set and validation set, the type of dataset; select the hyperparameters and activation functions suitable for the dataset training type; set the training worker threads, the total number of training rounds, etc. After the parameter setting is completed, start training the model to obtain the trained model weights. The model training process is as follows: Figure 6 shown.

[0036] After the model training is completed, the mAP performance indicator is used to judge the performance of the model training. The mAP value of the model is as follows: Figure 7 As shown, the Loss curve of the training process is as follows Figure 8 The video recorded in the coal mine is input into the model to detect the accuracy of the model in identifying the wearing of the helmet.

[0037] The helmet wearing detection model will output and display the detection results, distinguish between those wearing helmets and those not wearing helmets, and count the number of people wearing helmets and those not wearing helmets. Through this method, it is possible to effectively detect in real time whether the personnel in the monitoring area are wearing helmets or not. DETAILED DESCRIPTION

[0038] The present invention is applied to the detection of the wearing of safety helmets by coal mine workers, and adopts the following steps:

[0039] 1. Create a data set for detecting whether miners in coal mines wear helmets;

[0040] 2. Use the deep learning PyTorch framework to configure the network environment and complete the model construction in this environment;

[0041] 3. Before the data set is input into the network model, K-Means++ is used to re-cluster the prior boxes;

[0042] 4. Based on the YOLOv5s model, SPD-Conv is used to replace the conventional convolution with a step size of 2 in the original model;

[0043] 5. Based on YOLOv5s, SIoU loss function is used as the bounding box regression loss function;

[0044] 6. Use the improved YOLOv5s model, input the preprocessed data set into the model for training, and use the video images of miners wearing and not wearing safety helmets in coal mines to detect the network model.

[0045] The specific implementation steps of step 1 are:

[0046] Collect pictures of coal mine miners wearing and not wearing helmets, and use labelimg to annotate the pictures;

[0047] Convert the labeled data into yolo format;

[0048] Divide the dataset into training and testing sets.

[0049] The specific implementation steps of step 2 are:

[0050] Import the configuration environment required for the project;

[0051] Constructing the detection model mainly includes the input end, backbone feature extraction network, Neck network layer, and prediction end.

[0052] The specific implementation steps of step 3 are:

[0053] Introducing the K-Means++ clustering algorithm to cluster the data set prior frame is beneficial to improving the accuracy of network training; the first initialized cluster center of the K-Means++ algorithm is randomly selected from the sample points, and then the shortest distance between the selected cluster center and the remaining sample points in the sample is calculated. Based on the obtained shortest distance result, the possibility of these sample points becoming the next cluster center is calculated, and the sample point with the highest probability is selected as the next cluster center. Repeat these processes until k cluster centers are selected. Except for the process of initializing the cluster center, the rest of the process is the same as the prior frame clustering algorithm K-Means used in the original YOLOv5s model training.

[0054] The specific implementation steps of step 4 are:

[0055] SPD-Conv is introduced to replace the conventional convolution with a stride of 2 in the original model. Compared with the traditional strided convolution and pooling layer, SPD-Conv has stronger adaptability. SPD-Conv consists of an SPD layer and a non-strided convolution layer. The SPD layer first downsamples the feature map, but retains all the information in the channel dimension, so no information loss occurs. A non-strided convolution operation is added after the convolution layer. The reason for using non-strided convolution is to retain all feature information as much as possible. SPD downsamples the feature map. For the intermediate feature map X of size S×S×C1, a series of sub-feature maps are obtained:

[0056]

[0057]

[0058] The specific implementation steps of step 5 are:

[0059] The SIoU loss function is introduced as the bounding box regression loss function in the model. Compared with the CIoU loss function used in the original model, it does not take into account the mismatched direction between the real box and the predicted box, resulting in slower convergence and more difficult effective convergence. The SIoU loss function takes into account the angle measurement between the real box and the predicted box, redefines the penalty indicator, and improves the speed and reasoning ability. The SIoU loss function consists of four parts: Angle cost, Distance cost, Shapecost, and IoU cost.

[0060] The height difference between the center point of the real frame and the target frame is C h , the distance between the center point of the real box and the predicted box is σ, and the coordinates of the center of the real box are The center coordinates of the prediction box are The final calculation result of Angle cost is Λ.

[0061]

[0062]

[0063]

[0064]

[0065] Among them C w and C h is the width and height of the minimum bounding rectangle of the real box and the predicted box, ρ x and ρ y It is the square of the relative distance between the center points of the real box and the predicted box on the x-axis and y-axis and the aspect ratio of their minimum enclosing rectangle. Δ is the calculation result of Distance cost.

[0066]

[0067]

[0068] γ=2-Λ (7)

[0069]

[0070] Shape cost is defined as follows, where θ is the attention coefficient in the Shape cost calculation formula, which controls the degree of attention paid to the shape.

[0071]

[0072]

[0073]

[0074] IoU Cost is the intersection over union ratio between the real box and the predicted box, and finally the SIoU loss function definition is obtained.

[0075]

[0076]

[0077] The specific implementation steps of step 6 are:

[0078] The preprocessed data set is divided into training set and test set and input into the network;

[0079] Set the initial learning rate to 0.003, the cyclic learning rate to 0.1, the learning rate momentum to 0.937, the weight decay coefficient to 0.0005, the batch size to 32, and the epoch to 300;

[0080] mAP is selected as an indicator to measure model performance. The recall rate is used as the horizontal axis and the accuracy rate as the vertical axis. The area enclosed by the curve is the AP of a type of detection target. The average of the AP of all categories can be obtained to get mAP. The larger the value, the better the detection result. R is the recall rate, P is the accuracy rate, and AP is the average accuracy rate. In the samples to be tested, the number of positive samples is recorded as P, the number of negative samples is recorded as N, the number of positive samples that are correctly detected is recorded as TP, the number of negative samples that are correctly detected as negative samples is recorded as TN, the number of negative samples that are mistakenly detected as positive samples is recorded as FP, and the number of positive samples that are mistakenly detected as negative samples is recorded as FN. The relevant calculation formula is:

[0081]

[0082]

[0083]

[0084] Detect the mAP performance indicator of the training model; input the video files recorded underground in the coal mine into the network and detect the results of network recognition.

Claims

1. A method for detecting the wearing of safety helmets by coal mine workers, characterized in that: The method constructs an image detection network model for helmet wearing recognition, and the helmet wearing detection result is displayed on the image in real time, specifically comprising the following steps: 1) Create a data set for detecting whether miners in coal mines wear helmets; 2) Use the deep learning PyTorch framework to configure the network environment and complete the model construction in this environment; 3) Before the data set is input into the network model, K-Means++ is used to re-cluster the prior boxes; 4) Based on the YOLOv5s model, SPD-Conv is used to replace the conventional convolution with a step size of 2 in the original model; 5) Based on YOLOv5s, SIoU loss function is used as the bounding box regression loss function; 6) Using the improved YOLOv5s model, the preprocessed data set is input into the model for training, and the network model is tested using video images of miners wearing and not wearing safety helmets in underground coal mines.

2. A method for detecting the wearing of safety helmets by coal mine workers in underground mines according to claim 1, characterized in that: The system performs preprocessing operations on the image data set of the input model; for the underground coal mine environment, collects image data of miners wearing safety helmets and miners not wearing safety helmets; annotates the collected images using the labelimg tool, and outputs the label format in xml format; when training the network, the data needs to be divided into a training set and a test set, and the ratio of the training set to the test set is 7:

3.

3. A method for detecting the wearing of safety helmets by coal mine workers in underground mines according to claim 1, characterized in that: The image data set is preprocessed to complete the construction of the helmet wearing detection model; the model is mainly divided into four parts: input end, backbone feature extraction network, Neck network layer, prediction end, etc. The input end processes the input image and uses Masaic data enhancement and other methods to improve the performance of network training. The backbone feature extraction network adopts CSP structure and SPPF pyramid structure to better extract features. The Neck part adopts FPN+PAN structure to further extract features. The prediction end adopts SIoU loss function as the bounding box regression loss function to predict image features. SGD is used as the optimizer during model training, with an initial learning rate of 0.003, a cyclic learning rate of 0.1, a learning rate momentum of 0.937, and a weight attenuation coefficient of 0.0005.

4. A method for detecting the wearing of safety helmets by coal mine workers in underground mines according to claim 1, characterized in that: The helmet wearing detection model is constructed. In order to further improve the accuracy of the model, K-Means++ is first used to re-cluster the prior frame for the data set in this article, and the model training is started based on the re-clustered prior frame; secondly, SPD-Conv is used to replace the conventional convolution with a step size of 2 in YOLOv5s for low-resolution images underground and small targets caused by long-distance photography; finally, the SIoU loss function is used as the bounding box regression loss function to further improve the reasoning ability of the model.

5. A method for detecting the wearing of safety helmets by coal mine workers in underground mines according to claim 1, characterized in that: After the optimization of the model is completed, the model training parameters such as Batch_size, epoch, etc. are set and then the model training is started. After the model training is completed, check whether the model training results meet the requirements, and input image data for detection to check whether the output results meet the requirements.

6. A method for detecting the wearing of safety helmets by coal mine workers in underground mines according to claim 1, characterized in that: The model training is completed and tested. After the safety helmet wearing detection model completes the detection of whether the miners appearing in the image are wearing safety helmets, the detection results are output and displayed in the image. The displayed results are the number of miners wearing safety helmets and the number of miners not wearing safety helmets, which is convenient for timely and convenient viewing whether the miners in the monitoring area are wearing safety helmets in accordance with the underground safety production requirements.

Images