A power operation site safety protection equipment wearing state detection method based on YOLO-MobileNet

Abstract

The application belongs to the technical field of target detection, and specifically discloses a safety protection equipment wearing state detection method based on YOLO-MobileNet for electric power operation site, which comprises the following steps: constructing a data set and marking the data set images; constructing a safety protection equipment wearing state detection network based on YOLO-MobileNet; and training and testing a network model. The bottleneck module of MobileNetv2 is improved, an improved MobileNetv2 feature extraction network is constructed by using the improved bottleneck module and the original bottleneck module, the feature extraction network is used to replace the main feature extraction network CSPDarkNet53 of YOLOv4, and part of the standard convolution network of YOLOv4 is replaced into a deep separable convolution network, so that the safety helmet detection accuracy is reduced from the original 97.5% to 95.0%, while the detection speed is increased from the original 22 frames / second to 65 frames / second, which can fully meet the requirement of real-time detection.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology and relates to a method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet. It is used to detect the wearing status of safety protective equipment of workers at power operation sites, and is particularly suitable for detecting the wearing status of safety helmets of workers at power operation sites, and supervising the wearing status of safety helmets of workers. Background Technology

[0002] Safety protective equipment, especially safety helmets, is an important safety measure in electrical work and is crucial for ensuring personal safety. Safety helmets protect the heads of electrical workers, preventing electric shock and head injuries. During electrical work, when a worker is impacted by a falling object, the helmet shell and liner immediately distribute the impact force across the entire skull. Then, the helmet's various parts—the shell, liner, and its cushioning structures (plugs, straps, seams, padding, etc.)—absorb most of the impact force through elastic deformation, plastic deformation, and permissible structural damage, reducing the final impact force on the worker's head to below 4900N. This effectively protects the worker's head from injury or minimizes the risk of injury.

[0003] During power operations, due to a lack of safety awareness among some workers and high temperatures, some workers often fail to wear their safety protective equipment (PPE) completely, particularly without helmets. This poses a significant safety hazard to both power safety and worker safety. Therefore, it is necessary to constantly monitor the wearing of PPE, especially helmets. Traditional methods for monitoring PPE, particularly helmet use, require on-site management personnel to visually supervise the work, wasting considerable manpower. Furthermore, the effectiveness of this monitoring is greatly influenced by factors such as the supervisor's sense of responsibility. To improve the efficiency of detecting PPE, especially helmet use, in power operations, a remote video analysis-based detection method is currently being used. This method utilizes video acquisition devices such as surveillance cameras to capture video of the power operation site and transmits the captured video to a remote server. The server then uses artificial intelligence to analyze the video and detect helmet use. Existing methods for detecting helmet use in power operations mainly consist of YOLO-based methods and R-CNN-based methods. The former is used for real-time detection and has a relatively fast detection speed, while the latter offers better accuracy but is slower and not suitable for real-time detection. While YOLO-based detection methods offer relatively fast detection speeds, the large number of work sites and videos requiring analysis, coupled with limited server computing power, results in insufficient real-time performance. Therefore, in practical applications, detection speed needs further improvement to meet real-time requirements. To this end, it is necessary to reduce the complexity of YOLO-based methods while maintaining detection accuracy requirements, thereby increasing the detection speed of AI-based power work site safety equipment wearing status and meeting real-time requirements. Summary of the Invention

[0004] The technical problem to be solved by this invention is to overcome the shortcomings of the prior art and provide a method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet. While meeting the detection accuracy requirements, this method reduces the complexity of YOLO-like methods, improves the detection speed of the wearing status of safety protective equipment at power operation sites based on artificial intelligence, and meets the real-time requirements of detection.

[0005] The technical solution to the problem solved by this invention is: a method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet, characterized by the following specific steps:

[0006] 1) Construct the dataset and label the images in the dataset.

[0007] 2) Construct a security equipment wear status detection network based on YOLO-MobileNet

[0008] (1) Construct a MobileNetv2 bottleneck module that includes a channel attention network;

[0009] (2) Construct an improved MobileNetv2 network;

[0010] (3) Replace the YOLOv4 backbone feature extraction network CPSDarkNet with the improved MobileNetv2 network constructed in step (2) to construct a YOLO-MobileNet-based security equipment wearing status detection network.

[0011] 3) Network model training and testing

[0012] The network model is trained and tested using the established network model training dataset and network model test dataset.

[0013] Furthermore, step 1) of constructing the dataset and labeling the images in the dataset includes the following steps:

[0014] (1) Use video acquisition equipment to collect video footage from power operation sites;

[0015] (2) Filter the acquired video images;

[0016] (3) The selected images are combined into a training dataset and a test dataset for the network model, and then labeled.

[0017] Furthermore, in step 1), step (2) involves filtering the acquired video images, specifically as follows:

[0018] ① Select images of power workers in different scenarios and group them. Images of workers wearing safety protective equipment are in the first group, and images of workers not wearing safety protective equipment are in the second group.

[0019] ② Divide the first group of images into training group 1 and test group 1, and divide the second group of images into training group 2 and test group 2;

[0020] ③ The training dataset for the network model is composed of one set of training images and two sets of training images, and the test dataset for the network model is composed of two sets of training images and two sets of test images.

[0021] ④ The division ratio of the first group of images is the same as that of the second group of images.

[0022] Furthermore, step 2) (1) involves constructing a MobileNetv2 bottleneck module that includes a channel attention network, specifically as follows:

[0023] ①The MobileNetV2 bottleneck module consists of three parts:

[0024] The first part consists of an extension layer, a batch normalization layer, and an activation function layer;

[0025] The second part includes a depthwise separable convolution with a kernel of 3, a batch normalization layer, and an activation function layer;

[0026] The third part consists of a linear layer and a batch normalization layer;

[0027] ② Add a channel attention network between the batch normalization layer and the activation function layer in the second part to construct a MobileNetv2 bottleneck module containing a channel attention network.

[0028] Furthermore, the activation function of one activation function layer in the first part of the MobileNetV2 bottleneck module is the ReLU6 activation function, and the activation function of one activation function layer in the second part is the ReLU6 activation function.

[0029] Furthermore, step 2) (2) of constructing the improved MobileNetv2 network is as follows:

[0030] ① In the MobileNetv2 bottleneck module containing the channel attention network constructed in step 2), the ReLU6 activation function in the second part of the MobileNetV2 bottleneck module containing the channel attention network is replaced with the hard-Swish function, thereby constructing an improved MobileNetv2 network.

[0031] ② The improved bottleneck module and the original bottleneck module are used alternately in different stages of the improved MobileNetV2 network, and the number of bottleneck modules is adjusted according to the actual test results;

[0032] ③ The adjusted number of bottleneck modules is 1, 2, 3, 4, 3, 2, 1.

[0033] Furthermore, in step 2), (3) the improved MobileNetv2 network is used to replace the YOLOv4 backbone feature extraction network CPSDarkNet, which is represented as follows:

[0034] The improved MobileNetv2 network constructed in step 2) replaces the YOLOv4 backbone feature extraction network CPSDarkNet, and the input features of the three detector heads are obtained through the path aggregation network; depthwise separable convolutions are used instead of standard convolutions connected to the input and output of the spatial pyramid network.

[0035] Furthermore, in step 2), the input features of the three detection heads are: shallow input features, mid-layer input features, and deep input features, respectively, wherein:

[0036] In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 8 of the input image size, are used as the shallow input features of the YOLOv4 Path Aggregation Network, PANet.

[0037] In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 16 of the input image size, are used as the middle layer input features of the YOLOv4 path aggregation network.

[0038] In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 32 of the input image size, are used as input to the YOLOv4 Spatial Pyramid Pooling Network (SPPNet), and the output of the Spatial Pyramid Pooling Network is used as the deep input features of the YOLOv4 Path Aggregation Network.

[0039] Furthermore, step 3) of training and testing the network model is as follows:

[0040] ① Network model training

[0041] During model training, the total number of iterations was 300, the minimum batch size was 15, the initial learning rate was 0.06, the cosine annealing learning rate strategy was adopted, and the network parameters were optimized using the stochastic gradient descent method. The momentum of the stochastic gradient descent optimizer was 0.6, and the weight decay was 0.0004.

[0042] ②Test

[0043] The network model trained in step ① is tested using the test set to see if it meets the requirements. If it does, training ends; otherwise, the parameters are adjusted and the model is retrained and tested.

[0044] This invention provides a method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet. An ECA-Net attention module is added to the bottleneck layer of MobileNetv2 to increase the channel weights containing useful features, thereby improving the feature extraction capability of the bottleneck layer. An improved MobileNetv2 network is constructed using the improved bottleneck layer and the MobileNetv2 bottleneck layer. The last three layers of the improved MobileNetv2 network are then removed, and the resulting improved MobileNetv2 network replaces the YOLOv4 backbone feature extraction network CSPDarkNet53, resulting in a YOLO-MobileNet network with lower computational complexity. By improving the YOLO-MobileNet detection network, the complexity of the YOLOv4 detection network is reduced, solving the problems of slow detection speed and poor real-time performance caused by the high complexity of the YOLOv4 detection network in existing YOLOv4-based safety protective equipment wearing status detection methods. While meeting the detection accuracy requirements, this method improves the detection speed of the wearing status of safety protective equipment at power operation sites based on artificial intelligence and meets the real-time requirements. Compared to existing YOLOv4-based methods for detecting the wearing status of safety protective equipment, the detection method of this invention reduces the detection accuracy from 97.5% to 95.0%, but increases the detection speed from 22 frames / second to 65 frames / second, which fully meets the requirements of real-time detection. Attached Figure Description

[0045] Figure 1 This is a flowchart of the present invention;

[0046] Figure 2 This is a diagram of the deep convolutional network structure of the present invention;

[0047] Figure 3 This is a network structure diagram of the effective channel attention network (ECA-Net) of the present invention;

[0048] Figure 4 This is a diagram of the improved bottleneck layer network structure of MobileNetv2 according to the present invention;

[0049] Figure 5 This is a structural diagram of the present invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0051] See Figures 1-5This embodiment presents a method for detecting the wearing status of safety protective equipment at power work sites based on YOLO-MobileNet, which is used to improve the detection speed of the wearing status of safety helmets by workers at power work sites. The specific steps are as follows:

[0052] 1) Construct the dataset and label the images in the dataset.

[0053] (1) Use on-site surveillance cameras to collect video footage of power operation sites;

[0054] (2) Filter the acquired video images:

[0055] ① Select images of power workers in different scenarios and group them. The first group consists of 5,000 images of workers wearing safety protective equipment, and the second group consists of 5,000 images of workers not wearing safety protective equipment.

[0056] ② In the first group of images, 4000 frames are randomly selected as the first training group and 1000 frames are randomly selected as the first test group. In the second group of images, 4000 frames are randomly selected as the second training group and 1000 frames are randomly selected as the second test group.

[0057] ③ The 4000 frames of images from training group 1 and the 4000 frames of images from training group 2 are combined to form the network model training dataset, and the 1000 frames of images from training group 2 and the 1000 frames of images from test group 2 are combined to form the network model test dataset.

[0058] (3) The selected images are combined into a network model training dataset and a network model test dataset, and labeled using Labelme software respectively;

[0059] 2) Construct a security equipment wear status detection network based on YOLO-MobileNet

[0060] (1) Construct a MobileNetv2 bottleneck module that includes a channel attention network, as follows:

[0061] ①The MobileNetV2 bottleneck module consists of three parts:

[0062] The first part consists of an extension layer, a batch normalization layer, and an activation function layer;

[0063] The second part includes a depthwise separable convolution with a kernel of 3, a batch normalization layer, and an activation function layer;

[0064] The third part consists of a linear layer and a batch normalization layer;

[0065] The activation function of one activation function layer in the first part of the MobileNetv2 bottleneck module is the ReLU6 activation function.

[0066] The activation function of one of the activation function layers in the second part is the ReLU6 activation function;

[0067] ② Add a channel attention network (ECA-Net) between the batch normalization layer and the activation function layer in the second part to construct a MobileNetv2 bottleneck module containing a channel attention network;

[0068] The channel attention network assigns different weights to different channels based on the relationship between the channels and key information in the image. The more relevant the features contained in a channel are to the classification task, the greater the weight assigned to that channel. This makes the network pay more attention to channels containing important features and reduces the influence of useless information. ECA-Net has a simpler structure and lower computational cost compared to other attention modules, thus having a smaller impact on network complexity.

[0069] (2) Construct an improved MobileNetv2 network, as follows:

[0070] ① In the MobileNetv2 bottleneck module containing the channel attention network constructed in step (1), the ReLU6 activation function in the second part of the MobileNetV2 bottleneck module containing the channel attention network is replaced with the hard-Swish function, thereby constructing an improved MobileNetv2 network and improving detection accuracy;

[0071] ② If the improved bottleneck modules are directly used to build the network in a stacked manner, the channel weights will be continuously multiplied with the feature matrix, which may destroy the properties of shortcut connections in the module. This will weaken the ability of shortcut connections to improve network performance to some extent. Therefore, the improved bottleneck modules and the original bottleneck modules are used alternately in different stages of the improved MobileNetV2 network, and the number of bottleneck modules is adjusted according to the actual test results.

[0072] ③ The adjusted number of bottleneck modules is 1, 2, 3, 4, 3, 2, 1;

[0073] (3) Replace the YOLOv4 backbone feature extraction network CPSDarkNet with the improved MobileNetv2 network constructed in step (2) to reduce the computational cost of the YOLOv4 backbone feature extraction network; obtain the input features of the three detection heads through the path aggregation network; in order to reduce network complexity, use depthwise separable convolution to replace the standard convolution connected to the input and output of the spatial pyramid network, thereby constructing a YOLO-MobileNet-based security equipment wear status detection network.

[0074] The input features of the three detection heads are: shallow input features, mid-level input features, and deep input features, respectively, where:

[0075] In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 8 of the input image size, are used as the shallow input features of the YOLOv4 Path Aggregation Network, PANet.

[0076] In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 16 of the input image size, are used as the middle layer input features of the YOLOv4 path aggregation network.

[0077] In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 32 of the input image size, are used as input to the YOLOv4 Spatial Pyramid Pooling Network (SPPNet), and the output of the Spatial Pyramid Pooling Network is used as the deep input features of the YOLOv4 Path Aggregation Network.

[0078] 3) Network model training and testing

[0079] ① Network model training

[0080] During model training, the total number of iterations was 300, the minimum batch size was 15, the initial learning rate was 0.06, the cosine annealing learning rate strategy was adopted, and the network parameters were optimized using the stochastic gradient descent (SGD) method. The momentum of the stochastic gradient descent optimizer was 0.6, and the weight decay was 0.0004.

[0081] ②Test

[0082] The network model trained in step ① is tested using the test set to see if it meets the requirements. If it does, training ends; otherwise, the parameters are adjusted and the model is retrained and tested.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet, characterized in that, The specific steps are as follows: Step 1) Construct the dataset and label the images in the dataset; Step 2) Construct a YOLO-MobileNet-based network for detecting the wear status of security equipment: (1) Construct a MobileNetv2 bottleneck module that includes a channel attention network: ①The MobileNetv2 bottleneck module consists of three parts: The first part consists of an extension layer, a batch normalization layer, and an activation function layer; the activation function of the activation function layer is the ReLU6 activation function. The second part includes a depthwise separable convolution with a kernel size of 3, a batch normalization layer, and an activation function layer; the activation function of the activation function layer is the ReLU6 activation function. The third part consists of a linear layer and a batch normalization layer; ② Add a channel attention network between the batch normalization layer and the activation function layer in the second part to construct a MobileNetv2 bottleneck module containing a channel attention network; (2) Constructing an improved MobileNetv2 network: ① In the MobileNetv2 bottleneck module containing the channel attention network constructed in step 2) (1), the ReLU6 activation function in the second part of the MobileNetv2 bottleneck module containing the channel attention network is replaced with the hard-Swish function, thereby constructing an improved MobileNetv2 network. ② The improved bottleneck module and the original bottleneck module are used alternately in different stages of the improved MobileNetv2 network, and the number of bottleneck modules is adjusted according to the actual test results; ③ The adjusted number of bottleneck modules is 1, 2, 3, 4, 3, 2, 1; (3) Replace the YOLOv4 backbone feature extraction network CPSDarkNet with the improved MobileNetv2 network constructed in (2) to construct a security protection equipment wearing status detection network based on YOLO-MobileNet. Step 3) Network model training and testing: The network model is trained and tested using the established network model training dataset and network model test dataset.

2. The method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet as described in claim 1, characterized in that, Step 1) of constructing the dataset and annotating the images in the dataset includes the following steps: (1) Use video acquisition equipment to collect video of the power operation site; (2) Filter the acquired video images; (3) The selected images are combined into a network model training dataset and a network model test dataset, and then labeled.

3. The method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet as described in claim 2, characterized in that, The process of filtering the acquired video images is as follows: ① Select images of power workers in different scenarios and group them. Images of workers wearing safety protective equipment are in the first group, and images of workers not wearing safety protective equipment are in the second group. ② Divide the first group of images into training group 1 and test group 1, and divide the second group of images into training group 2 and test group 2; ③ The training dataset for the network model is composed of one set of training images and two sets of training images, and the test dataset for the network model is composed of two sets of training images and two sets of test images. ④ The division ratio of the first group of images is the same as that of the second group of images.

4. The method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet as described in claim 1, characterized in that, Step 2) (3) represents replacing the YOLOv4 backbone feature extraction network CPSDarkNet with the constructed improved MobileNetv2 network as follows: The improved MobileNetv2 network constructed in step 2) replaces the YOLOv4 backbone feature extraction network CPSDarkNet, and the input features of the three detection heads are obtained through the path aggregation network; depthwise separable convolution is used instead of the standard convolution connected to the input and output of the spatial pyramid network.

5. The method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet as described in claim 4, characterized in that, The input features of the three detection heads are: shallow input features, mid-level input features, and deep input features, respectively, where: In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 8 of the input image size, are used as the shallow input features of the YOLOv4 Path Aggregation Network, PANet. In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 16 of the input image size, are used as the middle layer input features of the YOLOv4 path aggregation network. In the improved MobileNetv2 network, the output features of the last layer of the bottleneck layer, whose output feature map size is 1 / 32 of the input image size, are used as inputs to the YOLOv4 Spatial Pyramid Pooling Network (SPPNet), and the output of the Spatial Pyramid Pooling Network is used as deep input features to the YOLOv4 Path Aggregation Network.

6. The method for detecting the wearing status of safety protective equipment at power operation sites based on YOLO-MobileNet as described in claim 1, characterized in that, Step 3), network model training and testing, is as follows: ① Network model training During model training, the total number of iterations was 300, the minimum batch size was 15, the initial learning rate was 0.06, the cosine annealing learning rate strategy was adopted, and the network parameters were optimized using the stochastic gradient descent method. The momentum of the stochastic gradient descent optimizer was 0.6, and the weight decay was 0.0004. ②Test The trained network model is tested using the test set to see if it meets the requirements. If it does, training ends; otherwise, the parameters are adjusted and the model is retrained and tested.

Images