PPE compliance detection method and device based on improved YOLOv8

Abstract

The invention relates to the technical field of data processing, and discloses an improved YOLOv8-based PPE compliance detection method and device, and the method comprises the steps: obtaining the PPE sample data of personal protection equipment in a construction site, marking the PPE sample data, and obtaining an initial PPE data set; performing online mixed data enhancement on the initial PPE data set to obtain an enhanced PPE data set; an original YOLOv8 model is improved, and an improved PPE compliance detection model is obtained; training the improved PPE compliance detection model through the enhanced PPE data set to obtain an optimized PPE compliance detection model; and detecting a to-be-detected image according to the optimized PPE compliance detection model to obtain a PPE compliance detection result. By implementing the method provided by the invention, the detection efficiency of a PPE wearing detection process can be improved while the accuracy of a PPE compliance detection result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and PPE wearing detection at construction sites, and in particular to a PPE compliance detection method and device based on improved YOLOv8. Background Art

[0002] With the acceleration of industrialization, various industrial production activities are becoming more frequent. However, safety accidents also occur from time to time in industrial production, posing a serious threat to the lives and property of the people. The use of protective equipment for construction workers (such as hard hats, reflective clothing, safety belts, etc.) is of great significance in the construction field. The main goal of personal protective equipment (PPE) testing is to measure health and safety compliance to improve construction safety.

[0003] At present, the traditional target detection method relies on the feature extractor specially designed by researchers to complete the PPE detection task. Although the model is small and runs fast, it depends largely on the feature extraction level and experience of researchers. Due to the dense construction personnel in the real detection task, the small detection target, the complex working environment, the great influence of weather and lighting, and the presence of many interference factors such as occlusion on site, these algorithms can only achieve simple recognition under ideal conditions, and have poor generalization ability for the complex environment of the construction site. The detection performance of small targets is poor, which may lead to inaccurate PPE compliance detection results.

[0004] Therefore, how to improve the accuracy of PPE compliance testing results has become an urgent problem to be solved. Summary of the invention

[0005] The present invention provides a PPE compliance detection method and device based on improved YOLOv8 to solve the technical problem of how to improve the accuracy of PPE compliance detection results.

[0006] In the first aspect, a PPE compliance detection method based on improved YOLOv8 is provided, including:

[0007] Obtaining personal protective equipment (PPE) sample data in a construction site, and annotating the PPE sample data to obtain an initial PPE data set;

[0008] Performing online hybrid data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE dataset to obtain an enhanced PPE dataset;

[0009] The feature fusion module in the original YOLOv8 model backbone network is replaced with an improved efficient multi-scale enhanced C2f-EMSEC module, the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network is replaced with an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model;

[0010] The improved PPE compliance detection model is trained by enhancing the PPE dataset to obtain an optimized PPE compliance detection model;

[0011] The image to be detected is detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result.

[0012] In the second aspect, a PPE compliance detection device based on improved YOLOv8 is provided, comprising:

[0013] An acquisition module is used to acquire sample data of personal protective equipment (PPE) in a construction site, and to annotate the sample data of PPE to obtain an initial PPE data set;

[0014] The first processing module is used to perform online hybrid data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE data set to obtain an enhanced PPE data set;

[0015] The second processing module is used to replace the feature fusion module in the original YOLOv8 model backbone network with an improved efficient multi-scale enhanced C2f-EMSEC module, replace the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network with an improved large separable fast spatial pyramid pooling LSPPF module, and introduce an improved bidirectional feature pyramid network BPNet at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model;

[0016] The third processing module is used to train the improved PPE compliance detection model by enhancing the PPE data set to obtain an optimized PPE compliance detection model.

[0017] The detection module is used to detect the image to be detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result.

[0018] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned PPE compliance detection method based on the improved YOLOv8 are implemented.

[0019] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned PPE compliance detection method based on improved YOLOv8 are implemented.

[0020] In the scheme implemented by the PPE compliance detection method and device based on improved YOLOv8, the initial PPE data set is obtained by obtaining the personal protective equipment PPE sample data in the construction site and annotating the PPE sample data, and the initial PPE data set is enhanced by online mixed data enhancement based on data mixing Mixup and data enhancement Mosaic to obtain an enhanced PPE data set, and the feature fusion module in the original YOLOv8 model backbone network is replaced with the improved efficient multi-scale enhancement C2f-EMSEC module, and the fast spatial glyph in the original YOLOv8 model backbone network is replaced with the improved high-efficiency multi-scale enhancement C2f-EMSEC module. The tower pooling SPPF module is replaced by an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced into the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model, so as to train the improved PPE compliance detection model by enhancing the PPE data set to obtain an optimized PPE compliance detection model; then, the image to be detected is detected according to the optimized PPE compliance detection model to obtain a more accurate PPE compliance detection result, which is beneficial to improve the accuracy of the PPE compliance detection result while improving the detection efficiency of the PPE wearing detection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0022] Figure 1 It is a flow chart of a PPE compliance detection method based on improved YOLOv8 in one embodiment of the present invention;

[0023] Figure 2 It is a structural schematic diagram of an improved PPE compliance detection model of a PPE compliance detection method based on an improved YOLOv8 in one embodiment of the present invention;

[0024] Figure 3 It is a schematic diagram of the operation flow of a C2f-EMSEC module and a LSPPF module of a PPE compliance detection method based on an improved YOLOv8 in one embodiment of the present invention;

[0025] Figure 4 is a structural schematic diagram of a PPE compliance detection device based on improved YOLOv8 in one embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0027] Figure 6 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] See also Figure 1 As shown, Figure 1 A flow chart of a PPE compliance detection method based on improved YOLOv8 provided in an embodiment of the present invention includes the following steps:

[0030] S10: Obtain personal protective equipment (PPE) sample data in the construction site, and annotate the PPE sample data to obtain an initial PPE data set.

[0031] Optionally, a surveillance camera or a mobile phone may be used to shoot to obtain an image dataset of the PPE worn by construction workers at the construction site, i.e., the above-mentioned PPE sample data. The PPE sample data may include sample data obtained by shooting from multiple angles, multiple distances, and multiple backgrounds.

[0032] The collected PPE sample data can be annotated in Yolo format using annotation software (such as the image annotation open source tool LabelImg) to produce an initial PPE data set.

[0033] S20: Perform online hybrid data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE dataset to obtain an enhanced PPE dataset.

[0034] It should be noted that since machine learning requires learning through a large number of image samples, data augmentation of the initial PPE dataset can expand the dataset to obtain richer dataset images, which can be better used for model training and thus improve the generalization ability of the model.

[0035] The core idea of ​​the data mixing method Mixup is to randomly select two pictures, mix them in a certain ratio to generate a new image, and use the new image in training. The main idea of ​​the data enhancement method Mosic (also known as "mosaic data enhancement") is to splice four pictures into one picture as a training sample, and pass such a new picture into the neural network for learning, which is equivalent to directly using the four pictures passed in for learning.

[0036] Optionally, online hybrid data augmentation can add two sample-label data pairs in proportion to generate new sample-label data. The process can be seen in the following formula:

[0037]

[0038] in, can represent the sample data in the newly generated sample-label data pairs; η can represent the proportional coefficient, which usually takes a value between to and can be used to determine the proportion of sample-label data pairs randomly selected from the training data in the newly generated sample and label data pairs; m i It can be used to represent sample data in a sample-label data pair randomly selected from training data, such as the first sample data in the first sample-label data pair; m j It can be used to represent sample data in another sample-label data pair randomly selected from the training data, such as the second sample data in the second sample-label data pair; It can represent the label data in the generated new sample-label data pair; n i It can be used to represent the label data in a sample-label data pair randomly selected from the training data, such as the first label data in the first sample-label data pair; n j It can be used to represent the label data in another sample-label data pair randomly selected from the training data, such as the second label data in the second sample-label data pair.

[0039] It should be noted that all images in the initial PPE dataset can have original labels, which can include: the horizontal coordinate of the upper left corner of the annotation box<bbox_left> , the vertical coordinate of the upper left corner of the annotation box<bbox_top> , Label box width<bbox_width> , Label box height<bbox_height> ,score <score>, target category<object_category> , cutoff rate <truncation>And the occlusion rate <occlusion>Etc., this embodiment does not impose any specific limitation on this.

[0040] It is understandable that the original labels of the initial data set can be converted into label formats to convert the original labels into target labels. The target labels may include: categories <c>, horizontal coordinate of the center of the label box <x>, the vertical coordinate of the center of the annotation box <y>, the annotation box is relatively wide <w>, the annotation box is relatively high <h>Etc., this embodiment does not impose any specific limitation on this.

[0041] S30: The feature fusion module in the original YOLOv8 model backbone network is replaced with an improved efficient multi-scale enhanced C2f-EMSEC module, the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network is replaced with an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model.

[0042] Among them, the backbone network of the original YOLOv8 (You Only Look Once version 8) model can include feature fusion modules, such as the C2f (Faster Implementation of CSP Bottleneck with 2convolutions) module, and the Spatial Pyramid Pooling-Fast (SPPF) module. Compared with other target detection algorithms, the backbone network of the original YOLOv8 model has certain advantages in feature extraction capability and model efficiency, and can achieve fast target detection while maintaining a high accuracy rate.

[0043] Therefore, this application is based on the original YOLOv8 model and makes a series of improvements on the backbone network to obtain an improved PPE compliance detection model, thereby further improving the detection speed and accuracy of the model. Specifically, the backbone network of the improved PPE compliance detection model replaces the C2f module with an improved C2f-EMSEC (C2f-EfficientMulti-Scale Enhanced Convolution) module and the SPPF module with an improved LSPPF (LargeSpatial Pyramid Pooling-Fast) module.

[0044] The C2f-EMSEC module is constructed based on the C2f module in the original YOLOv8 model, and the standard convolution in the Bottleneck structure in the C2f module is replaced by the EMSEC module. The LSPPF module is constructed based on the SPPF module in the original YOLOv8 model, and the Local and Shifted Kernel Attention (LSKA) module is embedded in the SPPF module.

[0045] Furthermore, an improved bidirectional feature pyramid network (BPNet) can be introduced at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model. BPNet is able to perform multi-scale fusion of feature maps output from the backbone network. BPNet introduces bidirectional connections, allowing information to propagate bidirectionally between different resolution levels. In BPNet, each bidirectional path (top-down and bottom-up) is regarded as a separate feature network layer, and then these layers can be repeated multiple times to promote higher-level feature fusion. An adaptive feature adjustment mechanism is used in BPNet, and weights can be learned to adjust features at different levels to better match the needs of different tasks.

[0046] like Figure 2 As shown, Figure 2 : is a schematic diagram of the structure of the improved PPE compliance detection model in the PPE compliance detection method based on improved YOLOv8 in this embodiment. Figure 2 It can be seen that the improved PPE compliance detection model can include three parts: backbone network (Backbone), neck (Neck, i.e. feature fusion network) and head (Head, also known as detection head network), which can include C2f module, C2f-EMSEC module, LSPPF module, BPNet structure, upsampling layer upsample, feature fusion layer Concat, detection head Detect, etc. This application does not impose any restrictions on this.

[0047] It can be understood that the BPNet structure is used in the neck layer network of the improved PPE compliance detection model. The neck layer network can be used for feature fusion. The BPNet block is designed to enhance the ability of multi-scale feature fusion to better capture multi-scale information and improve model performance.

[0048] S40: The improved PPE compliance detection model is trained by enhancing the PPE data set to obtain an optimized PPE compliance detection model.

[0049] The improved YOLOv8 network can be input with the sorted PPE wearing image dataset of construction workers on the construction site, that is, the enhanced PPE dataset mentioned above, for model training to obtain an optimized PPE compliance detection model. Specifically, the size of the input image in the enhanced PPE dataset can be adjusted to 640×640×3, the number of training rounds (epoch) is set to 300, the early stopping parameter (patience) is set to 50, the weight decay is set to 0.0005, and the batch size (batch) is 32; the optimizer uses stochastic gradient descent (SGD); the initial learning rate is 0.01; the momentum factor is 0.937; the confidence is set to 0.5; for non-maximum suppression (NMS), the intersection over union (IoU) is set to 0.7; wait for the training to complete, and obtain an optimized improved PPE compliance detection model.

[0050] It should be noted that the improved PPE compliance detection model is trained by the enhanced PPE dataset after data enhancement, the loss function is calculated according to the training results, and the model parameters are optimized by the loss function to improve the model performance and obtain the optimized PPE compliance detection model.

[0051] S50: Detect the image to be detected according to the optimized PPE compliance detection model to obtain a PPE compliance detection result.

[0052] The image to be detected is detected according to the optimized PPE compliance detection model, and a classification label and a positioning box of each detection object in the image to be detected can be output, which is not specifically limited in this embodiment.

[0053] In this embodiment, personal protective equipment PPE sample data in a construction site is obtained and the PPE sample data is annotated to obtain an initial PPE data set, and the initial PPE data set is enhanced by online mixed data enhancement based on data mixing Mixup and data enhancement Mosaic to obtain an enhanced PPE data set, and the feature fusion module in the original YOLOv8 model backbone network is replaced with an improved efficient multi-scale enhanced C2f-EMSEC module, and the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network is replaced with an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced into the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model, so that the improved PPE compliance detection model is trained by the enhanced PPE data set to obtain an optimized PPE compliance detection model; and then the image to be detected is detected according to the optimized PPE compliance detection model to obtain a more accurate PPE compliance detection result, which is beneficial to improving the accuracy of the PPE compliance detection result while improving the detection efficiency of the PPE wearing detection process.

[0054] In a possible implementation, detecting the image to be detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result refers to the process of using the trained model to perform PPE compliance detection. Among them, detecting the image to be detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result may include the following steps:

[0055] A1: Preprocess the image to be detected to obtain the initial input image;

[0056] A2: Input the initial input image into the optimized PPE compliance detection model, capture spatial features at multiple scales through the C2f-EMSEC module, and obtain a multi-scale fused spatial feature image;

[0057] A3: The multi-scale fusion spatial feature image is subjected to adaptive feature extraction through an LSPPF module to obtain an enhanced spatial feature image;

[0058] A4: Inputting the enhanced spatial feature image into the neck layer network of the optimized PPE compliance detection model for encoding to obtain a target feature image;

[0059] A5: According to the target feature image, a PPE compliance detection result is obtained.

[0060] The initial input image may be a model input image obtained by preprocessing the image to be detected, such as feature enhancement processing.

[0061] The standard convolution corresponding to the bottleneck structure of the C2f module in the original YOLOv8 model backbone network can be replaced with an EMSEC module. Optionally, the initial input image is input into the optimized PPE compliance detection model, and the spatial features are captured at multiple scales by the C2f-EMSEC module to obtain a multi-scale fused spatial feature image, which may include the following steps:

[0062] A21: Input the initial input image into the optimized PPE compliance detection model, divide the initial input image into K heads in the channel dimension through the C2f-EMSEC module, and obtain a set of head feature images;

[0063] A22: applying a corresponding convolution kernel to perform convolution processing on each head feature image in the head feature image set to obtain a multi-scale feature image set;

[0064] A23: performing feature stitching on corresponding multi-scale feature images in the multi-scale feature image set to obtain a stitching feature image set in which features are independent of each other in each channel;

[0065] A24: Performing point-by-point convolution on each of the spliced ​​feature images in the spliced ​​feature image set through 1×1 convolution to achieve fusion of channel information and obtain a multi-scale fused spatial feature image.

[0066] Optionally, the initial input image is input into the optimized PPE compliance detection model, and spatial features are captured at multiple scales through the C2f-EMSEC module to obtain a multi-scale fused spatial feature image, which can be understood as being processed by the EMSEC module in the C2f-EMSEC module.

[0067] In the specific implementation, take the EMSEC module that divides the input channel into 4 heads as an example. For example, the initial input image x = [x1, x2, x3, x4] represents that the input features are divided into 4 heads in the channel dimension; first, convolutions with kernel sizes of 1×1, 3×3, 5×5 and 7×7 can be applied to each head respectively, and then the convolution output features of different sizes can be spliced, and the spliced ​​features are independent of each other in each channel; finally, point-by-point convolution can be achieved by adding 1×1 convolution to complete the fusion of channel information.

[0068] Optionally, the EMSEC module captures spatial features at multiple scales to obtain a process of multi-scale fusion spatial features, see the following formula:

[0069] EMSEC=Conv 1×1 (Concat(Conv 1×1 (x1),Conv 3×3 (x2),

[0070] Conv 5×5 (x3),Conv 7×7 (x4)))

[0071] Among them, EMSEC means that the operation is performed through the EMSEC module; Conv 1×1 Represents a convolution kernel of size 1×1; Concat represents a concatenation operation; x1 represents a head into which the input feature is divided in the channel dimension, such as the first head; Conv 3×3 represents a convolution kernel of size 3×3; x2 represents a head into which the input features are divided in the channel dimension, such as the second head; Conv 5×5 Represents a convolution kernel of size 5×5; x3 represents a head into which the input features are divided in the channel dimension, such as the third head; Conv 7×7 represents a convolution kernel of size 7×7; x4 represents a head into which the input features are divided in the channel dimension, such as the fourth head.

[0072] The LSPPF module is constructed based on the SPPF module in the initial target detection model, and is obtained by embedding the LSKA module in the SPPF module. Optionally, the multi-scale fusion spatial feature image is subjected to adaptive feature extraction through the LSPPF module to obtain an enhanced spatial feature image, which may include the following steps:

[0073] A31: extracting local spatial features from the multi-scale fusion spatial feature image through a deep convolution block in an LSPPF module to obtain a local spatial feature image;

[0074] A32: performing convolution processing on the local spatial feature image through a depth extension convolution block to obtain a depth extension feature image;

[0075] A33: performing a 1×1 kernel convolution on the depth-extended feature image to obtain an attention feature image;

[0076] A34: Perform element-by-element multiplication of the attention feature image and the multi-scale fusion spatial feature image to obtain an enhanced spatial feature image.

[0077] Optionally, the adaptive feature extraction of the multi-scale fused spatial feature image through an LSPPF module to obtain an enhanced spatial feature image can be understood as processing through an LSKA module in the LSPPF module.

[0078] The LSKA module can adopt a residual structure consisting of four depth-separable convolution blocks and a convolution block with a convolution kernel size of 1×1. In the depth convolution stage, a separate convolution operation can be performed on each channel of the input. If the input has C channels, there will be C convolution kernels, each of which is responsible for processing information from one input channel. By fusing the LSKA module, the LSPPF is able to extract the global features of the backbone network.

[0079] Optionally, performing adaptive feature extraction on the multi-scale fused spatial feature image through an LSPPF module to obtain enhanced spatial features may include: performing adaptive feature extraction on the multi-scale fused spatial feature image through an LSPPF module to obtain an enhanced spatial feature image through the following formula:

[0080]

[0081] G C =W 1×1 *X C

[0082]

[0083] Among them, Z C represents the local spatial feature image; C represents the number of input channels; H represents the height of the multi-scale fusion spatial feature image in the spatial dimension; W represents the width of the multi-scale fusion spatial feature image in the spatial dimension; Indicates the operation of depth convolution with a kernel size of (2d-1)×1; * indicates the convolution operation; represents the operation of depthwise convolution with a kernel size of 1×(2d-1); F C represents the multi-scale fusion spatial feature image; X C represents the deep extended feature image; Indicates that the kernel size is The depth convolution operation; Indicates that the kernel size is The operation of depth convolution; G C represents the attention feature image; W 1×1 Represents a convolution operation with a kernel size of 1×1; represents the enhanced spatial feature image; Represents an element-wise product operation.

[0084] For example, Figure 3 As shown, Figure 3 This is a schematic diagram of the operation flow of the C2f-EMSEC module and the LSPPF module in the PPE compliance detection method based on the improved YOLOv8 of this embodiment.

[0085] The enhanced spatial feature image is input into the neck layer network of the optimized PPE compliance detection model for encoding to obtain a target feature map. The improved neck network may include a BPNet structure.

[0086] It is understandable that BPNet introduces bidirectional connections that allow information to propagate bidirectionally between different resolution levels. In BPNet, each bidirectional path (top-down and bottom-up) can be regarded as a separate feature network layer, and then these layers can be repeated multiple times to promote higher-level feature fusion. BPNet uses an adaptive feature adjustment mechanism that can learn weights to adjust features at different levels to better match the needs of different tasks.

[0087] Furthermore, through the BPNet structure, the enhanced spatial feature image is divided into multiple feature maps of different sizes across the channel dimension direction. BPNet performs feature fusion on the received feature maps of sizes of 160×160, 80×80, 40×40, and 20×20, and then performs cross-space learning and activation functions, such as Sigmoid function, to obtain the target feature image after calculation.

[0088] It can be understood that after the image to be detected is input into the optimized PPE compliance detection model and the model obtains the output result, it can first perform confidence filtering, then perform NMS non-maximum suppression, and finally perform coordinate transformation, so as to obtain the final result, that is, the PPE compliance detection result, which is helpful to detect whether the construction workers are wearing PPE in a standardized manner.

[0089] It can be understood that during the model training process, when the model obtains the target feature image, the target feature image can be input into the Head part of the model (such as the aforementioned improved PPE compliance detection model) for detection, and the marking box and classification label can be output, so as to determine the target loss function based on the marking box and classification label.

[0090] It should be noted that the role of the loss function is to measure the distance between the neural network prediction information and the expected information (label). The closer the predicted information is to the expected information, the smaller the loss function value is. In this embodiment, the loss may include classification loss and position loss, and the target loss function is obtained according to the classification loss and position loss.

[0091] Furthermore, determining the target loss function based on the marked box and the classification label may include the following steps: calculating based on the marked box and the real box to obtain a position loss function; calculating based on the classification label and the preset label to obtain a classification loss function; determining the target loss function based on the position loss function and the classification loss function.

[0092] The parameters of the improved target detection model can be further optimized by the target loss function to obtain an optimized target detection model. It should be noted that the model parameters are updated by the target loss function until the confidence of the model prediction exceeds a preset value, such as 90%, then the training and updating can be stopped, and the current model can be used as an optimized PPE compliance detection model.

[0093] In this embodiment, the image to be detected is preprocessed to obtain an initial input image, and the initial input image is input into the optimized PPE compliance detection model. The spatial features are captured at multiple scales through the C2f-EMSEC module to obtain a multi-scale fused spatial feature image, and the multi-scale fused spatial feature image is adaptively extracted through the LSPPF module to obtain an enhanced spatial feature image, and the enhanced spatial feature image is further input into the neck layer network of the optimized PPE compliance detection model for encoding to obtain a target feature image, so that accurate PPE compliance detection results can be obtained according to the target feature image, which is beneficial to improving the accuracy of the PPE compliance detection results while improving the detection efficiency of the PPE wearing detection process.

[0094] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0095] In one embodiment, a PPE compliance detection device based on improved YOLOv8 is provided, and the PPE compliance detection device based on improved YOLOv8 corresponds one-to-one to the PPE compliance detection method based on improved YOLOv8 in the above embodiment. Figure 4 As shown, the PPE compliance detection device based on improved YOLOv8 includes an acquisition module 101, a first processing module 102, a second processing module 103, a third processing module 104 and a detection module 105. Each functional module is described in detail as follows:

[0096] An acquisition module 101 is used to acquire sample data of personal protective equipment (PPE) in a construction site, and to annotate the sample data of PPE to obtain an initial PPE data set;

[0097] The first processing module 102 is used to perform online hybrid data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE data set to obtain an enhanced PPE data set;

[0098] The second processing module 103 is used to replace the feature fusion module in the original YOLOv8 model backbone network with an improved efficient multi-scale enhanced C2f-EMSEC module, replace the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network with an improved large separable fast spatial pyramid pooling LSPPF module, and introduce an improved bidirectional feature pyramid network BPNet at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model;

[0099] The third processing module 104 is used to train the improved PPE compliance detection model by enhancing the PPE data set to obtain an optimized PPE compliance detection model.

[0100] The detection module 105 is used to detect the image to be detected according to the optimized PPE compliance detection model to obtain a PPE compliance detection result.

[0101] In one embodiment, the detection module 105 is used to detect the image to be detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result, specifically for:

[0102] Preprocess the image to be detected to obtain an initial input image;

[0103] Inputting the initial input image into the optimized PPE compliance detection model, capturing spatial features at multiple scales through the C2f-EMSEC module, and obtaining a multi-scale fused spatial feature image;

[0104] The multi-scale fusion spatial feature image is subjected to adaptive feature extraction through an LSPPF module to obtain an enhanced spatial feature image;

[0105] Inputting the enhanced spatial feature image into the neck layer network of the optimized PPE compliance detection model for encoding to obtain a target feature image;

[0106] A PPE compliance detection result is obtained according to the target feature image.

[0107] In one embodiment, the detection module 105 is used to input the initial input image into the optimized PPE compliance detection model, capture spatial features at multiple scales through the C2f-EMSEC module, and obtain a multi-scale fused spatial feature image, which is specifically used to:

[0108] Inputting the initial input image into the optimized PPE compliance detection model, dividing the initial input image into K heads in the channel dimension through the C2f-EMSEC module, and obtaining a set of head feature images;

[0109] Applying a corresponding convolution kernel to perform convolution processing on each head feature image in the head feature image set to obtain a multi-scale feature image set;

[0110] Performing feature stitching on corresponding multi-scale feature images in the multi-scale feature image set to obtain a stitching feature image set with independent features in each channel;

[0111] Each spliced ​​feature image in the spliced ​​feature image set is convolved point by point through 1×1 convolution to achieve fusion of channel information and obtain a multi-scale fused spatial feature image.

[0112] In one embodiment, the detection module 105 is used to perform adaptive feature extraction on the multi-scale fusion spatial feature image through the LSPPF module to obtain an enhanced spatial feature image, specifically for:

[0113] The multi-scale fusion spatial feature image is subjected to local spatial feature extraction by a deep convolution block in an LSPPF module to obtain a local spatial feature image;

[0114] Convolutionally processing the local spatial feature image through a depth-expanded convolution block to obtain a depth-expanded feature image;

[0115] Performing a 1×1 kernel convolution on the depth-extended feature image to obtain an attention feature image;

[0116] The attention feature image is element-wise multiplied with the multi-scale fusion spatial feature image to obtain an enhanced spatial feature image.

[0117] In one embodiment, the detection module 105 is used to perform adaptive feature extraction on the multi-scale fusion spatial feature image through the LSPPF module to obtain an enhanced spatial feature image, specifically for:

[0118] The step of performing adaptive feature extraction on the multi-scale fused spatial feature image through the LSPPF module to obtain an enhanced spatial feature image is implemented by the following formula:

[0119]

[0120] G C =W 1×1 *X C

[0121]

[0122] Among them, Z C represents the local spatial feature image; C represents the number of input channels; H represents the height of the multi-scale fusion spatial feature image in the spatial dimension; W represents the width of the multi-scale fusion spatial feature image in the spatial dimension; Indicates the operation of depth convolution with a kernel size of (2d-1)×1; * indicates the convolution operation; represents the operation of depthwise convolution with a kernel size of 1×(2d-1); F C represents the multi-scale fusion spatial feature image; X C represents the deep extended feature image; Indicates that the kernel size is The depth convolution operation; Indicates that the kernel size is The operation of depth convolution; G C represents the attention feature image; W 1×1 Represents a convolution operation with a kernel size of 1×1; represents the enhanced spatial feature image; Represents an element-wise product operation.

[0123] The present invention provides a PPE compliance detection device based on improved YOLOv8, which obtains personal protective equipment (PPE) sample data in a construction site and annotates the PPE sample data to obtain an initial PPE data set, performs online mixed data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE data set to obtain an enhanced PPE data set, and replaces the feature fusion module in the original YOLOv8 model backbone network with an improved efficient multi-scale enhancement C2f-EMSEC module, and replaces the fast spatial pyramid pool in the original YOLOv8 model backbone network with an improved efficient multi-scale enhancement C2f-EMSEC module. The SPPF module is replaced by an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced into the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model, so as to train the improved PPE compliance detection model through the enhanced PPE data set to obtain an optimized PPE compliance detection model; then, the image to be detected is detected according to the optimized PPE compliance detection model to obtain a more accurate PPE compliance detection result, which is beneficial to improving the accuracy of the PPE compliance detection result while improving the detection efficiency of the PPE wearing detection process.

[0124] For the specific definition of the PPE compliance detection device based on improved YOLOv8, please refer to the definition of the PPE compliance detection method based on improved YOLOv8 above, which will not be repeated here. Each module in the above-mentioned PPE compliance detection device based on improved YOLOv8 can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0125] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it implements a function or step on the server side of a PPE compliance detection method based on an improved YOLOv8.

[0126] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes a function or step on the client side of a PPE compliance detection method based on an improved YOLOv8.

[0127] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0128] Obtaining personal protective equipment (PPE) sample data in a construction site, and annotating the PPE sample data to obtain an initial PPE data set;

[0129] Performing online hybrid data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE dataset to obtain an enhanced PPE dataset;

[0130] The feature fusion module in the original YOLOv8 model backbone network is replaced with an improved efficient multi-scale enhanced C2f-EMSEC module, the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network is replaced with an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model;

[0131] The improved PPE compliance detection model is trained by enhancing the PPE dataset to obtain an optimized PPE compliance detection model;

[0132] The image to be detected is detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result.

[0133] The present invention provides a computer device, which obtains personal protective equipment (PPE) sample data in a construction site and annotates the PPE sample data to obtain an initial PPE data set, performs online mixed data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE data set to obtain an enhanced PPE data set, replaces a feature fusion module in a backbone network of an original YOLOv8 model with an improved efficient multi-scale enhanced C2f-EMSEC module, replaces a fast spatial pyramid pooling SPPF module in the backbone network of the original YOLOv8 model with an improved large separable fast spatial pyramid pooling LSPPF module, and introduces an improved bidirectional feature pyramid network BPNet at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model, thereby training the improved PPE compliance detection model through the enhanced PPE data set to obtain an optimized PPE compliance detection model; and then detects an image to be detected according to the optimized PPE compliance detection model to obtain a more accurate PPE compliance detection result, which is beneficial to improving the accuracy of the PPE compliance detection result while improving the detection efficiency of the PPE wearing detection process.

[0134] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0135] Obtaining personal protective equipment (PPE) sample data in a construction site, and annotating the PPE sample data to obtain an initial PPE data set;

[0136] Performing online hybrid data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE dataset to obtain an enhanced PPE dataset;

[0137] The feature fusion module in the original YOLOv8 model backbone network is replaced with an improved efficient multi-scale enhanced C2f-EMSEC module, the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network is replaced with an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model;

[0138] The improved PPE compliance detection model is trained by enhancing the PPE dataset to obtain an optimized PPE compliance detection model;

[0139] The image to be detected is detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result.

[0140] The present invention provides a computer-readable storage medium. The personal protective equipment (PPE) sample data in a construction site is obtained, and the PPE sample data is annotated to obtain an initial PPE data set, and the initial PPE data set is enhanced by online mixed data enhancement based on data mixing Mixup and data enhancement Mosaic to obtain an enhanced PPE data set, and a feature fusion module in an original YOLOv8 model backbone network is replaced with an improved efficient multi-scale enhanced C2f-EMSEC module, and a fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network is replaced with an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced into the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model, so that the improved PPE compliance detection model is trained by the enhanced PPE data set to obtain an optimized PPE compliance detection model; and then, the image to be detected is detected according to the optimized PPE compliance detection model to obtain a more accurate PPE compliance detection result, which is beneficial to improving the accuracy of the PPE compliance detection result while improving the detection efficiency of the PPE wearing detection process.

[0141] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0143] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0144] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.< / h> < / w> < / y> < / x> < / c> < / occlusion> < / truncation> < / score>

Claims

1. A PPE compliance detection method based on improved YOLOv8, characterized in that: The method comprises: Obtaining personal protective equipment (PPE) sample data in a construction site, and annotating the PPE sample data to obtain an initial PPE data set; Performing online hybrid data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE dataset to obtain an enhanced PPE dataset; The feature fusion module in the original YOLOv8 model backbone network is replaced with an improved efficient multi-scale enhanced C2f-EMSEC module, the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network is replaced with an improved large separable fast spatial pyramid pooling LSPPF module, and an improved bidirectional feature pyramid network BPNet is introduced at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model; The improved PPE compliance detection model is trained by enhancing the PPE dataset to obtain an optimized PPE compliance detection model; The image to be detected is detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result.

2. According to a PPE compliance detection method based on improved YOLOv8 according to claim 1, it is characterized in that: The method of detecting the image to be detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result includes: Preprocess the image to be detected to obtain an initial input image; Inputting the initial input image into the optimized PPE compliance detection model, capturing spatial features at multiple scales through the C2f-EMSEC module, and obtaining a multi-scale fused spatial feature image; The multi-scale fusion spatial feature image is subjected to adaptive feature extraction through an LSPPF module to obtain an enhanced spatial feature image; Inputting the enhanced spatial feature image into the neck layer network of the optimized PPE compliance detection model for encoding to obtain a target feature image; A PPE compliance detection result is obtained according to the target feature image.

3. A PPE compliance detection method based on improved YOLOv8 according to claim 2, characterized in that: The initial input image is input into the optimized PPE compliance detection model, and spatial features are captured at multiple scales by the C2f-EMSEC module to obtain a multi-scale fused spatial feature image, including: Inputting the initial input image into the optimized PPE compliance detection model, dividing the initial input image into K heads in the channel dimension through the C2f-EMSEC module, and obtaining a set of head feature images; Applying a corresponding convolution kernel to perform convolution processing on each head feature image in the head feature image set to obtain a multi-scale feature image set; Performing feature stitching on corresponding multi-scale feature images in the multi-scale feature image set to obtain a stitching feature image set with independent features in each channel; Each spliced ​​feature image in the spliced ​​feature image set is convolved point by point through 1×1 convolution to achieve fusion of channel information and obtain a multi-scale fused spatial feature image.

4. A PPE compliance detection method based on improved YOLOv8 according to claim 3, characterized in that: The step of performing adaptive feature extraction on the multi-scale fusion spatial feature image through an LSPPF module to obtain an enhanced spatial feature image includes: The multi-scale fusion spatial feature image is subjected to local spatial feature extraction by a deep convolution block in an LSPPF module to obtain a local spatial feature image; Convolutionally processing the local spatial feature image through a depth-expanded convolution block to obtain a depth-expanded feature image; Performing a 1×1 kernel convolution on the depth-extended feature image to obtain an attention feature image; The attention feature image is element-wise multiplied with the multi-scale fusion spatial feature image to obtain an enhanced spatial feature image.

5. A PPE compliance detection method based on improved YOLOv8 according to claim 4, characterized in that: The step of performing adaptive feature extraction on the multi-scale fusion spatial feature image through an LSPPF module to obtain an enhanced spatial feature image includes: The step of performing adaptive feature extraction on the multi-scale fused spatial feature image through the LSPPF module to obtain an enhanced spatial feature image is implemented by the following formula: Among them, Z C represents the local spatial feature image; C represents the number of input channels; H represents the height of the multi-scale fusion spatial feature image in the spatial dimension; W represents the width of the multi-scale fusion spatial feature image in the spatial dimension; Indicates the operation of depth convolution with a kernel size of (2d-1)×1; * indicates the convolution operation; represents the operation of depthwise convolution with a kernel size of 1×(2d-1); F C represents the multi-scale fusion spatial feature image; X C represents the deep extended feature image; Indicates that the kernel size is The operation of depth convolution; Indicates that the kernel size is The operation of depth convolution; G C represents the attention feature image; W 1×1 Represents a convolution operation with a kernel size of 1×1; represents the enhanced spatial feature image; Represents an element-wise product operation.

6. A PPE compliance detection device based on improved YOLOv8, characterized in that: The PPE compliance detection device based on improved YOLOv8 includes: An acquisition module is used to acquire sample data of personal protective equipment (PPE) in a construction site, and to annotate the sample data of PPE to obtain an initial PPE data set; The first processing module is used to perform online hybrid data enhancement based on data mixing Mixup and data enhancement Mosaic on the initial PPE data set to obtain an enhanced PPE data set; The second processing module is used to replace the feature fusion module in the original YOLOv8 model backbone network with an improved efficient multi-scale enhanced C2f-EMSEC module, replace the fast spatial pyramid pooling SPPF module in the original YOLOv8 model backbone network with an improved large separable fast spatial pyramid pooling LSPPF module, and introduce an improved bidirectional feature pyramid network BPNet at the neck of the original YOLOv8 model to obtain an improved PPE compliance detection model; A third processing module is used to train the improved PPE compliance detection model by enhancing the PPE data set to obtain an optimized PPE compliance detection model; The detection module is used to detect the image to be detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result.

7. A PPE compliance detection device based on improved YOLOv8 according to claim 6, characterized in that: The detection module is used to detect the image to be detected according to the optimized PPE compliance detection model to obtain the PPE compliance detection result, specifically for: Preprocess the image to be detected to obtain an initial input image; Inputting the initial input image into the optimized PPE compliance detection model, capturing spatial features at multiple scales through the C2f-EMSEC module, and obtaining a multi-scale fused spatial feature image; The multi-scale fusion spatial feature image is subjected to adaptive feature extraction through an LSPPF module to obtain an enhanced spatial feature image; Inputting the enhanced spatial feature image into the neck layer network of the optimized PPE compliance detection model for encoding to obtain a target feature image; A PPE compliance detection result is obtained according to the target feature image.

8. A PPE compliance detection device based on improved YOLOv8 according to claim 7, characterized in that: The detection module is used to input the initial input image into the optimized PPE compliance detection model, capture spatial features at multiple scales through the C2f-EMSEC module, and obtain a multi-scale fused spatial feature image, specifically for: Inputting the initial input image into the optimized PPE compliance detection model, dividing the initial input image into K heads in the channel dimension through the C2f-EMSEC module, and obtaining a set of head feature images; Applying a corresponding convolution kernel to perform convolution processing on each head feature image in the head feature image set to obtain a multi-scale feature image set; Performing feature stitching on corresponding multi-scale feature images in the multi-scale feature image set to obtain a stitching feature image set with independent features in each channel; Each spliced ​​feature image in the spliced ​​feature image set is convolved point by point through 1×1 convolution to achieve fusion of channel information and obtain a multi-scale fused spatial feature image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the PPE compliance detection method based on improved YOLOv8 is implemented as described in any one of claims 1 to 5.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the PPE compliance detection method based on improved YOLOv8 is implemented as described in any one of claims 1 to 5.

Images