A mask-wearing detection method fusing 3D attention mechanism and a hollow convolution

Abstract

The application relates to a mask wearing detection method fusing a 3D attention mechanism and a hollow convolution, relates to the technical field of machine vision target detection, and solves the problem of the existing detection method which needs to provide a detection method with fast training speed, high detection precision and fast detection speed, and comprises the following steps: labeling whether wearing a mask is standard for a first data set; establishing an improved YOLOv5 deep learning network model; using a second data set to pre-train the improved YOLOv5 deep learning network model to obtain optimal pre-training model weights; using a transfer learning mode to initialize parameters of the improved YOLOv5 deep learning network model with the optimal pre-training model weights, using the first data set to train the model, and obtaining a mask wearing detection model. The application balances the two indexes of detection precision and detection speed, improves the detection precision of the model under the premise of not affecting the detection speed, and can meet the real-time detection demand of mask wearing.

Description

Technical Field

[0001] This invention relates to the field of machine vision target detection technology, specifically to a mask-wearing detection method that integrates 3D attention mechanism and dilated convolution. Background Technology

[0002] Under the influence of the COVID-19 virus, wearing masks has become an essential daily preventative measure. Correct mask-wearing in public places is highly effective in preventing the spread of the virus. However, due to a lack of awareness among some individuals, improper mask-wearing (including not wearing masks at all) is common. To reduce the risk of an outbreak, it is necessary to conduct real-time monitoring of mask-wearing in public places. However, the dense crowds, small size of individuals, and obstructions in public places undoubtedly make mask-wearing monitoring more challenging.

[0003] In the *Journal of Graphics*, Li Xiaobo et al. proposed a YOLOv5 mask detection algorithm that integrates attention mechanisms. This algorithm integrates four attention mechanisms—SE, CBAM, CA, and NAM—at the backbone feature extraction network. Experiments showed that the system integrating the CBAM attention mechanism achieved the greatest improvement in detection accuracy, and the GIOU loss function was selected as the bounding box loss function. The CBAM attention mechanism combines channel attention and spatial attention, and performs global pooling on the channels, incorporating positional information. Therefore, introducing the CBAM attention mechanism yields better performance, and its accuracy improvement is the most significant. However, CBAM has a high computational cost and is difficult to converge, resulting in a significant loss in model detection speed, making it difficult to achieve real-time detection. Although the GIOU loss function adds a way to measure the intersection scale between the target box and the predicted box, its convergence speed is slow, and when the predicted box is contained within the target box, GIOU completely degenerates into IOU, affecting localization accuracy.

[0004] To detect whether masks are worn correctly, a detection method that is fast to train, highly accurate, and fast is needed. Summary of the Invention

[0005] In view of the above problems, the present invention provides a mask wearing detection method that integrates 3D attention mechanism and dilated convolution.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] A mask-wearing detection method integrating 3D attention mechanism and dilated convolution includes:

[0008] The first dataset was labeled to indicate whether the mask-wearing was standardized. All images in the first dataset included information about whether the mask was worn on the face.

[0009] An improved YOLOv5 deep learning network model was established. The improvements to the YOLOv5 deep learning network model are as follows: a small object detection layer was added, a SimAM attention mechanism layer was added, the max pooling layer in the SPPF module was replaced by a dilated convolutional layer, α-CIOU was used as the bounding box loss function, and BCEWithLogitsLoss was used as the confidence loss function.

[0010] The improved YOLOv5 deep learning network model was pre-trained using the second dataset to obtain the optimal pre-trained model weights.

[0011] Training the mask-wearing detection model: Using transfer learning, the parameters of the improved YOLOv5 deep learning network model are initialized with the optimal pre-trained model weights. The improved YOLOv5 deep learning network model is then trained using the first dataset to obtain the mask-wearing detection model.

[0012] The beneficial effects of this invention are:

[0013] This invention presents a mask-wearing detection method integrating 3D attention mechanisms and dilated convolution. It improves multi-scale detection capabilities by adding a small target detection layer; enhances detection accuracy without compromising detection speed by introducing the SimAM attention mechanism; improves detection accuracy by incorporating SPPF structures into dilated convolution to learn contextual information; and accelerates model training and detection speed by improving the bounding box loss function and confidence loss function. This invention balances detection accuracy and speed, improving model accuracy without affecting detection speed, thus meeting the real-time detection requirements for mask wearing. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the overall implementation of a mask-wearing detection method that integrates 3D attention mechanism and dilated convolution according to the present invention.

[0015] Figure 2 This is a diagram of the network structure based on the improved YOLOv5 of this invention.

[0016] Figure 3 This is a schematic diagram of the SimAM attention mechanism used in this invention.

[0017] Figure 4 This is a structural diagram of the SPPF-D module of the present invention. Detailed Implementation

[0018] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0020] A mask-wearing detection method integrating 3D attention mechanism and dilated convolution includes:

[0021] The first dataset was labeled to indicate whether the mask-wearing was standardized. All images in the first dataset included information about whether the mask was worn on the face.

[0022] An improved YOLOv5 deep learning network model was established. The improvements to the YOLOv5 deep learning network model are as follows: a small object detection layer was added, a SimAM attention mechanism layer was added, the max pooling layer in the SPPF module was replaced by a dilated convolutional layer, α-CIOU was used as the bounding box loss function, and BCEWithLogitsLoss was used as the confidence loss function.

[0023] The improved YOLOv5 deep learning network model was pre-trained using the second dataset to obtain the optimal pre-trained model weights.

[0024] Using transfer learning, the parameters of the improved YOLOv5 deep learning network model are initialized with the optimal pre-trained model weights. The improved YOLOv5 deep learning network model is then trained using the first dataset to obtain the optimal detection model, which is the mask-wearing detection model.

[0025] The following section details a mask-wearing detection method that integrates 3D attention mechanisms and dilated convolution. Figure 1 It includes the following steps:

[0026] Step 1: Filter and label the first dataset, dividing it into training, validation, and test sets. All images in the dataset include information about whether faces are wearing masks. Labels are categorized into two types: those showing proper mask-wearing and those showing improper mask-wearing, including not wearing a mask.

[0027] Face images with and without masks in various scenarios were manually selected from the WIDER FACE and MAPA datasets. Images were also collected using search engines and video extraction methods to obtain the dataset. The first dataset was specifically filtered as follows: only one image was retained from those with a similarity (i.e., the S(x,y) value below) reaching a preset threshold. Structural similarity (SSIM) was calculated for similar images. The SSIM index, calculated using a formula, is between 0 and 1. Images with an index value greater than 0.85 are defined as similar. Only one similar image is selected to reduce duplicate data. The structural similarity calculation formula is as follows:

[0028]

[0029] Where x and y represent an image; C1 and C2 are constants, typically C1 is 0.01 and C2 is 0.03, μ x σ represents the average brightness of graph x. x To show the contrast of Figure x, μ y σ represents the average brightness of graph y. y The contrast of image y is shown. In this embodiment, after filtering, 6120 image data points were finally obtained.

[0030] The selected image set was labeled, categorized into two types: one for proper mask wearing and the other for improper mask wearing, including not wearing a mask. The LabelImg software was used to label the dataset in YOLO format, with image labels divided into two categories: mask (properly worn mask) and nomask (not wearing a mask or improperly wearing a mask). After labeling, each image corresponded to a txt file with the same name.

[0031] The labeled image dataset was divided into training, validation, and test sets in a ratio of 8:1:1.

[0032] Step 2: Based on the YOLOv5 deep learning network model, build an improved YOLOv5 deep learning network model. The improvements to the original YOLOv5 deep learning network model are as follows: add a small object detection layer to detect small objects; add a SimAM attention mechanism layer; replace the max pooling layer with a dilated convolutional layer in the SPPF module; use α-CIOU as the bounding box loss function; and use BCEWithLogitsLoss as the confidence loss function.

[0033] The original YOLOv5 network has three detection layers, using three different sized feature maps to detect targets of different sizes. These feature maps are obtained through downsampling at 8x, 16x, and 32x, resulting in 80×80, 40×40, and 20×20 feature maps, respectively. The three detection layers correspond to the detection of the 80×80, 40×40, and 20×20 feature maps. In the feature extraction pyramid network, downsampling at 32x results in the largest receptive field, mapping a larger area of ​​the original image, making it more suitable for predicting larger targets. Similarly, downsampling at 16x and 8x is more suitable for medium and small targets. However, the multiple convolutional processes in the feature extraction and fusion stages can cause deep networks to lose some target location information, while shallower networks retain more accurate target location information. Considering that some targets in the image occupy a small proportion, a detection layer for small targets is added to the improved network to detect these objects. Simultaneously, a new fusion circuit is introduced in the Neck part of the small target detection layer to fuse low-level features (information from shallow network targets) with high-level features (information from deep network targets). This improves the network model's multi-scale detection capability; the feature map output by the small target detection layer is 160×160, enabling the detection of small targets with receptive fields larger than 4×4. Figure 2 As shown. Figure 2The improved YOLOv5 deep learning network model is as follows: Images from the training set or the image to be detected are used as input to the Focus layer; the output of the Focus layer is used as input to the first Conv layer; the output of the first Conv layer is used as input to the first C3 convolutional layer; the output of the first C3 convolutional layer is used as input to the second Conv layer and the third Concat layer; the output of the second Conv layer is used as input to the second C3 convolutional layer; the output of the second C3 convolutional layer is used as input to the third Conv layer and the second Concat layer; the output of the third Conv layer is used as input to the third C3 convolutional layer; and the output of the third C3 convolutional layer is used as input to the first SimAM attention mechanism layer and the first Concat layer. The input to the first SimAM attention mechanism layer is used as the input to the fourth Conv layer. The output of the fourth Conv layer is used as the input to the fourth C3 convolutional layer. The output of the fourth C3 convolutional layer is used as the input to the second SimAM attention mechanism layer. The output of the second SimAM attention mechanism layer is used as the input to the improved SPPF module. The output of the improved SPPF module is used as the input to the fifth Conv layer. The output of the fifth Conv layer is used as the input to the first Upsample layer and the sixth Concat layer. The output of the first Upsample layer is used as the input to the first Concat layer. The output of the first Concat layer is used as the input to the fifth C3 convolutional layer. The output of the fifth C3 convolutional layer serves as the input to the sixth Conv layer. The output of the sixth Conv layer serves as the input to the second Upsample layer and the fifth Concat layer. The output of the second Upsample layer serves as the input to the second Concat layer. The output of the second Concat layer serves as the input to the sixth C3 convolutional layer. The output of the sixth C3 convolutional layer serves as the input to the seventh Conv layer. The output of the seventh Conv layer serves as the input to the third Upsample layer and the fourth Concat layer. The output of the third Upsample layer serves as the input to the third Concat layer. The output of the third Concat layer serves as the input to the seventh C3 convolutional layer. The output of the seventh C3 convolutional layer serves as... The inputs to the eighth Conv layer and the first Conv2d layer are used. The output of the first Conv2d layer is the detected small target, which serves as the small target detection head. The output of the eighth Conv layer is the input to the fourth Concat layer. The output of the fourth Concat layer is the input to the eighth C3 convolutional layer. The output of the eighth C3 convolutional layer is the input to the ninth Conv layer and the second Conv2d layer. The second Conv2d layer outputs an 80×80 image. The output of the ninth Conv layer is the input to the fifth Concat layer. The output of the fifth Concat layer is the input to the ninth C3 convolutional layer. The output of the ninth C3 convolutional layer is the input to the tenth Conv layer and the third Conv2d layer.The third Conv2d layer outputs a 40×40 image. The output of the tenth Conv layer serves as the input to the sixth Concat layer. The output of the sixth Concat layer serves as the input to the tenth C3 convolutional layer. The output of the tenth C3 convolutional layer serves as the input to the fourth Conv2d layer. The fourth Conv2d layer outputs a 20×20 image.

[0034] We introduce the 3D attention mechanism SimAM. SimAM is a parameter-free attention mechanism, unlike existing 1D channel attention and 2D spatial attention. It is simple and efficient, does not burden the model, and improves the model's detection accuracy without compromising its detection speed. See details... Figure 3 (Channel, Height, Width, and 3D-weight represent the channel, height, width, and 3D attention weights, respectively). By adding two SimAM attention mechanism layers, there is no need for additional parameters to derive the 3D attention weights. Attention weights are calculated using a single Energy function. Each point in the feature is assigned a separate scalar weight using these 3D weights. This unified weighting makes the features more global. The calculation process is as follows:

[0035]

[0036] The output is the enhanced feature X′, where X is the input feature, ⊙ is the dot product operation, and the sigmoid function is used to limit the excessively large values ​​in E. E is the energy function for each channel, and the calculation process is as follows:

[0037]

[0038] Where t is the value of the input feature, t∈X, λ is a constant 1e-4, and μ and σ 2 Let X represent the mean and variance of each channel in X, respectively.

[0039] The SPPF (Spatial Pyramid Pooling-Fast) module uses a 5×5 max-pooling layer. While this expands the receptive field, it reduces the resolution of the feature maps, which is detrimental to network learning. Dilated convolutions, on the other hand, have advantages such as learning contextual information, expanding the receptive field without reducing feature resolution. This invention replaces Maxpooling with dilated convolutions of different dilation rates. Due to the influence of the receptive field module, the improved SPPF module replaces the max-pooling layer with dilated convolutional layers. The improved SPPF structure contains four branches, which are referred to as the SPPF-D module in this invention. The two branches on the left extract contextual information from distant locations, and the two branches on the right extract contextual information from adjacent locations. See details... Figure 4The SPPF-D module is as follows: the output of the second SimAM attention mechanism layer serves as the input to the eleventh Conv layer (with a 1*1 kernel). The output of the eleventh Conv layer serves as the input to four branches, namely, the first dilated convolutional layer, the second dilated convolutional layer, the fourteenth Conv layer, and the fifteenth Conv layer. The output of the first dilated convolutional layer serves as the input to the twelfth Conv layer. The output of the second dilated convolutional layer serves as the input to the thirteenth Conv layer. The output of the fourteenth Conv layer serves as the input to the third dilated convolutional layer. The output of the fifteenth Conv layer serves as the input to the fourth dilated convolutional layer. The outputs of the second SimAM attention mechanism layer, the third dilated convolutional layer, the fourth dilated convolutional layer, the twelfth Conv layer, and the thirteenth Conv layer together serve as the input to the seventh Concat layer. The output of the seventh Concat layer serves as the input to the sixteenth Conv layer. The output of the sixteenth Conv layer serves as the input to the fifth Conv layer.

[0040] This invention replaces the original bounding box loss function CIOU with α-CIOU, enabling better boundary regression. It combines the power loss function α-IOU with CIOU, using α-CIOU as the bounding box loss function. The α-IOU formula is as follows:

[0041]

[0042] Where α is a power parameter, a value that can be manually adjusted and is always greater than 0, and IOU is the intersection-over-union ratio. α-IOU allows for more flexible accuracy adjustment when the model performs bounding box regression at different levels by changing the value of α. The CIOU formula is as follows:

[0043]

[0044] Where β is the weighting coefficient, and v is the similarity measured by aspect ratio. It is the ratio of the distance between the center points of the two bounding boxes to the diagonal distance. In summary, the improved bounding box loss function α-CIOU is as follows:

[0045]

[0046] in, It is the power of α, which is the ratio of the distance between the center points of the two borders to the distance between their diagonals.

[0047] This invention replaces the original confidence loss function CrossEntroyLoss with BCEWithLogitsLoss. YOLOv5 uses two types of loss functions to calculate the confidence loss function CrossEntroyLoss, with the following formulas:

[0048] CrossEntroyLoss=LogSoftmax+NLLLoss (7)

[0049] This loss function is generally used for multi-class classification. Since the mask-wearing detection studied in this invention is a binary classification problem, CrossEntroyLoss is no longer applicable. Instead, BCEWithLogitsLoss (binary cross-entropy loss with Logits) is used to solve single-label binary classification or multi-label binary classification problems. The formula is as follows:

[0050] BCEWithLogitsLoss=Sigmoid+BCELoss (8)

[0051] Step 3: The second dataset uses the Microsoft CoCo dataset. The improved YOLOv5 deep learning network model obtained in Step 2 is pre-trained using the Microsoft CoCo (Microsoft Common Objects in Context) dataset. The optimal pre-trained model weights are obtained through training and saved as the pre-trained model.

[0052] Step 4: Using transfer learning, initialize the parameters of the improved YOLOv5 deep learning network model with the optimal pre-trained model weights, and train the improved YOLOv5 deep learning network model using the training set obtained in Step 1 to obtain the optimal detection model.

[0053] Using transfer learning, the optimal pre-trained model weights are imported to initialize the model parameters. The improved YOLOv5 deep learning network model is trained using the training set obtained in step 1. This not only prevents gradient vanishing during training but also accelerates the convergence speed of the improved YOLOv5 detection model. The SGD optimizer is used to adaptively update the parameters. The training parameters are set to a learning rate of 0.01, momentum of 0.937, number of iterations of 400, and batch size of 32. Mosaic data augmentation is used to expand the data and improve the model's ability to detect small targets.

[0054] Step 4 specifically includes:

[0055] S4.1 Initialize the parameters of the improved YOLOv5 deep learning network model with the optimal pre-trained model weights;

[0056] S4.2. Perform image feature extraction and target recognition on the images in the training set;

[0057] S4.3 Calculate the loss function of the improved YOLOv5 deep learning network model;

[0058] S4.4 Use the SGD optimizer to update the parameters of the improved YOLOv5 deep learning network model;

[0059] S4.5 Calculate the MAP value (mean Average Precision, i.e., the average of the average precision of each class) of the current model on the validation set;

[0060] S4.6 When the MAP value at this time is greater than the preset best performance measurement value, or when the MAP value obtained by executing S4.5 in this round is greater than the MAP value obtained by executing S4.5 in the previous round, the improved YOLOv5 deep learning network model at this time is used to cover the original improved YOLOv5 deep learning network model, and the improved YOLOv5 deep learning network model at this time is saved as the current best model.

[0061] S4.7. Determine if the number of iterations is the final number. If not, return to S4.2 and re-execute S4.2. Then, re-execute S4.4 to S4.7. If the number of iterations has reached the final number, the iteration is complete, and the best detection model is obtained. That is, the current best model is taken as the best detection model.

[0062] Step 5: After training is complete, the test set data is fed into the optimal detection model to obtain the detection accuracy and detection speed.

[0063] Specifically, the test set data is input into the optimal detection model, the average detection time for each image is calculated, the average detection time includes the forward propagation and NMS (non-maximum suppression) time, and the detection results and average detection time are output.

[0064] Step 6: Detect mask-wearing status using the optimal detection model. Take the image to be detected as input and output the mask-wearing status, indicating whether the mask is worn correctly or incorrectly.

[0065] This invention improves multi-scale detection capabilities by adding a small target detection layer; it enhances detection accuracy without compromising detection speed by introducing the SimAM attention mechanism; it improves detection accuracy by incorporating the SPPF structure into dilated convolutions to learn contextual information; and it accelerates model training and improves detection speed by modifying the bounding box loss function and confidence loss function. This invention balances detection accuracy and speed, improving model accuracy without affecting detection speed, thus meeting the real-time detection requirements for mask wearing.

Claims

1. A method for detecting mask wearing that integrates 3D attention mechanisms and dilated convolution, characterized in that, include: The first dataset was labeled to indicate whether the mask-wearing was standardized. All images in the first dataset included information about whether the mask was worn on the face. An improved YOLOv5 deep learning network model was established. The improvements to the YOLOv5 deep learning network model are as follows: a small object detection layer was added, a SimAM attention mechanism layer was added, the max pooling layer in the SPPF module was replaced by a dilated convolutional layer, α-CIOU was used as the bounding box loss function, and BCEWithLogitsLoss was used as the confidence loss function. The improved YOLOv5 deep learning network model was pre-trained using the second dataset to obtain the optimal pre-trained model weights. Training the mask-wearing detection model: Using transfer learning, the parameters of the improved YOLOv5 deep learning network model are initialized with the optimal pre-trained model weights. The improved YOLOv5 deep learning network model is then trained using the first dataset to obtain the mask-wearing detection model.

2. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 1, characterized in that, The Neck portion of the small target detection layer can fuse information about targets in shallow and deep networks.

3. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 1, characterized in that, The SimAM attention mechanism layer calculates attention weights using an Energy function. The calculation of the SimAM attention mechanism layer is as follows: Where X is the input feature, and the output is the enhanced feature X. ′ ⊙ represents the dot product operation, and E is the energy function for each channel. The calculation process is as follows: Where t is the value of the input feature, t∈X, λ is a constant 1e-4, and μ and σ 2 Let X represent the mean and variance of each channel in X, respectively.

4. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 1, characterized in that, The formula for the bounding box loss function α-CIOU is: Where α represents the power parameter, and IOU represents the cross-connection ratio. α represents the ratio of the distance between the center points of the two borders to the distance between their diagonals, β is the weighting coefficient, and v is the similarity measure of aspect ratio.

5. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 1, characterized in that, Before labeling the first dataset, the first dataset is first filtered. Specifically, the filtering is as follows: only one image is kept among the images whose similarity reaches a preset threshold.

6. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 5, characterized in that, The similarity calculation formula is as follows: Where x and y represent an image, C1 and C2 are constants, and μ x σ represents the average brightness of graph x. x To show the contrast of Figure x, μ y σ represents the average brightness of graph y. y Let y be the contrast of the graph.

7. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 1, characterized in that, The first dataset is divided into a training set, a validation set, and a test set. The improved YOLOv5 deep learning network model is trained using the training set and the validation set, and the test set is used to test the mask-wearing detection model.

8. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 1, characterized in that, The method also includes a step of detecting mask-wearing status using an optimal detection model.

9. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 1, characterized in that, The training of the mask-wearing detection model specifically includes the following steps: S4.1 Initialize the parameters of the improved YOLOv5 deep learning network model with the optimal pre-trained model weights; S4.

2. Perform image feature extraction and target recognition on the images in the training set; S4.3 Calculate the loss function of the improved YOLOv5 deep learning network model; S4.4 Update the parameters of the improved YOLOv5 deep learning network model using the SGD optimizer; S4.5 Calculate the MAP value of the current model on the validation set; S4.6 When the MAP value at this time is greater than the preset best performance measurement value, or when the MAP value obtained by executing S4.5 in this round is greater than the MAP value obtained by executing S4.5 in the previous round, the improved YOLOv5 deep learning network model at this time is used to cover the original improved YOLOv5 deep learning network model. S4.7 Determine if the number of iterations is the final number. If not, return to S4.2 and re-execute S4.2 to S4.

7. If the number of iterations has reached the final number, the mask wearing detection model is obtained.

10. The mask-wearing detection method integrating 3D attention mechanism and dilated convolution as described in claim 1, characterized in that, The second dataset is the Microsoft CoCo dataset.

Images