A vision-based method for detecting whether a mask is worn correctly

Abstract

The present application belongs to the technical field of artificial intelligence, and in particular relates to a method for detecting whether a mask is correctly worn based on visual technology, comprising the following steps: preparing a detection object data set; improving the algorithm for K-means clustering prior frame size in the existing YOLOv3-SPP algorithm to obtain an improved CBAM-YOLOv3-SPPF algorithm; training the CBAM-YOLOv3-SPPF algorithm using the data set prepared in S1, and feeding back the best weight file obtained after training to the CBAM-YOLOv3-SPPF algorithm to obtain a target detection network; using the target detection network obtained in step S3 to detect a target, and after combining the detected human face information, using a specific position skin color area measurement method to determine whether the target person correctly wears a mask. The present application has good robustness and generalization when actually detecting whether a target correctly wears a mask.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of visual technology in artificial intelligence, in particular, it is a method for detecting whether a mask is correctly worn based on visual technology. BACKGROUND

[0002] Currently, the most important way of spreading the new coronavirus is droplet transmission, so it is particularly important to correctly wear a mask in public places. In recent years, domestic public places have also begun to pay attention to health and safety, and efforts have been made to ensure that people in public places wear masks correctly. However, the existing mask inspection scheme in public places still has many problems, and there are many deficiencies in details, and the degree of automation is low.

[0003] The existing mask wearing detection has the following schemes:

[0004] (1) Mask wearing detection scheme based on manpower

[0005] Manpower inspection refers to the traditional manual method of having inspection personnel wear masks and other epidemic prevention equipment at the entrance and exit of public places. The judgment of whether the person entering or leaving wears a mask is made by the human eye.

[0006] (2) Mask wearing visual detection scheme based on traditional skin color discrimination

[0007] This scheme first needs to use a face detector to capture the target face. Then, the target face skin color area and non-skin color area are extracted and calculated using the HSV color space or YcrCb color space, and the target person is judged whether to wear a mask through a certain threshold relationship.

[0008] (3) Mask wearing visual detection scheme based on deep learning

[0009] By building a target detection neural network, a mask wearing data set is made, the target detection network is trained, and the corresponding target detector is obtained to directly detect whether the target person wears a mask.

[0010] The human security check whether to wear a mask scheme is simple in form and convenient to operate, but due to the relationship between people and people, it is easy to cause cross infection, and it is too subjective for the security personnel to judge whether the target person correctly wears the mask, and it is easy to appear a loophole; The mask wearing visual detection scheme based on traditional skin color discrimination first depends on the excellence of human detector, and secondly uses a single color space to identify the skin color in advance. The effect of mask discrimination for complex light and similar skin color is poor, which easily affects the judgment result, and at the same time, it cannot judge whether the target correctly wears the mask in the normal state; The mask wearing visual detection scheme based on deep learning is one of the more mainstream methods at present, but in most cases, the target detection network can only judge whether the target person wears the mask, and cannot judge whether the target correctly wears the mask. The mask wearing visual detection scheme based on deep learning depends on the quality of the standardization of the mask in the training set, and due to the difference of the target person wearing the mask in the actual detection, the robustness of this scheme is weak and easy to be disturbed, and the generalization is weak and difficult to popularize. SUMMARY

[0011] The main purpose of the present application is to overcome the defects of the existing visual detection and calibration technology, and a correct mask wearing detection method based on visual technology is designed. The present application designs an improved CBAM-YOLOv3-SPPF algorithm based on YOLOv3, and judges whether the target person correctly wears the mask by comparing the detection of key information such as eyes, nose and mouth in the face. The detection method designed by the present application mainly detects the front view, which not only greatly improves the detection accuracy, but also can effectively judge whether the target person correctly wears the mask.

[0012] The specific technical scheme adopted by the present application is as follows:

[0013] A correct mask wearing detection method based on visual technology, comprising the following steps:

[0014] S1: Make a detection object data set, randomly extract 10000 valid pictures in the existing Large-scale CelebFaces Attributes (CelebA) Dataset face attribute data set, use LabelImg software to calibrate the eyes, nose and mouth in the selected 10000 pictures, and randomly divide them into a training set of 8000 and a test set of 2000;

[0015] S2: The algorithm for K-means clustering prior box size in the existing YOLOv3-SPP algorithm is improved, the SPPF module is used to improve the YOLOv3-SPP network model, an extra large-size feature output graph is added in the detection head Head part, and the CBAM attention mechanism module is fused in the detection program module Head, and the improved CBAM-YOLOv3-SPPF algorithm is obtained;

[0016] S3: The CBAM-YOLOv3-SPPF algorithm is trained by using the data set prepared in step S1, and the best weight file obtained after training is fed back to the CBAM-YOLOv3-SPPF algorithm to obtain a target detection network;

[0017] S4: The target detection network is used to detect the target, and after combining the detected eye, nose, mouth and other information, a specific position skin color area measurement method is used to determine whether the target person correctly wears a mask;

[0018] Further improvement of the application, the process of S1 comprises:

[0019] S1.1, obtain a large number of high-definition pictures containing human face features:

[0020] Randomly extract 10,000 different effective human face pictures from the Large-scale CelebFaces Attributes (CelebA) Dataset human face attribute dataset using the written python script;

[0021] S1.2, 10,000 pictures are labeled to make a data set:

[0022] The eye, nose and mouth region of each face picture is drawn with a target marker frame using LabelImg software, the position information of the target in the image is the width and height information of the target marker frame (which can be indirectly obtained), and the coordinate information of the diagonal point of the target marker frame in the image coordinate system. The automatically generated.xml file cannot be trained by the YOLO network, and the xml file format is converted into a txt file format using a python script voc_yolo. The converted txt file contains: class ID number, target center point x coordinate / picture total width, target center point y coordinate / picture total height, target box width / picture total width, and target box height / picture total height. After conversion, it is divided into 8000 training sets and 2000 test sets.

[0023] Further improvement of the application, the process of S2 comprises:

[0024] S2.1, improve the K-means clustering algorithm, calculate the prior box size of YOLOv3-SPP:

[0025] The algorithm first uses the relationship between the distance of the cluster center and the roulette algorithm to select excellent initial cluster centers, then uses the K-means clustering algorithm for iterative updating, and finally adjusts the 12 anchor values clustered to seek the optimal anchor value;

[0026] S2.2, improvement of YOLOv3-SPP network model:

[0027] The CBAM-YOLOv3-SPPF network model obtained after improvement is composed of Backbone, Neck and Head three parts, wherein Backbone adopts DarkNet53 structure for feature extraction, replaces the original SPP module of Neck part with SPPF module, constitutes the combination of SPPF and FPN, enhances the feature expression ability of feature map and improves the inference speed, additionally adds a large size feature output map in the Head detection head part, improves the detection precision of small targets of the network, and adds CBAM attention mechanism module in four detection heads Head respectively, improves the network learning ability.

[0028] Further improvement of the application, the process of S3 comprises:

[0029] After the sample images in the training set are processed by the Mosaic data enhancement method, they are input into the CBAM-YOLOv3-SPPF algorithm, and the weight value is updated until the loss function tends to be stable and minimum, the weight file obtained by training is retained, and it is loaded into the CBAM-YOLOv3-SPPF algorithm to make a target detection network.

[0030] The loss function used in the application is CIOU_Loss function, which is composed of three parts: target confidence loss L conf (o,c), target positioning offset loss L CIoU and target classification loss L cla (O,C)

[0031] L(O,o,C,c,t,g)=λ1L conf (o,c)+λ2L cla (O,C)+λ3L CIoU (6)

[0032] Wherein, λ1, λ2, λ3 are balance coefficients, L conf (o,c) is the target confidence loss, L CIoU is the target positioning offset loss, and Lcla (O,C) represents the target classification loss. Specifically, different weights are added to the target confidence loss of the four detection heads in the improved CBAM-YOLOv3-SPPF network structure:

[0033]

[0034] In equation (7) above, α1, α2, α3, and α4 are weighting coefficients and satisfy the condition α1 > α2 > α3 > α4. The target confidence loss is calculated from the feature output maps of decreasing size, in descending order.

[0035] Among them, the target confidence loss is:

[0036]

[0037] Target confidence loss L conf (o,c) uses binary cross-entropy loss, where o i ∈{0,1} indicates whether the target actually exists in the predicted target bounding box i, where 0 indicates that it does not exist and 1 indicates that it exists. This represents the Sigmoid probability of predicting whether a target exists within the target bounding box i.

[0038] Target category loss:

[0039]

[0040] Target classification loss L cla (O,C) also uses binary cross-entropy loss, where O ij ∈{0,1} indicates whether the j-th type of target actually exists in the bounding box i of the predicted target, where 0 indicates that it does not exist and 1 indicates that it exists. This represents the Sigmoid probability of the j-th class of target within the target bounding box i predicted by the network.

[0041] Target positioning offset loss:

[0042]

[0043] Target positioning offset loss L CIoU The CIOU Loss function is used for calculation, where b pred b gt ρ represents the center point location of the predicted bounding box and the ground truth bounding box. 2 (b pred ,b gtLet α be the Euclidean distance between the centers of the two bounding boxes, and c be the diagonal distance between the bounding rectangles of the two bounding boxes. α can be understood as a weighting coefficient; by definition, the loss function optimizes towards areas with larger overlap. v measures the similarity in aspect ratio between the predicted and target bounding boxes, and w... gt h gt The width and height of the actual bounding box.

[0044] A further improvement to the present invention is that the process of S4 includes:

[0045] The trained object detection network is used to detect the target person (determining whether the detected target is eyes, nose, mouth, etc., all belong to the discrimination module of the YOLO algorithm, that is, the target score is greater than the set threshold. This step is only for application and has no originality, so it will not be described in detail). When the target is detected to have eyes, nose, and mouth at the same time, it is determined that the target is not wearing a mask. When the target is detected to have only eyes, it is further determined whether the target person is wearing a mask correctly. Take the width of the smallest rectangle outside the detected eye frames and twice the height, and take another rectangle below the smallest rectangle outside the smallest rectangle. This area is called the nose decision area. Calculate its area and denote it as S1. This invention designs an H-CrCb method to extract skin color in the nose decision area and calculate its skin color area, denoted as S2. If S2 > αS1 (α is the decision threshold, and the initial value is set to 0.4), it is considered that the target is not wearing a mask correctly. Otherwise, it is considered that the target person is wearing a mask correctly.

[0046] Compared with existing technical solutions, the present invention has the following beneficial effects:

[0047] (1) This invention improves upon the existing YOLOv3-SPP algorithm by designing an improved K-means clustering algorithm to solve for the prior anchor box size, thereby increasing the speed of K-means clustering prior boxes while also improving the clustering effect. Furthermore, the YOLOv3-SPP network model structure is improved and optimized, enhancing the detection accuracy for human eyes, nose, and mouth while maintaining a good detection speed.

[0048] (2) Compared with deep learning-based visual detection schemes for mask wearing, this invention is more flexible and versatile. It can not only detect whether a target is wearing a mask, but also determine with high accuracy whether the target is wearing a mask correctly. Furthermore, the determination of whether a target is wearing a mask correctly is no longer fixed; the degree of requirement for correct mask wearing can be dynamically changed by adjusting the decision threshold.

[0049] (3) The designed H-CrCb skin color extraction method can effectively and accurately detect skin color areas under various complex ambient light conditions. At the same time, it effectively overcomes the situation where white mask areas may be misdetected as skin color areas when there is insufficient light, thus improving the detection accuracy of whether the target is wearing a mask correctly;

[0050] (4) This invention does not employ a direct detection method for mask wearing, but instead uses an indirect method for detection and judgment. This method does not rely on the calibration quality of improperly worn masks in the training set. The solution in this invention has excellent robustness and generalization when actually detecting whether the target is wearing a mask correctly. Attached Figure Description

[0051] Figure 1 This is a flowchart of the present invention.

[0052] Figure 2 This is a flowchart of the improved K-means clustering method for solving anchors in this invention.

[0053] Figure 3 This is a block diagram of the improved CBAM-YOLOv3-SPPF algorithm model obtained in this invention.

[0054] Figure 4 This refers to the CBAM module added to the CBAM-YOLOv3-SPPF network model in this invention.

[0055] Figure 5 This is a flowchart of the network weight training process of the CBAM-YOLOv3-SPPF algorithm in this invention.

[0056] Figure 6 This is a flowchart illustrating the use of the H-CrCb color enhancement method in this invention to indirectly determine whether a target person is wearing a mask correctly.

[0057] Specific implementation

[0058] To enhance understanding of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. These embodiments are only used to explain the invention and do not limit the scope of protection of the invention.

[0059] like Figure 1 As shown, the present invention provides a method for detecting whether a mask is worn correctly based on visual technology, which includes the following specific steps:

[0060] S1: Acquire a large number of high-resolution images containing human facial features, use LabelImg software to label the detection targets, and create the dataset required for neural network training.

[0061] S2: Improve the existing YOLOv3-SPP algorithm for determining the size of the K-means clustering prior boxes. Use the SPPF module to improve the YOLOv3-SPP network model, adding a large-sized detection head. Integrate the CBAM attention mechanism module into the detection program module head to obtain the improved CBAM-YOLOv3-SPPF algorithm. The specific process is as follows:

[0062] S2.1 Calculating the Prior Box Size of YOLOv3-SPP Based on the Improved K-means Clustering Algorithm: According to the requirements of the detection head in the improved YOLOv3-SPP algorithm of this invention, 12 prior anchor boxes need to be successfully clustered. The diagonal coordinates of all bounding boxes are extracted from the initially generated .xml file using a Python script, and the width and height of all bounding boxes are calculated using the diagonal coordinates (calculation method: width = bottom right x-coordinate - top left x-coordinate, height = bottom right y-coordinate - top left y-coordinate).

[0063] This invention improves and optimizes the K-means clustering algorithm. The specific steps of the improved K-means clustering algorithm to solve for the YOLOv3-SPP prior anchor box size are as follows, and its flowchart is shown in the figure. Figure 2 As shown.

[0064] S2.1.1 Randomly select the width and height of one bounding box from all extracted bounding boxes as the first cluster center;

[0065] S2.1.2 Calculate the distance D = 1 - IOU from each of the remaining bounding boxes to the cluster center, and at the same time calculate the probability that each of the remaining bounding boxes will be selected as the next cluster center (the larger the distance D, the greater the probability of being selected). Then use the roulette wheel algorithm to determine the next cluster center.

[0066] S2.1.3, Repeat step S2.1.2 until 12 initial cluster centers are selected;

[0067] S2.1.4 Calculate the distance of each bounding box to the nearest cluster center and assign it to the cluster to which the nearest cluster center belongs;

[0068] S2.1.5 Calculate the median width and height of the bounding box for each cluster, and use it as the cluster center for the next iteration;

[0069] S2.1.6 Repeat steps S2.1.4 and S2.1.5 until the elements in each cluster no longer change, the clustering ends, and the final output is the value of 12 anchors;

[0070] S2.1.7. Perform minor mutation operations (addition and subtraction) on the width and height of the 12 obtained anchors, and evaluate the performance of the 12 mutated anchors (evaluation method: calculate the average distance from the bounding box in each cluster to the mutated cluster center; if the average value decreases, it indicates a better mutation effect), and retain the mutated anchor values. Perform 300 mutation operations and retain the best mutated anchor values;

[0071] Finally, the width and height dimensions of the 12 prior anchor boxes in this training set were successfully clustered, and the obtained width and height dimensions of the prior anchor boxes were used to replace the default prior anchor box size in the original YOLOv3 algorithm.

[0072] S2.2, Improved YOLOv3-SPP Network Model:

[0073] In this invention, the CBAM-YOLOv3-SPPF network model consists of a Backbone, Neck, and Head. The specific network model structure is as follows: Figure 3 As shown: The Backbone uses a DarkNet53 architecture for feature extraction. The original SPP module in the Neck section is replaced with an SPPF module, forming a combination of SPPF and FPN, which enhances the feature map's expressive power while improving inference speed. An additional large-size feature output map is added to the Head section of the detection head. After the improvement, the Head section has four output feature maps of different sizes, set to 16×16, 32×32, 64×64, and 128×128 respectively. Furthermore, a CBAM attention mechanism module is added to each of the four detection heads to improve the network's learning ability.

[0074] To simplify the description of the network model, the basic component of the CBAM-YOLOv3-SPPF network model is simply referred to as Convolutional. This Convolutional component consists of a Conv2d convolutional layer, a Batch Normalization layer, and a Leaky ReLU activation function. A residual unit, Residual, is defined, consisting of two paths: the main path passes through two Convolutionals, and the other path is the original output, which is then summed to obtain the final output. A residual component, ResX, is defined as consisting of one Convolutional unit and X residual units.

[0075] After the training set is input into DarkNet53, it passes through one Convolutional, Res1, Res2, Res8, Res8, and Res4 in sequence. The outputs of the first Res2, the first Res8, and the second Res8 respectively produce three layers of feature maps C1, C2, and C3.

[0076] After passing through the DarkNet53 structure and then through three convolutional components, the feature map enters the SPPF module. The SPPF module first uses a 1×1 max-pooling layer to output feature map F1. This feature map then uses a 5×5 max-pooling layer to output feature map F2. Feature map F2 then uses another 5×5 max-pooling layer to output feature map F3. Feature map F3 then uses another 5×5 max-pooling layer to output feature map F4. Finally, the feature maps F1, F2, F3, and F4 at different scales are concatenated before being output.

[0077] After passing through the SPPF module, we arrive at the FPN structure in the CBAM-YOLOv3-SPPF network model. The fused information feature map output from the SPPF module is processed by three convolutional components to obtain P1. P1 is then combined with C3 through one convolutional and one upsampling operation, followed by five convolutional components to obtain P2. P2 is then combined with C2 through one convolutional and one upsampling operation, followed by five convolutional components to obtain P3. P3 is then combined with C1 through one convolutional and one upsampling operation, followed by five convolutional components to obtain P4.

[0078] P1, P2, P3, and P4 are input into the detection head, and each is processed through a convolutional component, a CBAM attention mechanism module, and a 1×1 convolutional layer, resulting in output feature maps of different sizes. The sizes are set to 16×16, 32×32, 64×64, and 128×128, respectively.

[0079] The CBAM attention mechanism module is a sequential combination of the channel attention module and the spatial attention module, with the following structure: Figure 4 As shown, the channel attention module comes first, followed by the spatial attention module.

[0080] The Channel Attention module processes the input feature map through two parallel MaxPool max pooling layers and AvgPool average pooling layers, reducing the feature map from C×H×W to C×1×1. Then, it passes through a Shared MLP module, where the number of channels is first compressed to 1 / r of the original number, then expanded to the original number, followed by a ReLU activation function. The two outputs are then element-wise summed, and the result is passed through a Sigmoid activation function to obtain the Channel Attention output. This output is then multiplied by the original image to restore the feature map to C×H×W.

[0081] Spatial Attention Module: The input feature map is passed through the MaxPool layer and the AvgPool layer to obtain two 1×H×W feature maps. Then, the two feature maps are concatenated by the Concat operation, and then transformed into a 1-channel feature map by a 7×7 convolution. After passing through a Sigmoid, the Spatial Attention feature map is obtained. Finally, the output result is multiplied by the original image to restore it to the size of C×H×W.

[0082] The improved CBAM-YOLOv3-SPPF algorithm performs exceptionally well on the VOC dataset, and its inference computation speed has also been improved to some extent.

[0083] S3: Train the CBAM-YOLOv3-SPPF algorithm using the created dataset, and load the best weight file obtained after training into the CBAM-YOLOv3-SPPF algorithm to obtain the object detection network;

[0084] In this embodiment, a total of 8000 training images and 2000 test images were divided. The training set was input into the CBAM-YOLOv3-SPPF network to continuously train the network model and optimize the network's weight file. The test set was used to calculate the average precision (mAP), and the mAP value was used to determine whether the network model met the target. Simultaneously, the network model weight file with the highest mAP was saved and loaded into the CBAM-YOLOv3-SPPF algorithm to obtain the object detection network. The training process is as follows: Figure 5 As shown.

[0085] In this embodiment, the Mosaic data augmentation method was used during training. Four images were randomly selected from the training set each time, scaled or cropped, and then logically stitched together to form a new image. The label data of the new image was then processed to correspond with the new image information. The image generated after Mosaic data augmentation was then resized to a fixed size of 512×512.

[0086] The 512×512 image generated after data augmentation is input into the CBAM-YOLOv3-SPPF network model. First, the DarkNet53 feature extraction network extracts features from the input image. The outputs of the first Res2, the first Res8, and the second Res8 in DarkNet53 are used to extract feature maps of three levels, C1, C2, and C3, for subsequent feature fusion.

[0087] The feature map directly output by the feature extraction network DarkNet53 is processed by three Convolutional components and then enters the SPPF module. The SPPF module uses multiple combined max pooling layers to perform multi-scale feature fusion and outputs a fused feature map.

[0088] The fused feature maps then enter the FPN structure for the next step of feature fusion. After processing by three convolutional components, a 16×16 feature map P1 is obtained. Feature map P1 is then processed by one convolutional component and one upsampling component, combined with a feature map C3 output by DarkNet53 through a concatenate operation, and then processed by five convolutional components to obtain a 32×32 feature map P2. Feature map P2 is then processed by one convolutional component and one upsampling component, combined with a feature map C2 output by DarkNet53 through a concatenate operation, and then processed by five convolutional components to obtain a 64×64 feature map P3. Feature map P3 is then processed by one convolutional component and one upsampling component, combined with a feature map C1 output by DarkNet53 through a concatenate operation, and then processed by five convolutional components to obtain a 128×128 feature map P4.

[0089] The feature maps P1, P2, P3, and P4 are then input into the YOLO detection head, passing through one convolutional component, one CBAM attention mechanism module, and one 1×1 convolutional layer, respectively, to obtain effective output feature maps of different sizes. Their sizes are set to 16×16, 32×32, 64×64, and 128×128 to detect large, medium, and small targets, respectively.

[0090] In this embodiment, the shapes of the final output effective feature maps are (16, 16, 24), (32, 32, 24), (64, 64, 24), and (128, 128, 24). The last dimension, 24, contains 4+1+3, representing x_offset, y_offset, h and w, confidence score, and classification result, respectively. (The training set of this invention has 3 types.)

[0091] The calculated prediction results are then decoded to obtain the predicted bounding boxes on the original image. Finally, the bounding boxes and their scores for each class with scores greater than a threshold are selected, and non-maximum suppression is performed using the box positions and scores to remove redundant detection boxes.

[0092] During training, the loss function is calculated by comparing the predicted data with the actual data of the training images. The loss function used in this invention is the CIOU_Loss function, which consists of three parts: target confidence loss L... conf (o,c), target positioning offset loss L CIoU And target classification loss L cla (O,C)

[0093] L(O,o,C,c,t,g)=λ1L conf (o,c)+λ2L cla (O,C)+λ3L CIoU (11)

[0094] Where λ1, λ2, λ3 are balance coefficients, and L conf (o,c) represents the target confidence loss, and L CIoU For target location offset loss, L cla (O,C) represents the target classification loss. Specifically, different weights are added to the target confidence loss of the four detection heads in the improved CBAM-YOLOv3-SPPF network structure:

[0095]

[0096] In equation (12) above, α1, α2, α3, and α4 are weighting coefficients and satisfy the condition α1 > α2 > α3 > α4. The target confidence loss is calculated from the feature output maps of decreasing size, in descending order.

[0097] The essence of its training is to continuously update the weights using the computational error until the loss function tends to be stable and minimized. Then, the weight file obtained from the training is retained and loaded into the CBAM-YOLOv3-SPPF algorithm to create an object detection network.

[0098] To further improve the training effect, the present invention also employs the following optimization methods: cosine annealing to adjust the learning rate, which initially increases and then decreases, thus accelerating the training speed; and stochastic gradient descent with momentum.

[0099] S4: Using the above target detection network to detect targets, and combining the detected information such as human eyes, nose, and mouth, a method based on skin color area is used to determine whether the target person is wearing a mask correctly.

[0100] The aforementioned target detection network is used to detect the eyes, nose, and mouth of a target person in real time. Based on the indirect detection method designed in this invention, it is used to determine whether the target person is wearing a mask correctly. The flowchart of the entire detection and judgment process is as follows: Figure 6 As shown.

[0101] S4.1 Determination of not wearing a mask: When the system successfully identifies that the target person has the features of eyes, nose and mouth, it determines that the target person is not wearing a mask.

[0102] S4.2 Determination of Incorrect Mask Wearing: Incorrect mask wearing is divided into two situations:

[0103] Scenario 1: When the target detection network successfully detects eye and nose features but not mouth features, the system determines that the target is not wearing a mask correctly.

[0104] Scenario 2: When the target detection network only detects eye features but not nose and mouth features, further analysis is needed to determine whether the target is wearing a mask correctly.

[0105] At this point, read the coordinates (x) of the upper left corner of the left eye socket. left y left ), the coordinates of the lower right corner of the right eye socket (x right y right Take the smallest bounding rectangle formed by the two eye sockets as the block, take the width of the smallest rectangle and twice the height, and take another rectangle below the smallest bounding rectangle of the two eye sockets as the nose decision area, with an area of ​​S1.

[0106] To better extract skin color and reduce the impact of complex lighting and other factors on the color enhancement results, the H-CrCb method was designed to select skin color pixels.

[0107] The H-CrCb method for extracting skin tone is a hybrid color extraction scheme that combines the HSV color model and the YCrCb color model.

[0108] The formula for converting from RGB space to HSV space is as follows:

[0109]

[0110] When using the HSV color space for skin color detection, it is more stable because HSV separates color into a single channel, namely the hue channel. The HSV color space is divided into hue (H), saturation (S), and lightness (V). The hue range of human skin color is indicated in the literature as follows: H∈[9°, 29°].

[0111] Rule 1: In HSV space, a pixel with a hue value H∈[9°, 29°] can be considered a skin color pixel.

[0112] Considering practical realities, masks currently available on the market come in three colors: blue, white, and black. Under Rule 1, the HSV space does not provide a very good distinction between white masks and skin tones, especially in low light conditions.

[0113] This situation can be effectively addressed by using the H-CrCb method to extract skin color from the YCrCb color space.

[0114] Formula for converting from RGB space to YCrCb space:

[0115]

[0116] in, Y represents the brightness of the light source, Cr represents the red component information, and Cb represents the blue component information.

[0117] The most significant characteristic of the YCrCb color space is its separation of luminance and chrominance, making it better suited for image processing. In the YCrCb color space, ignoring the influence of luminance, skin tones exhibit better clustering results. Some literature suggests that the CrCb values ​​for human skin tones should satisfy: Cr∈[140,175], Cb∈[100,120].

[0118] Rule 2: In the YCrCb space, if the red component Cr∈[140,175] and the blue component Cb∈[100,120] of a pixel, it can be regarded as a skin color pixel.

[0119] The H-CrCb method for extracting skin color means that if a pixel satisfies rule 1 or rule 2, then that pixel is considered a skin color pixel.

[0120] Then, the designed H-CrCb method is used to extract the skin color area in the nose decision region. The extracted skin color area is denoted as S2. If S2 > αS1 (α is the decision threshold, and the initial value is set to 0.4), it is considered that the target is not wearing a mask correctly.

[0121] S4.3 Determination of Correct Mask Wearing: When only eye feature information is detected on the target, and no nose and mouth feature information is detected on the target, and S2≤αS1 is satisfied, then the target is considered to be wearing a mask correctly.

[0122] The value of α can be changed according to the actual epidemic prevention and security requirements of public places. If the epidemic prevention and security requirements of the place are relatively strict, the value of α can be appropriately reduced and the mask wearing requirements can be increased.

[0123] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions, and these changes and improvements shall fall within the scope of the present invention as claimed.

Claims

1.A method for detecting whether a mask is correctly worn based on a vision technology, characterized in that, Comprise the following steps: S1: make detection object data set; S2: improve the algorithm of K-means clustering prior box size in existing YOLOv3-SPP algorithm, use SPPF module to improve YOLOv3-SPP network model, additionally increase a large size feature output graph in detection head Head part, and fuse CBAM attention mechanism module in detection program module Head, obtain improved CBAM-YOLOv3-SPPF algorithm; S3: using the data set made by S1, the CBAM-YOLOv3-SPPF algorithm is trained, the best weight file obtained after training is fed back to the CBAM-YOLOv3-SPPF algorithm to obtain the target detection network; S4: using the target detection network obtained in step S3 to detect the target, combining the detected human face information, using the method of measuring the area of specific position skin color to judge whether the target person correctly wears the mask or not; The specific process of S1 comprises: S1.1, obtain a large number of high-definition pictures containing human face features: use python script to randomly extract 10000 different effective human face pictures from public Large-scale CelebFaces Attributes (CelebA) Dataset face attribute dataset; S1.2: 10000 pictures are labeled and made into data set: use LabelImg software to draw target marker frame in eye, nose and mouth area of each face picture, the automatically generated.xml file cannot be trained by YOLO network, use python script voc_yolo to convert.xml file format into.txt file format, the converted.txt file contains: class ID number, target center point x coordinate / picture total width, target center point y coordinate / picture total height, target box width / picture total width, target box height / picture total height, after conversion, divide it into 8000 training sets and 2000 test sets; The specific process of S2 comprises: S2.1, improve K-means clustering algorithm, calculate the prior box size of YOLOv3-SPP: first, use the relationship between the distance of clustering center and the roulette to select the excellent initial clustering center, then use K-means clustering algorithm for iterative update, finally, adjust the 12 anchor values obtained by clustering to seek the optimal anchor value; S2.2, YOLOv3-SPP network model improvement: the CBAM-YOLOv3-SPPF network model obtained after improvement is composed of three parts of Backbone, Neck and Head, wherein the Backbone adopts the DarkNet53 structure to perform feature extraction, the SPP module of the original Neck part is replaced with the SPPF module to form the combination of SPPF and FPN, the feature expression ability of the feature map is enhanced, and the inference speed is improved, a large-size feature output map is additionally added to the Head detection head part to improve the detection precision of the network on small targets, and the CBAM attention mechanism module is added to the four detection heads Head respectively to improve the network learning ability; The specific process of S4 includes: Using the trained target detection network to detect the target person, when the target is detected to have eyes, nose and mouth at the same time, it is judged that the target does not wear a mask; when the target is detected to have only eyes, it is further determined whether the target person correctly wears a mask: taking the width of the minimum rectangle circumscribed by the two eye frames and twice the height, taking another rectangle below the minimum circumscribed rectangle, recording the area as the nose decision area, calculating the area as S1, extracting the skin color in the nose decision area and calculating the skin color area as S2, if S2> alpha S1, it is considered that the target does not correctly wear a mask, otherwise, it is considered that the target person correctly wears a mask. 2.The method of claim 1, wherein, The specific process of S3 includes: After the sample images in the training set are processed by the Mosaic data enhancement method, they are input into the CBAM-YOLOv3-SPPF algorithm, the weight is updated continuously until the loss function tends to be stable and the minimum, the weight file obtained by training is retained and loaded into the CBAM-YOLOv3-SPPF algorithm to make a target detection network. 3.The method of claim 2, wherein, The loss function used in the S3 is a CIOU_Loss function, which is composed of three parts: target confidence loss L conf (o,c), target positioning offset loss L CIoU and target classification loss L cla (O,C) L (O, o, C, c, t, g) = λ1L conf (o, c) + λ2L cla (O, C) + λ3L CIoU (1) Wherein, λ1, λ2, λ3 are balance coefficients, L conf (o,c) is a target confidence loss, L CIoU (o,c) is a target classification loss, L cla (o,c) is a target positioning offset loss, and different size weights are added to the target confidence loss of the four detection heads Head of the improved CBAM-YOLOv3-SPPF network structure In the above formula (2), α1, α2, α3, α4 are weight coefficients and satisfy the condition of α1>α2>α3>α4, and The target confidence loss calculated by the feature output map in descending order of size is sequentially output. 4.The method of claim 3, wherein, In S3, the target confidence loss: Target confidence loss L conf (o, c) is a binary cross-entropy loss, where o i ∈ {0, 1} represents whether there is a target in the predicted target bounding box i, 0 represents that there is no target, 1 represents that there is a target, represents the Sigmoid probability of whether there is a target in the predicted target rectangular frame i. 5.The method of claim 4, wherein, In S3, the target category loss: Target classification loss L cla (O,C) also adopts binary cross-entropy loss, where O ij ∈{0,1} represents whether the j-th target exists in the predicted target bounding box i, 0 represents non-existence, and 1 represents existence, represents the Sigmoid probability of the network predicting that the j-th target exists in the target bounding box i. 6.The method of claim 5, wherein the method further comprises: determining whether the face of the user is covered by the mask based on the face information. In S3, the target positioning offset loss: Target positioning offset loss L CIoU The CIOU Loss function is used for calculation, wherein b pred , b gt are the center positions of the predicted frame and the real frame, p 2 (b pred , b gt ) is the Euclidean distance of the center positions of the two frames, c is the diagonal distance of the outer rectangle of the two frames, a is a weight coefficient, w gt , and hgt are the width and height of the real frame.

Images