Mask wearing detection and health code color checking system and method

Abstract

The application discloses a mask wearing detection and health code color checking system and method, comprising a camera, a data processing unit and a result notification module connected in sequence; the data processing unit comprises a Yolov5s-GGS network and a decision module connected in sequence. The Yolov5s-GGS network performs classification identification, the decision module is used for decoding the classification identification result, and the result notification module displays the final checking result according to the decoding result. The application can realize automatic checking of mask wearing and health code color, and compared with the mask or health code recognition by using a Yolov5s algorithm, the application occupies less computing resources and can be deployed on a low-power embedded device.

Description

Technical Field

[0001] This invention relates to the field of target detection and recognition, specifically to a mask-wearing detection and health code color verification system and method. Background Technology

[0002] Some respiratory diseases can be controlled by wearing masks, and the health code system facilitates the identification of an individual's health status. It has been proven that the combination of masks and health codes can effectively control the spread of some infectious diseases. However, current mask-wearing detection devices are susceptible to limitations in hardware computing power and algorithmic limitations, resulting in high error rates and low frame rates during real-time detection. Therefore, improving detection accuracy and speed is crucial. Object detection algorithms are divided into two categories: two-stage detection algorithms utilize anchor box mechanisms to detect objects within regions of different sizes, but this generates many candidate boxes and is time-consuming. The one-stage detection algorithm Yolov5 directly obtains the target location and classification results through forward inference of the network. This algorithm not only has higher detection accuracy but also shorter inference time than two-stage detection algorithms. However, these methods are difficult to deploy on low-computing-power embedded devices.

[0003] In addition, the existing health code recognition is mainly done manually by relevant managers, which has the problem of low recognition efficiency. In order to improve the convenience for the public, it is necessary to develop a mask and health code recognition solution that can be deployed on low-computing-power embedded devices. Summary of the Invention

[0004] In view of the above-mentioned shortcomings in the prior art, the present invention provides a mask wearing detection and health code color verification system and method that can be deployed on low computing power embedded devices and ensure recognition accuracy.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0006] A mask-wearing detection and health code color verification system is provided, which includes a camera, a data processing unit, and a result notification module connected in sequence; the data processing unit includes a Yolov5s-GGS network and a decision module connected in sequence.

[0007] The decision module is used to decode the data output by the Yolov5s-GGS network, obtain the highest confidence bounding box for each of the five categories, as well as the location coordinates and confidence of the highest confidence bounding box for each category; the five categories are not wearing a mask, wearing a mask, green health code, red health code, and yellow health code;

[0008] The result notification module is used to select the category with the higher confidence level between "not wearing a mask" and "wearing a mask" from the five categories of the same object output by the judgment module as the verification result of whether or not a mask is worn; and to select the category with the highest confidence level among the three categories of green health code, red health code, and yellow health code as the verification result of health code color.

[0009] The Yolov5s-GGS network includes a Focus module, a first convolutional layer, a first GhostBottleneck module, a second GhostBottleneck module, and a first SELayer module connected in sequence; the input of the Focus module is the input of the Yolov5s-GGS network.

[0010] The output of the first SELayer module is connected to one input of the third GhostBottleneck module and the second stitching module, respectively; the third GhostBottleneck module is connected to the fourth GhostBottleneck module and the second SELayer module in sequence; the output of the second SELayer module is connected to one input of the first stitching module and the second convolutional layer, respectively; the output of the second convolutional layer is connected to the RFB module.

[0011] The output of the RFB module is connected in sequence to the fifth GhostBottleneck module and the first depthwise separable convolution module; the output of the first depthwise separable convolution module is connected to one input of the fourth stitching module and the first upsampling module; the output of the first upsampling module is connected to the other input of the first stitching module; the output of the first stitching module is connected in sequence to the sixth GhostBottleneck module and the second depthwise separable convolution module.

[0012] The output of the second depthwise separable convolutional module is connected to one input of the second upsampling module and the third stitching module; the output of the second upsampling module is connected to the other input of the second stitching module; the output of the second stitching module is connected to the seventh GhostBottleneck module; the output of the seventh GhostBottleneck module is connected to the third convolutional layer and the third depthwise separable convolutional module; the output of the third depthwise separable convolutional module is connected to the other input of the third stitching module; the output of the third stitching module is connected to the eighth GhostBottleneck module; the output of the eighth GhostBottleneck module is connected to the fourth convolutional layer and the fourth depthwise separable convolutional module; the output of the fourth depthwise separable convolutional module is connected to the other input of the fourth stitching module; the output of the fourth stitching module is connected to the ninth GhostBottleneck module; the output of the ninth GhostBottleneck module is connected to the fifth convolutional layer.

[0013] The third, fourth, and fifth convolutional layers each output a feature map as the output of the Yolov5s-GGS network; each feature map includes several predicted boxes, the location coordinates of each predicted box, the category, and the confidence level.

[0014] Furthermore, the GhostBottleneck module includes a first input unit, the output of which is connected to one input of the first Ghost submodule and the first summing submodule, respectively; the output of the first Ghost submodule is connected in sequence to the BN layer and the stride recognition unit; when the stride recognition unit recognizes a stride of 1, it directly inputs the received data into the second Ghost submodule; when the stride recognition unit recognizes a stride of 2, it inputs the received data into the depthwise separable convolution submodule; the output of the depthwise separable convolution submodule is connected to the second Ghost submodule; the output of the second Ghost submodule is connected to the other input of the first summing submodule; the output of the first summing submodule is the output of the GhostBottleneck module.

[0015] Furthermore, the SELayer module includes a squeeze submodule, a first fully connected submodule, a first excitation submodule, a second fully connected submodule, and a second excitation submodule connected in sequence; wherein, the squeeze operation expression of the squeeze submodule is:

[0016]

[0017] Where F sq (·) indicates an extrusion operation; Z C This represents the output of the extrusion operation, with a size of 1×1 and a dimension of C; u C (i,j) represents a feature map with dimension C and channel size H×W;

[0018] The first fully connected layer is used to reduce the output dimension of the extrusion submodule to... Where r is a constant;

[0019] The first activation submodule is ReLU activated;

[0020] The second fully connected submodule is used to restore the dimension of the vector to C;

[0021] The second excitation submodule is sigmoid activated.

[0022] Furthermore, the RBF module includes a second input unit, the input of which is the input of the RBF module; the output of the second input unit is connected to a first 1×1 convolutional layer, a second 1×1 convolutional layer, and a shorting submodule; the output of the first 1×1 convolutional layer is connected to a first 3×3 convolutional layer; the second 1×1 convolutional layer is connected to a second 3×3 convolutional layer; the output of the second 3×3 convolutional layer is connected to a third 3×3 convolutional layer and a fourth 3×3 convolutional layer; the output of the third 3×3 convolutional layer is connected to a fifth 3×3 convolutional layer; the outputs of the first 3×3 convolutional layer, the fourth 3×3 convolutional layer, and the fifth 3×3 convolutional layer are connected to the input of a first splicing submodule; the output of the first splicing submodule is connected to the third 1×1 convolutional layer; the output of the third 1×1 convolutional layer is connected to one input of a second addition submodule; the output of the shorting submodule is connected to the other output of the second addition submodule; the output of the second addition submodule is the output of the RBF module.

[0023] Furthermore, the porosity of the first 3×3 convolutional layer is 1; the porosity of the fourth 3×3 convolutional layer is 3; and the porosity of the fifth 3×3 convolutional layer is 5.

[0024] Furthermore, the results notification module includes a display and / or a speaker.

[0025] A method for detecting mask wearing and verifying health code color is provided, which includes the following steps:

[0026] S1. Obtain the training dataset to train the Yolov5s-GGS network and obtain the trained Yolov5s-GGS network; the training dataset includes images of five categories: no mask, mask, green health code, red health code, and yellow health code.

[0027] S2. The trained Yolov5s-GGS network is used to classify and recognize the images acquired by the camera, and three feature maps are obtained.

[0028] S3. Obtain the prediction boxes with confidence scores greater than the threshold value from the three feature maps to obtain the initial screening prediction boxes;

[0029] S4. Obtain the confidence scores of all categories of the initial screening prediction boxes. Use the non-maximum suppression method to obtain the prediction boxes with the highest confidence scores for each of the five categories, as well as the location coordinates and confidence scores of the prediction boxes with the highest confidence scores for each category.

[0030] S5. For the same object with five categories, select the category with the higher confidence level between "not wearing a mask" and "wearing a mask" as the verification result for whether or not a mask is worn; select the category with the highest confidence level among the three categories of green health code, red health code, and yellow health code as the verification result for the health code color.

[0031] Furthermore, when an image captured by the camera is obtained, it is first determined whether a face exists. If so, mask wearing detection and health code color detection are performed. If the health code color is not green, red, or yellow, the next frame image is loaded.

[0032] The beneficial effects of this invention are as follows: This invention can realize automatic verification of mask wearing and health code color. Compared with using the Yolov5s algorithm for mask or health code recognition, this invention consumes less computing resources and can be deployed on low-computing-power embedded devices. Attached Figure Description

[0033] Figure 1 This is a structural block diagram of the system;

[0034] Figure 2 Draw the structure of the Yolov5s-GGS network;

[0035] Figure 3 This is a structural diagram of the GhostBottleneck module;

[0036] Figure 4 This is a schematic diagram of the RBF module.

[0037] Figure 5 This is a map showing the locations of the ground truth bounding boxes and different predicted bounding boxes. Detailed Implementation

[0038] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0039] like Figure 1 As shown, the mask-wearing detection and health code color verification system includes a camera, a data processing unit, and a result notification module connected in sequence; the data processing unit includes a Yolov5s-GGS network and a decision module connected in sequence.

[0040] The decision module is used to decode the data output by the Yolov5s-GGS network, obtain the highest confidence bounding box for each of the five categories, as well as the location coordinates and confidence of the highest confidence bounding box for each category; the five categories are not wearing a mask, wearing a mask, green health code, red health code, and yellow health code;

[0041] The result notification module is used to select the category with the higher confidence level between "not wearing a mask" and "wearing a mask" from the five categories of the same object output by the judgment module as the verification result of whether or not a mask is worn; and to select the category with the highest confidence level among the three categories of green health code, red health code, and yellow health code as the verification result of health code color.

[0042] like Figure 2 As shown, the Yolov5s-GGS network includes a Focus module, a first convolutional layer, a first GhostBottleneck module, a second GhostBottleneck module, and a first SELayer module connected in sequence; the input of the Focus module is the input of the Yolov5s-GGS network.

[0043] The output of the first SELayer module is connected to one input of the third GhostBottleneck module and the second stitching module, respectively; the third GhostBottleneck module is connected to the fourth GhostBottleneck module and the second SELayer module in sequence; the output of the second SELayer module is connected to one input of the first stitching module and the second convolutional layer, respectively; the output of the second convolutional layer is connected to the RFB module.

[0044] The output of the RFB module is connected in sequence to the fifth GhostBottleneck module and the first depthwise separable convolution module; the output of the first depthwise separable convolution module is connected to one input of the fourth stitching module and the first upsampling module; the output of the first upsampling module is connected to the other input of the first stitching module; the output of the first stitching module is connected in sequence to the sixth GhostBottleneck module and the second depthwise separable convolution module.

[0045] The output of the second depthwise separable convolutional module is connected to one input of the second upsampling module and the third stitching module; the output of the second upsampling module is connected to the other input of the second stitching module; the output of the second stitching module is connected to the seventh GhostBottleneck module; the output of the seventh GhostBottleneck module is connected to the third convolutional layer and the third depthwise separable convolutional module; the output of the third depthwise separable convolutional module is connected to the other input of the third stitching module; the output of the third stitching module is connected to the eighth GhostBottleneck module; the output of the eighth GhostBottleneck module is connected to the fourth convolutional layer and the fourth depthwise separable convolutional module; the output of the fourth depthwise separable convolutional module is connected to the other input of the fourth stitching module; the output of the fourth stitching module is connected to the ninth GhostBottleneck module; the output of the ninth GhostBottleneck module is connected to the fifth convolutional layer.

[0046] The third, fourth, and fifth convolutional layers each output a feature map as the output of the Yolov5s-GGS network; each feature map includes several predicted boxes, the location coordinates of each predicted box, the category, and the confidence level.

[0047] The Focus module is used to perform slicing operations, so that an input 640×640×3 image is transformed into a 320×320×12 feature map.

[0048] like Figure 3 As shown, the GhostBottleneck module includes a first input unit, the output of which is connected to one input of the first Ghost submodule and the first summing submodule, respectively; the output of the first Ghost submodule is connected to the BN layer and the stride recognition unit in sequence; when the stride recognition unit recognizes a stride of 1, it directly inputs the received data into the second Ghost submodule; when the stride recognition unit recognizes a stride of 2, it inputs the received data into the depthwise separable convolution submodule; the output of the depthwise separable convolution submodule is connected to the second Ghost submodule; the output of the second Ghost submodule is connected to the other input of the first summing submodule; the output of the first summing submodule is the output of the GhostBottleneck module.

[0049] The GhostBottleneck module uses fewer parameters and less computational complexity to obtain redundant information, which can reduce the number of network parameters and shorten the detection time.

[0050] The SELayer module is an attention mechanism module, comprising a squeeze submodule, a first fully connected submodule, a first activation submodule, a second fully connected submodule, and a second activation submodule connected in sequence; wherein, the squeeze operation expression of the squeeze submodule is:

[0051]

[0052] Where F sq (·) indicates an extrusion operation; Z C This represents the output of the extrusion operation, with a size of 1×1 and a dimension of C; u C (i,j) represents a feature map with dimension C and channel size H×W;

[0053] The first fully connected layer is used to reduce the output dimension of the extrusion submodule to... Where r is a constant;

[0054] The first activation submodule is ReLU activated;

[0055] The second fully connected submodule is used to restore the dimension of the vector to C;

[0056] The second excitation submodule is sigmoid activated.

[0057] Since masks and health codes are small targets in real-world scenarios, the SELayer module used in this invention can automatically learn the importance of different channel features by focusing on the relationships between channels, thereby improving the detection accuracy of small targets in the model.

[0058] like Figure 4 As shown, the RBF module is a module that expands the receptive field. It includes a second input unit, the input of which is the input of the RBF module. The output of the second input unit is connected to a first 1×1 convolutional layer, a second 1×1 convolutional layer, and a shorting submodule. The output of the first 1×1 convolutional layer is connected to a first 3×3 convolutional layer. The second 1×1 convolutional layer is connected to a second 3×3 convolutional layer. The output of the second 3×3 convolutional layer is connected to a third 3×3 convolutional layer and a fourth 3×3 convolutional layer. The output of the third 3×3 convolutional layer is connected to a fifth 3×3 convolutional layer. The outputs of the first, fourth, and fifth 3×3 convolutional layers are connected to the input of a first splicing submodule. The output of the first splicing submodule is connected to the third 1×1 convolutional layer. The output of the third 1×1 convolutional layer is connected to one input of a second addition submodule. The output of the shorting submodule is connected to the other output of the second addition submodule. The output of the second addition submodule is the output of the RBF module.

[0059] In RFB, a 1×1 convolution (Conv) is first performed to adjust the number of channels. Then, different receptive field branches A, B, and C are generated through 1×1, 3×3, and 5×5 convolutions, respectively. Next, dilated convolutions with dilation rates of 1, 3, and 5 are used to increase the receptive field. The improved RFB uses two 3×3 convolutions instead of 5×5 convolutions, and branches B and C share a single 3×3 convolution. This not only reduces network parameters but also reduces information loss caused by multiple convolutions. Finally, the three branches A, B, and C are connected and then subjected to a 1×1 convolution to obtain feature map D. This feature map D is then added to branch E after the shortcut operation to obtain feature map F.

[0060] Existing YOLOv5s networks use SPP (Special Perspective Pooling) after the backbone feature network output to increase the network's receptive field. Compared to SPP, which employs max pooling, the RBF (Receptive Field Facility) module used in this invention employs horizontally connected fusion network layers. By expanding convolutions, it increases the receptive field for small targets while reducing computational cost, resulting in stronger robustness and a larger receptive field.

[0061] The method for detecting mask wearing and verifying health code color includes the following steps:

[0062] S1. Obtain the training dataset to train the Yolov5s-GGS network and obtain the trained Yolov5s-GGS network; the training dataset includes images of five categories: no mask, mask, green health code, red health code, and yellow health code.

[0063] S2. The trained Yolov5s-GGS network is used to classify and recognize the images acquired by the camera, and three feature maps are obtained.

[0064] S3. Obtain the prediction boxes with confidence scores greater than the threshold value from the three feature maps to obtain the initial screening prediction boxes;

[0065] S4. Obtain the confidence scores of all categories of the initial screening prediction boxes. Use the non-maximum suppression method to obtain the prediction boxes with the highest confidence scores for each of the five categories, as well as the location coordinates and confidence scores of the prediction boxes with the highest confidence scores for each category.

[0066] S5. For the same object with five categories, select the category with the higher confidence level between "not wearing a mask" and "wearing a mask" as the verification result for whether or not a mask is worn; select the category with the highest confidence level among the three categories of green health code, red health code, and yellow health code as the verification result for the health code color.

[0067] In the specific implementation process, when an image captured by the camera is obtained, it is first determined whether a face exists. If so, mask wearing detection and health code color detection are performed. If the health code color is not green, red, or yellow, the next frame image is loaded. The operation expression from the first fully connected layer to the second activation submodule is as follows:

[0068] F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z))

[0069] Where F ex (·) represents the output of the second activation submodule; σ(·) represents sigmoid activation; δ(·) represents ReLU activation; g(z,W) represents the vector after dimensionality increase; W1 represents the weights of the first fully connected layer; W2 represents the weights of the second fully connected layer; W is the weight matrix of the two fully connected layers; z is the input of the first fully connected layer.

[0070] Both Ghost submodules are 1×1 in size. The depthwise separable convolutional submodule is 3×3 in size with a stride of 2. The first 3×3 convolutional layer has a dilatancy of 1; the fourth 3×3 convolutional layer has a dilatancy of 3; and the fifth 3×3 convolutional layer has a dilatancy of 5. The results notification module includes a display and / or a speaker.

[0071] The three feature maps output by the Yolov5s-GGS network have tensor sizes of 80×80×30, 40×40×30, and 20×20×30, respectively. 80×80 represents the number of grid points. Each grid point has three prior boxes, each with its width, height, center x-coordinate, center y-coordinate, confidence score, and the number of classes it contains (num_classes). Since there are a total of 5 classes, num_classes = 5, and therefore 30 = 3×(4+1+5), where 3 represents the three prior boxes, 4 represents the width, height, center x-coordinate, and center y-coordinate, 1 represents one confidence score, and 5 represents the five classes. Obtaining three scales aims to obtain more comprehensive prediction boxes, thereby improving the performance of the Yolov5s-GGS network. Regardless of the number of scales, prediction boxes (possibly one or more) that meet the confidence threshold are selected based on their confidence scores, and the prediction boxes with the highest confidence scores for each of the five classes are selected using non-maximum suppression.

[0072] In one embodiment of the present invention, to address the problem that the bounding box loss function GIOU fails to select the optimal small target prediction box due to the existence of identical overlap between different predicted boxes and ground truth boxes, the bounding box loss function is improved to obtain DGIOU. Furthermore, depthwise separable convolution is used instead of ordinary convolution to reduce the number of model parameters and improve inference speed. IOU and GIOU cannot fully reflect the true situation of predicted bounding boxes. Figure 5 In the image, box 1 is the ground truth box, and the rest are predicted boxes. The center coordinates of each box are denoted as Center. iThe values ​​of IOU, GIOU, and GIOU loss between each bounding box and the ground truth bounding box were calculated and are shown in Table 1.

[0073] Table 1: Comparison of Predicted Box Similarities

[0074]

[0075] The first scenario involves intersection with the ground truth bounding box. Comparing bounding boxes 2 and 3, when both predicted bounding boxes 2 and 3 have the same amount of overlapping area with the ground truth bounding box 1, the IOU and GIOU between bounding boxes 2 and 3 and the ground truth bounding box are also completely identical. However, in reality, the center coordinates of bounding box 2 (Center2) are closer to the center coordinates of the ground truth bounding box 1 (Center1) than the center coordinates of bounding box 3 (Center3). Their corresponding center distance ratio is...

[0076] The second scenario is that it does not intersect with the actual bounding box. Compare box 4 and box 5, through... Figure 5 Observation shows that box 4 is obviously the better choice, but this situation cannot be reflected by calculating IOU and GIOU. The IOU values ​​of boxes 4 and 5 are equal, but the GIOU of box 4 is lower than that of box 5. Therefore, the GIOU loss of box 4 is greater than that of box 5, indicating that box 5 is better than box 4, which does not match the actual situation.

[0077] The above experiments demonstrate that calculating the IOU and GIOU between the predicted and ground truth boxes when the overlap is the same cannot select the optimal predicted box. However, predicted boxes with the same overlap have different center coordinate distances with the ground truth boxes. Therefore, an improvement can be made to GIOU. The improved bounding box loss function is named (DGIOU, Distance Generalized Intersection Over Union), which considers the center distance. Therefore, a distance coefficient k can be assigned to DGIOU. The specific algorithm flow is as follows:

[0078] Input: Two rectangular prediction boxes, A and B.

[0079] Output: The value of DGIOU

[0080] DGIOU value: The magnitude of the DGIOU loss value affects the performance of the entire network. During the training phase, as the number of training rounds increases, DGIOU gradually decreases and eventually converges.

[0081] Step 1: Locate the center coordinates of prediction boxes A and B. A and Center B ;

[0082] Step 2: Calculate the distance between the center coordinates of prediction box A and prediction box B:

[0083] Step 3: Calculate the value of DGIOU:

[0084] Step 4: Optimize the value of parameter k, where the value of k ranges from [0, 0.25].

[0085] Step 5: Repeat step 1;

[0086] Where C * The smallest bounding rectangle representing prediction boxes A and B.

[0087] Optimization process: During the training phase, as the training epochs increase, the value of parameter k is optimized by comparing the DGIOU value of the current epoch with that of the previous epoch. This process continues until the DGIOU value stabilizes, at which point the optimal parameter k value can be determined. The changes in the DGIOU value can be used to judge the quality and extent of our Yolov5s-GGS network training.

[0088] In practice, if no face is detected in the current frame, the next video frame is loaded. If a face is detected, a determination is made regarding whether a mask is being worn.

[0089] If a person is not wearing a mask, the display will show and announce, "Warning, please wear a mask." When a person is wearing a mask and shows a green health code, the display will show and announce, "Green code, please communicate." When a person is wearing a mask and shows a yellow health code, the display will show and announce, "Yellow code holders, please wait." When a person is wearing a mask and shows a red health code, the display will show and announce, "Warning, red code holders." The next frame will load when a person's health code is not green, yellow, or red.

[0090] In one embodiment of the present invention, the data processing unit may employ a Jetson nano processing unit, the hardware of which is a Jetson nano B01 developer kit. This developer kit provides the following interfaces: Interface 1: micro SD card slot; Interface 2: 40-pin expansion connector; Interface 3: Micro-USB interface for 5V power input or device mode; Interface 4: Gigabit Ethernet port; Interface 5: USB 3.0 ports (4 ports); Interface 6: HDMI output port; Interface 7: DisplayPort interface; Interface 8: DC power connector for 5V power input; Interface 9: MIPI CSI-2 camera interface, etc. The main hardware component is the Jetson nano B01, and the remaining components include a 13.3-inch display, USB sound card driver and speaker, wireless WIFI module, USB camera, etc. The 13.3-inch display is connected to the main hardware component via Interface 6, and the USB sound card driver and speaker, wireless WIFI module, and USB camera are all connected to the main hardware component via the USB port of Interface 5. An HD91-1080P external camera is used. A 13.3-inch IPS touchscreen is used as the external display for the entire system and is connected to the Jetsonnano B01 module via an HDMI cable.

[0091] TensorRT is a high-performance deep learning inference optimizer that, combined with NVIDIA GPUs, enables efficient inference deployment across virtually all deep learning frameworks. TensorRT operates in two phases: build and deploy. The build phase primarily optimizes the network configuration and performs model conversion (from PyTorch to TensorRT). The output of this phase is an optimized TensorRT model, called a plan file. The plan file is optimized target code that can be serialized and stored in both memory and disk. The deployment phase first deserializes the plan file generated in the previous phase, creates a runtime engine, and then receives batches of input data, performs inference, and finally returns the output data through a long-running service. This process primarily utilizes a combination of layers and tensors.

[0092] For the specific training, we prepared the WiderFace face detection dataset, the MAFA mask detection dataset, and a manually created health code color dataset. After filtering and removing some unreasonable images, we ultimately retained 9120 images. Of these, 3006 images came from the MAFA dataset, mostly showing faces obscured by masks. The remaining 3114 images came from the WiderFace dataset, mostly showing unobstructed faces. The self-made dataset contained 1000 green, 1000 yellow, and 1000 red health codes. These were then divided into training, validation, and test sets in a 7:2:1 ratio.

[0093] Training parameter settings: learning rate set to 0.001, training epochs to 400, batch size to 16, momentum to 0.9, and weight decay to 0.005. The Yolov5s-GGS network was then trained on a PC with an RTX3090, yielding the training weights file Yolov5s-GGS.pt (pt is the weight file). This weights were then converted into an engine file using the TensorRT algorithm, which can then be deployed on an embedded device, the Jetson Nano.

[0094] Compared to the YOLOv5 algorithm model, the size of this YOLOv5s-GGS network has been reduced from 14.5MB to 11.8MB, a decrease of 18.6%, while the detection rate has been improved by 14.4%. After accelerating the weight pt file of the improved YOLOv5s model using the TensorRT strategy, the resulting engine file, after being burned into a Jetson nano, has a weight file size of 1.52MB. This significant reduction in model parameter size facilitates deployment on embedded devices, increasing the frame rate of the original algorithm from 3.58 frames / s to 13.6 frames / s. Even on the relatively low-performance embedded device, the Jetson nano, real-time requirements can be met.

[0095] In this embodiment, the performance of the present invention was compared with that of various mainstream algorithms, and the comparison results are shown in Table 2.

[0096] Table 2: Performance Comparison of Various Mainstream Algorithms

[0097]

[0098]

[0099] As shown in Table 2, compared to the Yolov5s algorithm, the detection rate of Yolov5s-GGS increased from 74.2 frames / s to 84.9 frames / s, and the mAP improved by 4.60%, reaching 96.80%. Compared to Faster R-CNN, SSD, Yolov3, Yolov4, and Yolov5x algorithms, the detection rate improved by 55.6, 32.2, 22.5, 26.4, and 24.3 frames / s, respectively, and the mAP improved by 20.18, 22.55, 14.18, 6.37, and 1.67 percentage points, respectively. Experimental data shows that the detection results on the publicly available datasets MFMA and WilderFace, as well as the self-made health code color dataset, prove that the Yolov5s-GGS algorithm outperforms several other mainstream algorithms and is more suitable for mask-wearing detection in real-world scenarios.

[0100] In summary, this invention can automatically verify mask wearing and health code color. Compared with using the Yolov5s algorithm for mask or health code recognition, this invention consumes less computing resources and can be deployed on low-computing-power embedded devices.

Claims

1. A mask wearing detection and health code color verification system, characterized in that, The device comprises a camera, a data processing unit and a result notification module connected in sequence; the data processing unit comprises a Yolov5s-GGS network and a decision module connected in sequence; The decision module decodes the data output by the Yolov5s-GGS network, obtains the prediction box with the highest confidence in each of the five categories, and the position coordinate information and confidence of the prediction box with the highest confidence corresponding to each category; the five categories are not wearing a mask, wearing a mask, green health code, red health code and yellow health code; The result notification module selects the category with higher confidence in the two categories of not wearing a mask and wearing a mask as the verification result of whether a mask is worn according to the five categories of the same object output by the decision module; and selects the category with the highest confidence in the three categories of green health code, red health code and yellow health code as the verification result of the color of the health code; The Yolov5s-GGS network comprises a Focus module, a first convolutional layer, a first GhostBottleneck module, a second GhostBottleneck module and a first SELayer module connected in sequence; the input end of the Focus module is the input end of the Yolov5s-GGS network; The output end of the first SELayer module is connected with a third GhostBottleneck module and one input end of a second splicing module; the third GhostBottleneck module is connected with a fourth GhostBottleneck module and a second SELayer module in sequence; the output end of the second SELayer module is connected with one input end of a first splicing module and a second convolutional layer; the output end of the second convolutional layer is connected with an RFB module; The output end of the RFB module is connected with a fifth GhostBottleneck module and a first depth separable convolution module in sequence; the output end of the first depth separable convolution module is connected with one input end of a fourth splicing module and a first upsampling module; the output end of the first upsampling module is connected with the other input end of the first splicing module; the output end of the first splicing module is connected with a sixth GhostBottleneck module and a second depth separable convolution module in sequence; An output end of the second deep separable convolution module is connected with one input end of a second upsampling module and a third concatenation module respectively; an output end of the second upsampling module is connected with another input end of the second concatenation module; an output end of the second concatenation module is connected with a seventh GhostBottleneck module; an output end of the seventh GhostBottleneck module is connected with a third convolution layer and a third deep separable convolution module respectively; an output end of the third deep separable convolution module is connected with another input end of the third concatenation module; an output end of the third concatenation module is connected with an eighth GhostBottleneck module; an output end of the eighth GhostBottleneck module is connected with a fourth convolution layer and a fourth deep separable convolution module respectively; an output end of the fourth deep separable convolution module is connected with another input end of a fourth concatenation module; an output end of the fourth concatenation module is connected with a ninth GhostBottleneck module; an output end of the ninth GhostBottleneck module is connected with a fifth convolution layer; The third convolution layer, the fourth convolution layer and the fifth convolution layer respectively output a feature map as the output of the Yolov5s-GGS network; each feature map includes a plurality of prediction boxes, position coordinate information of each prediction box, a category and a confidence.

2. The mask wearing detection and health code color checking system according to claim 1, characterized in that, The GhostBottleneck module includes a first input unit, and output ends of the first input unit are connected with one input end of a first Ghost sub-module and a first adding sub-module respectively; an output end of the first Ghost sub-module is connected with a BN layer and a step recognition unit in sequence; When the step recognition unit recognizes that the step is 1, the data received by the step recognition unit is directly input into a second Ghost sub-module; When the step recognition unit recognizes that the step is 2, the data received by the step recognition unit is input into a deep separable convolution sub-module; an output end of the deep separable convolution sub-module is connected with the second Ghost sub-module; an output end of the second Ghost sub-module is connected with another input end of the first adding sub-module; and an output end of the first adding sub-module is an output end of the GhostBottleneck module.

3. The mask wearing detection and health code color verification system according to claim 1, wherein, The SELayer module includes an extrusion sub-module, a first full connection sub-module, a first excitation sub-module, a second full connection sub-module and a second excitation sub-module connected in sequence; wherein an extrusion operation expression of the extrusion sub-module is: where F sq (·) denotes the squeezing operation; Z C denotes the output of the squeezing operation, with size 1 x 1 and dimension C; u C (i,j) denotes a feature map with dimension C and channel size H x W; a first fully connected layer for reducing the output dimension of the extrusion sub-module to where r is a constant; The first excitation sub-module is relu activation; The second full connection sub-module is used for restoring the dimension of a vector to C; The second excitation sub-module is sigmoid activation.

4. The mask wearing detection and health code color checking system according to claim 1, characterized in that, The RBF module comprises a second input unit, an input end of the second input unit being an input end of the RBF module; an output end of the second input unit being connected with a first 1*1 convolution layer, a second 1*1 convolution layer and a short connection sub-module respectively; an output end of the first 1*1 convolution layer being connected with a first 3*3 convolution layer; the second 1*1 convolution layer being connected with a second 3*3 convolution layer; an output end of the second 3*3 convolution layer being connected with a third 3*3 convolution layer and a fourth 3*3 convolution layer respectively; an output end of the third 3*3 convolution layer being connected with a fifth 3*3 convolution layer; an output end of the first 3*3 convolution layer, an output end of the fourth 3*3 convolution layer and an output end of the fifth 3*3 convolution layer being connected with an input end of a first splicing sub-module respectively; an output end of the first splicing sub-module being connected with a third 1*1 convolution layer; an output end of the third 1*1 convolution layer being connected with one input end of a second adding sub-module; an output end of the short connection sub-module being connected with the other output end of the second adding sub-module; An output end of the second adding sub-module being an output end of the RBF module.

5. The mask wearing detection and health code color verification system according to claim 4, characterized in that, The first 3*3 convolution layer has a hole rate of 1; the fourth 3*3 convolution layer has a hole rate of 3; and the fifth 3*3 convolution layer has a hole rate of 5.

6. The mask wearing detection and health code color verification system according to claim 1, wherein, The result notification module comprises a display and / or a loudspeaker.

7. A verification method based on the mask wearing detection and health code color verification system according to any one of claims 1-6, characterized in that, The method comprises the following steps: S1, obtaining a training data set to train a Yolov5s-GGS network, and obtaining a trained Yolov5s-GGS network; The training data set comprises images of five categories of no mask, mask, green health code, red health code and yellow health code; S2, using the trained Yolov5s-GGS network to classify and identify images obtained by a camera, and obtaining three feature maps; S3, obtaining a prediction box with a confidence greater than a threshold value in the three feature maps, and obtaining a preliminary screening prediction box; S4, obtaining the confidence corresponding to all categories of the preliminary screening prediction box, and obtaining the prediction box with the highest confidence of each category and the position coordinate information and confidence of the prediction box with the highest confidence of each category by a non-maximum suppression method; S5, for the five categories of the same object, selecting the category with a greater confidence in the two categories of no mask and mask as the verification result of whether a mask is worn; and selecting the category with the greatest confidence in the three categories of green health code, red health code and yellow health code as the verification result of the color of the health code.

8. The verification method of the mask wearing detection and health code color verification system according to claim 7, characterized in that, When an image captured by the camera is obtained, it is first determined whether a face exists, and if so, mask wearing detection and health code color detection are performed; when the color of the health code is not green, red or yellow, the next frame of image is loaded.

Images