Mask detection system and method in elevator scenario based on edge computing

Abstract

The application discloses a mask detection system and method based on edge computing in an elevator scene, wherein a main network of a traditional Yolov5 network is improved, a 3*3 Conv module in the original main network is modified into a RepVGG module, different convolution kernels are used in the multi-branch structure of the RepVGG module to obtain different receptive fields, information obtained by the different receptive fields is added, and then the extraction of feature information is strengthened, and the model performance is improved, so that even if a large number of people enter the elevator at the same time and a large number of shielding phenomena occur in the narrow elevator space, feature extraction can still be effectively performed. The application can realize real-time high-precision mask wearing condition detection, and can give a reminder and record by means of a knowledge distillation method, so as to protect the health and safety of people in the epidemic situation, and reduce the management and intelligent equipment cost of a property.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of target detection based on deep learning, and particularly refers to a mask detection system and method in an elevator scene based on edge computing. BACKGROUND

[0002] In recent years, with the rapid development of deep learning, deep learning has achieved satisfactory results in the field of target detection. In the fields of security, autonomous driving, unmanned aerial vehicles, etc., various target detection and recognition applications based on deep learning are also increasing.

[0003] Wearing a mask in crowded places such as airports, stations, or closed places such as buses and elevators can effectively reduce the probability of virus transmission. In these scenarios, using manual supervision is inefficient and wastes public resources. Using a camera combined with deep learning can efficiently detect and record the mask-wearing situation of people.

[0004] However, deep learning requires a large amount of computing power to support, so it has high requirements for the performance of the device processing algorithm. Taking the classic target detection model VGG16 as an example, when processing a 224x224 size image, it requires 138.36 million parameters, as many as 15.5x10 9 floating-point operations.

[0005] The huge demand for computing resources to some extent limits the application of these algorithms on embedded devices, so most deep learning applications on embedded devices currently use cloud computing. The camera captures the picture and transmits it to the cloud server through the network. After the cloud server uses the deep model for recognition, the result is returned to the embedded device. However, this client-server model based on cloud computing requires a large amount of data to be transmitted to the cloud for calculation, which requires high network overhead and may cause high latency problems. In addition, the high-performance cloud server that can process massive data and recognize in real time has a high cost, which is not friendly to some real-time monitoring scenarios that require low cost. For example, real-time detection and recognition in the elevator scene, as elevators are widely distributed and different elevators in different communities and office buildings may be managed by different property management companies, but there is currently no unified cloud computing platform and system for elevator target detection for all property management companies. Therefore, how to reduce the cost of property management and equipment has become a key problem.

[0006] With the development of edge computing, this problem has been further addressed. By offloading computing and storage resources from the cloud to the edge, data transmission volume can be reduced, solving the problem of insufficient network bandwidth, while also reducing the computing pressure on cloud service centers and lowering server costs. However, because deep learning places high demands on processing device performance, deploying existing neural networks on low-cost devices while balancing accuracy and computational speed remains a challenge.

[0007] There are two problems with detecting whether people are wearing masks in an elevator: First, because the position of the surveillance cameras in the elevator is not fixed, and most of the cameras are set behind people, it is difficult to detect whether people with their backs to the camera are wearing masks when detecting masks in an elevator. Second, if people are crowded in the elevator, there is also the problem of mutual obstruction. Summary of the Invention

[0008] To address the aforementioned issues, the present invention aims to provide a mask detection system and method for elevator scenarios based on edge computing. This system can effectively improve the mask detection accuracy for people with dense occlusion and facing away from the camera in elevator scenarios. Furthermore, the lightweight network allows for real-time mask wearing detection on edge devices, which can reduce latency, budget, and network traffic.

[0009] This invention provides a mask detection method for elevator scenarios based on edge computing, comprising the following steps:

[0010] Step 100: Construct an improved Yolov5 network

[0011] The traditional YOLOv5 network mainly consists of three parts: a backbone network for feature extraction, a neck network for feature fusion, and a head network for detection. It adopts an FPN+PANeT structure, in which the feature pyramid network FPN transmits semantic information from the high level to the low level, while the PANET structure transmits localization information from the low level to the high level, thereby fusing semantic and localization information.

[0012] The improved Yolov5 network, comprising a feature extraction backbone, a feature fusion neck network, and a feature detection head network, is an improvement on the traditional Yolov5 network using an FPN+PANeT structure. The original 3×3 convolutional modules in the feature extraction backbone are replaced with repeatable convolutional neural networks (CNNs) with multi-branch structures and different kernels. These CNNs use different kernels to obtain different receptive fields, summing the information from different receptive fields, and replacing the original activation function with a sigmoid-weighted linear combination. The PANet module in the original feature fusion neck network is replaced with a weighted bidirectional feature pyramid network. This network transmits high-level semantic information and low-level feature location information via a bottom-up channel, enhancing target classification and localization capabilities. This bottom-up channel connects the input and output nodes of the same layer of the backbone network, achieving bidirectional cross-layer feature mapping with weighted features through the weighted bidirectional feature pyramid network. The feature fusion process involves a convolutional attention module in each 3* convolutional layer of the neck network for feature fusion and the head network for feature detection. This module, used for the feedforward convolutional neural network, allows the intermediate feature maps to sequentially infer attention maps along the two independent dimensions of channel and space. The attention maps are then multiplied with the input feature maps for adaptive feature optimization to obtain more features, enabling the backbone network for feature extraction to quickly learn channel and spatial features. The head network for feature detection outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. These feature maps predict the target category and bounding box position, with the 80×80 feature map detecting small targets, the 40×40 feature map detecting medium targets, and the 20×20 feature map detecting large targets. Each feature map layer contains three predicted bounding boxes with different aspect ratios, each containing the target's position information and confidence level. Finally, a non-maximum suppression algorithm discards overlapping predicted bounding boxes with an intersection-union ratio exceeding a set threshold.

[0013] Step 200: Train the teacher network using the data in the training set of the mask wearing detection dataset through the improved YOLOv5 network. After knowledge distillation, obtain a lightweight student network. Deploy the lightweight student network on a low-cost edge device to obtain the mask wearing detection module. The mask wearing detection module performs image prediction based on the improved YOLOv5 network and outputs the predicted behavior and related target labels, confidence scores and location information.

[0014] Step 300: The teacher network is trained on the training set of the prior knowledge dataset using the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask removal behavior detection module. The mask removal behavior detection module performs image prediction based on the improved YOLOv5 network and outputs the predicted behavior and the labels, confidence scores and location information of related targets.

[0015] Step 400: The elevator starts running, triggering the start control module to control other modules to work, and the camera starts to acquire images; when the elevator door opens, the mask wearing detection module on the mobile edge computing device acquires images of people entering and exiting the elevator from the camera and detects whether people are wearing masks; when the elevator door closes, the mask removal behavior detection module on the mobile edge computing device acquires images of people in the elevator from the camera and identifies whether mask removal behavior has occurred. If there is no mask being worn, the mask being worn improperly, or mask removal behavior has occurred, an alarm will be issued to the people in the elevator.

[0016] An attention transfer method based on activation function values ​​is used between teacher and student networks, and the degree of attention of neurons to the input is measured by calculating the absolute value statistics of the output function.

[0017] A multi-scale attention module is designed and inserted at the end of five CSP modules in the YOLOv5 networks of both the teacher and student networks. This allows the convolutional networks at different levels of the student network to learn feature extraction capabilities similar to those of the teacher network. This multi-scale attention module uses pooling layers of three different scales (1×1, 3×3, and 5×5) to extract feature map vectors R. W×H×C For features at the corresponding scale, calculate the maximum P-th power of the absolute value of each of the three feature map vectors in dimension C to obtain the input feature map vector R. W×H×C Attention feature maps at different scales; finally, the pixels at corresponding positions in the three attention feature maps are summed to obtain a single-channel multi-scale attention feature map R. W×H After obtaining the multi-scale attention feature maps of the teacher network and the student network, the L2 distance between them is calculated using the minimum mean square error loss function, as shown in the following formula:

[0018]

[0019] Step 200 specifically involves:

[0020] (201) Create a mask-wearing detection dataset:

[0021] Images of people entering and exiting the elevator when the elevator doors open and images of the elevator interior during operation are collected at different time periods; the collected images are filtered and cleaned to obtain a dataset consisting of color images and depth images; data augmentation preprocessing is performed on the collected dataset;

[0022] For images of people entering elevators, annotation tools are used to annotate the faces of people in color images to create a mask-wearing detection dataset, which is divided into three categories: not wearing a mask, wearing a mask, and wearing a mask improperly. The annotated mask-wearing detection dataset is then divided into training and validation sets according to the proportions.

[0023] (202) The teacher network is trained on the training set of the mask wearing detection dataset by the improved YOL0v5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask wearing detection module.

[0024] (203) The mask-wearing detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information:

[0025] The preprocessed image is input into the feature extraction backbone network. Different receptive fields are obtained through different convolutional kernels in a multi-branch repeatable convolutional neural network, and the information obtained from different receptive fields is added together. The feature extraction backbone network outputs primary feature maps of different sizes to the feature fusion neck network. By strengthening the feature extraction through a weighted bidirectional feature pyramid network, high-level semantic information and low-level feature location information are transmitted through a bottom-up channel, thereby enhancing the target classification and target localization capabilities. The bottom-up channel connects the input and output nodes of the same layer of the backbone network. The weighted bidirectional feature pyramid network achieves bidirectional cross-level and weighted feature map fusion. The feature fusion neck network combines the feature maps obtained from the FPN module and the weighted bidirectional feature pyramid network. The feature map is fused with the feature map obtained from the backbone network to fully utilize the features extracted by the backbone network. The fused feature map obtained from the neck network is input into the head network of feature detection. The head network of feature detection outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. The target category and bounding box position are predicted for the feature maps of different sizes. The 80×80 feature map is responsible for detecting small targets in the image, the 40×40 feature map is responsible for detecting medium targets in the image, and the 20×20 feature map is responsible for detecting large targets in the image. On each feature map layer, three prediction boxes with different aspect ratios are preset. Each prediction box contains the target's position information and confidence score. Finally, the non-maximum suppression algorithm discards overlapping prediction boxes with an intersection-union ratio exceeding a set threshold.

[0026] Step 300 specifically involves:

[0027] (301) Create a prior knowledge dataset

[0028] For the images of people entering the elevator and images of the elevator running in the data augmentation dataset, the annotation tool is used to annotate the hands, heads and masks of people in the color images as a prior knowledge dataset, which is divided into three categories: head, hand and mask. The annotated prior knowledge dataset is divided into training set and validation set according to the proportion.

[0029] (304) The teacher network is obtained by training the data of the training set of the prior knowledge dataset through the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask removal behavior detection module.

[0030] (303) The mask removal behavior detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information. The label L represents the mask removal behavior, and T represents the confidence threshold for the occurrence of the mask removal behavior. The specific process is as follows:

[0031] Step 1. When the detection result predicts label L and the confidence level p0 > T, directly determine that the act of removing the mask has occurred;

[0032] Step 2. When the detection result predicts label L and the confidence level p0≤T, it is necessary to re-determine whether the mask removal behavior has occurred by using the distance relationship between targets related to the mask removal behavior, determine the actual increment of the confidence level of the mask removal behavior, and then correct the confidence level obtained by the improved YOLOv5 network to obtain the final confidence level of the mask removal behavior. This includes the following steps:

[0033] The distances between the center points of each predicted bounding box are calculated to obtain the distance values ​​Dist(head, mask), Dist(hand, mask), and Dist(head, hand). The probability of mask removal in the image is correlated with these distance values. Using the minor axis distance Len(face) of the detected head predicted bounding box as a reference distance, a distance-based judgment rule is established to determine the actual confidence increment of mask removal behavior.

[0034] Rule 1: When the distance condition 1 is met: Dist(hand, mask) ≤ a × Len(face), the actual increment of the confidence in the occurrence of the mask removal behavior is p1;

[0035] Rule 2: When distance condition 2 is met: Dist(head, mask) ≥ b × Len(face), the actual increment of confidence in the occurrence of mask removal is p2.

[0036] Rule 3: When the distance condition 3 is met: Dist(head, hand) ≤ c × Len(face), the actual increment of the confidence level for the mask removal behavior is p3.

[0037] The behavioral weights a, b, and c are obtained through statistical analysis of the information labeled in the training samples in the prior knowledge dataset. The parameters p1, p2, and p3 are first assigned manually based on the contribution of prior knowledge to the occurrence of mask removal behavior. Then, p1, p2, and p3 are normalized. That is, when the above rules are satisfied at the same time, p1 + p2 + p3 = 1. After obtaining the normalized parameters p1, p2, and p3, they are verified and adjusted in the validation set of the prior knowledge dataset so that the parameters p1, p2, and p3 meet the expectations for mask removal behavior detection.

[0038] By determining whether the above three distance conditions are met, the actual increase in confidence level for the mask-removal behavior is determined. This is then used to correct the confidence level obtained from the improved YOLOv5 network, resulting in the final confidence level for the mask-removal behavior.

[0039] If the final confidence level p of the mask removal behavior is higher than the threshold T, then the mask removal behavior is determined to have occurred; otherwise, the mask removal behavior is determined not to have occurred.

[0040] Step 3. When the detection result has no label L and p0 = 0, it is necessary to re-determine whether the mask removal behavior has occurred by using the distance relationship between the targets related to the mask removal behavior, determine the actual increment of the confidence that the mask removal behavior has occurred, and then correct the confidence obtained by the improved YOLOv5 network to obtain the final confidence p that the mask removal behavior has occurred.

[0041] If the final confidence level p of the mask removal action is higher than the threshold T, then the mask removal action is determined to have occurred; otherwise, the mask removal action is determined not to have occurred.

[0042] The aforementioned attention transfer method based on activation function values ​​between the teacher and student networks measures the degree of attention a neuron pays to the input by calculating the absolute value statistic of the output function. Specifically:

[0043] The spatial attention feature map is generated by calculating the P-squared maximum of the absolute value of the feature map vector Vi in dimension C. The formula is as follows:

[0044] This invention discloses a mask detection system for elevator scenarios based on edge computing, comprising a start control module, an image acquisition module, a mask wearing detection module, and a mask removal behavior detection module, wherein...

[0045] The start-up control module is triggered in response to the opening of the elevator door, controlling all modules to leave standby and enter the working state, and controlling the image acquisition module to collect images of people entering and exiting the elevator and images of the elevator interior during operation from the elevator monitoring camera.

[0046] The image acquisition module enhances the images of people entering and exiting the elevator and the images inside the elevator during operation captured by the elevator monitoring camera. Then, it sends the images of people entering and exiting the elevator to the mask wearing detection module and the images inside the elevator during operation to the mask removal behavior detection module.

[0047] The mask-wearing detection module detects whether people are wearing masks based on images of people entering and exiting the elevator transmitted by the camera when the elevator door opens, using an improved YOLOv5 network. It outputs the predicted behavior and related target labels, confidence scores, and location information to obtain detection results for not wearing masks or wearing masks improperly.

[0048] The mask removal behavior detection module detects whether people have removed their masks based on the images of the elevator inside the elevator transmitted by the camera when the elevator door is closed, using an improved YOL0v5 network. It outputs the predicted behavior and the labels, confidence scores, and location information of related targets to obtain the detection result of the mask removal behavior.

[0049] The improved Yolov5 network, comprising a feature extraction backbone, a feature fusion neck network, and a feature detection head network, is an improvement on the traditional Yolov5 network using an FPN+PANeT structure. The original 3×3 convolutional modules in the feature extraction backbone are replaced with repeatable convolutional neural networks (CNNs) with multi-branch structures and different kernels. These CNNs use different kernels to obtain different receptive fields, summing the information from different receptive fields, and replacing the original activation function with a sigmoid-weighted linear combination. The PANet module in the original feature fusion neck network is replaced with a weighted bidirectional feature pyramid network. This network transmits high-level semantic information and low-level feature location information via a bottom-up channel, enhancing target classification and localization capabilities. This bottom-up channel connects the input and output nodes of the same layer of the backbone network, achieving bidirectional cross-layer feature mapping with weighted features through the weighted bidirectional feature pyramid network. The feature fusion process involves a convolutional attention module in each 3* convolutional layer of the neck network for feature fusion and the head network for feature detection. This module, used for the feedforward convolutional neural network, allows the intermediate feature maps to sequentially infer attention maps along the two independent dimensions of channel and space. The attention maps are then multiplied with the input feature maps for adaptive feature optimization to obtain more features, enabling the backbone network for feature extraction to quickly learn channel and spatial features. The head network for feature detection outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. These feature maps predict the target category and bounding box position, with the 80×80 feature map detecting small targets, the 40×40 feature map detecting medium targets, and the 20×20 feature map detecting large targets. Each feature map layer contains three predicted bounding boxes with different aspect ratios, each containing the target's position information and confidence level. Finally, a non-maximum suppression algorithm discards overlapping predicted bounding boxes with an intersection-union ratio exceeding a set threshold.

[0050] An attention transfer method based on activation function values ​​is used between teacher and student networks, and the degree of attention of neurons to the input is measured by calculating the absolute value statistics of the output function.

[0051] A multi-scale attention module is designed and inserted at the end of five CSP modules in the YOLOv5 networks of both the teacher and student networks. This allows the convolutional networks at different levels of the student network to learn feature extraction capabilities similar to those of the teacher network. This multi-scale attention module uses pooling layers of three different scales (1×1, 3×3, and 5×5) to extract feature map vectors R. W×H×CFor features at the corresponding scale, calculate the maximum P-th power of the absolute value of each of the three feature map vectors in dimension C to obtain the input feature map vector R. W×H×C Attention feature maps at different scales; finally, the pixels at corresponding positions in the three attention feature maps are summed to obtain a single-channel multi-scale attention feature map R. W×H After obtaining the multi-scale attention feature maps of the teacher network and the student network, the L2 distance between them is calculated using the minimum mean square error loss function, as shown in the following formula:

[0052]

[0053] The mask-wearing detection module is constructed and performs detection functions through the following steps:

[0054] (201) Create a mask-wearing detection dataset

[0055] Images of people entering and exiting the elevator when the elevator doors open and images of the elevator interior during operation are collected at different time periods; the collected images are filtered and cleaned to obtain a dataset consisting of color images and depth images; data augmentation preprocessing is performed on the collected dataset;

[0056] For images of people entering elevators, annotation tools are used to annotate the faces of people in color images to create a mask-wearing detection dataset, which is divided into three categories: not wearing a mask, wearing a mask, and wearing a mask improperly. The annotated mask-wearing detection dataset is then divided into training and validation sets according to the proportions.

[0057] (202) A teacher network is obtained by training the mask-wearing detection dataset on the training set of the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is then deployed on a low-cost edge device to obtain the mask-wearing detection module.

[0058] (203) The mask-wearing detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information:

[0059] The preprocessed image is input into the feature extraction backbone network. Different receptive fields are obtained through different convolutional kernels in a multi-branch repeatable convolutional neural network, and the information obtained from different receptive fields is added together. The feature extraction backbone network outputs primary feature maps of different sizes to the feature fusion neck network. By strengthening the feature extraction through a weighted bidirectional feature pyramid network, high-level semantic information and low-level feature location information are transmitted through a bottom-up channel, thereby enhancing the target classification and target localization capabilities. The bottom-up channel connects the input and output nodes of the same layer of the backbone network. The weighted bidirectional feature pyramid network achieves bidirectional cross-level and weighted feature map fusion. The feature fusion neck network combines the feature maps obtained from the FPN module and the weighted bidirectional feature pyramid network. The feature map is fused with the feature map obtained from the backbone network to fully utilize the features extracted by the backbone network. The fused feature map obtained from the neck network is input into the head network of feature detection. The head network of feature detection outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. The target category and bounding box position are predicted for the feature maps of different sizes. The 80×80 feature map is responsible for detecting small targets in the image, the 40×40 feature map is responsible for detecting medium targets in the image, and the 20×20 feature map is responsible for detecting large targets in the image. On each feature map layer, three prediction boxes with different aspect ratios are preset. Each prediction box contains the target's position information and confidence score. Finally, the non-maximum suppression algorithm discards overlapping prediction boxes with an intersection-union ratio exceeding a set threshold.

[0060] The mask removal behavior detection module is constructed and performs detection functions through the following steps:

[0061] (301) Create a prior knowledge dataset

[0062] Images of people entering and exiting the elevator when the elevator doors open and images of the elevator interior during operation are collected at different time periods; the collected images are filtered and cleaned to obtain a dataset consisting of color images and depth images; data augmentation preprocessing is performed on the collected dataset;

[0063] For the images of people entering the elevator and images of the elevator running in the data augmentation dataset, the annotation tool is used to annotate the hands, heads and masks of people in the color images as a prior knowledge dataset, which is divided into three categories: head, hand and mask. The annotated prior knowledge dataset is divided into training set and validation set according to the proportion.

[0064] (305) The teacher network is obtained by training the data of the training set of the prior knowledge dataset through the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask removal behavior detection module.

[0065] (303) The mask removal behavior detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information. The label L represents the mask removal behavior, and T represents the confidence threshold for the occurrence of the mask removal behavior. The specific process is as follows:

[0066] Step 1. When the detection result predicts label L and the confidence level p0 > T, directly determine that the act of removing the mask has occurred;

[0067] Step 2. When the detection result predicts label L and the confidence level p0≤T, it is necessary to re-determine whether the mask removal behavior has occurred by using the distance relationship between targets related to the mask removal behavior, determine the actual increment of the confidence level of the mask removal behavior, and then correct the confidence level obtained by the improved YOLOv5 network to obtain the final confidence level of the mask removal behavior. This includes the following steps:

[0068] The distances between the center points of each predicted bounding box are calculated to obtain the distance values ​​Dist(head, mask), Dist(hand, mask), and Dist(head, hand). The probability of mask removal in the image is correlated with these distance values. Using the minor axis distance Len(face) of the detected head predicted bounding box as a reference distance, a distance-based judgment rule is established to determine the actual confidence increment of mask removal behavior.

[0069] Rule 1: When the distance condition 1 is met: Dist(hand, mask) ≤ a × Len(face), the actual increment of the confidence in the occurrence of the mask removal behavior is p1;

[0070] Rule 2: When distance condition 2 is met: Dist(head, mask) ≥ b × Len(face), the actual increment of confidence in the occurrence of mask removal is p2.

[0071] Rule 3: When the distance condition 3 is met: Dist(head, hand) ≤ c × Len(face), the actual increment of the confidence level for the mask removal behavior is p3.

[0072] The behavioral weights a, b, and c are obtained through statistical analysis of the information labeled in the training samples in the prior knowledge dataset. The parameters p1, p2, and p3 are first assigned manually based on the contribution of prior knowledge to the occurrence of mask removal behavior. Then, p1, p2, and p3 are normalized. That is, when the above rules are satisfied at the same time, p1 + p2 + p3 = 1. After obtaining the normalized parameters p1, p2, and p3, they are verified and adjusted in the validation set of the prior knowledge dataset so that the parameters p1, p2, and p3 meet the expectations for mask removal behavior detection.

[0073] By determining whether the above three distance conditions are met, the actual increase in confidence level for the mask-removal behavior is determined. This is then used to correct the confidence level obtained from the improved YOLOv5 network, resulting in the final confidence level for the mask-removal behavior.

[0074] If the final confidence level p of the mask removal behavior is higher than the threshold T, then the mask removal behavior is determined to have occurred; otherwise, the mask removal behavior is determined not to have occurred.

[0075] Step 3. When the detection result has no label L and p0 = 0, it is necessary to re-determine whether the mask removal behavior has occurred by using the distance relationship between the targets related to the mask removal behavior, determine the actual increment of the confidence that the mask removal behavior has occurred, and then correct the confidence obtained by the improved YOLOv5 network to obtain the final confidence p that the mask removal behavior has occurred.

[0076] If the final confidence level p of the mask removal action is higher than the threshold T, then the mask removal action is determined to have occurred; otherwise, the mask removal action is determined not to have occurred.

[0077] This invention utilizes knowledge distillation on a teacher model to obtain a lightweight student network model with detection accuracy approaching that of the teacher network model. By employing knowledge distillation to guide the student network through the teacher network, the "knowledge" learned from complex models can be transferred to simpler models with fewer parameters, resulting in a model with a simpler network structure and faster inference speed. This invention also compresses neural networks originally designed for high-performance computers and deploys them on low-cost mobile edge devices through knowledge distillation, enabling real-time, high-precision detection of mask-wearing status, providing reminders and recording, thus safeguarding public health and safety during the pandemic. Simultaneously, it reduces the costs of property management and intelligent equipment, demonstrating broad application prospects and practical significance in the current context.

[0078] This invention improves the backbone of the traditional Yolov5 network by replacing the original 3×3 Conv module with a RepVGG module. The multi-branch structure of the RepVGG module uses different convolutional kernels to obtain different receptive fields. The information obtained from different receptive fields is added together to enhance the extraction of feature information and improve the model performance. Thus, even in the confined space of an elevator, where a large number of people enter the elevator at the same time and there is a lot of occlusion, feature extraction can still be performed effectively.

[0079] In the improved YOLOv5 network of this invention, a primary feature map is obtained through the backbone network, and then input into the neck network for feature fusion to obtain a fused feature map. This improved YOLOv5 model employs an FPN+PANeT structure, where the feature pyramid network (FPN) transmits semantic information from higher levels to lower levels, while the PANET structure transmits localization information from lower levels to higher levels, thus better fusing semantic and localization information and fully utilizing the features extracted by the backbone network. This invention replaces the PANet structure in the traditional YOLOv5 network neck network with a weighted bidirectional feature pyramid network, and fuses the feature maps obtained through the FPN and weighted bidirectional feature pyramid network with the feature map obtained from the backbone network, achieving full utilization of the features extracted by the backbone network. This invention optimizes the network structure of the YOLOv5 model, simplifies network complexity, enhances target localization capabilities, fuses more feature information, and makes feature fusion simpler and more efficient.

[0080] To optimize small object detection capabilities and reduce interference from complex backgrounds during elevator door opening, this invention adds a convolutional attention module between the neck and head networks of a traditional YOLOv5 network. This is a simple yet effective attention module for feedforward convolutional neural networks (CBAM). Given an intermediate feature map, the CBAM module sequentially infers attention maps along two independent dimensions (channel and spatial), then multiplies the attention map with the input feature map for adaptive feature optimization to acquire more features, enabling the network to learn channel and spatial features more effectively and quickly. This improved YOLOv5 network acquires more features, allowing it to learn channel and spatial features more effectively and quickly, thus preventing interference and occlusion from complex backgrounds during elevator door opening.

[0081] To enable the improved YOLOv5 model of this invention to better perform knowledge distillation and run on low-cost edge devices, this invention also provides a multi-scale attention module, which is inserted at the end of the five CSP modules in the improved YOLOv5 network, allowing the convolutional networks at different levels of the student network to learn feature extraction capabilities similar to those of the teacher network.

[0082] To address the issue that elevator cameras are not fixed in location, and most are positioned behind people, making it difficult to detect whether people facing away from the camera are wearing masks, this invention determines mask wearing by detecting whether people are wearing masks when entering the elevator and whether they remove their masks while the elevator is running. Considering that removing a mask is highly similar to other actions, and that the trained model depends on the training set, the actual behavior of mask removal varies considerably. A typical training set is unlikely to cover all situations, leading to low accuracy. Furthermore, considering that mask removal often involves prior conditions, such as constraints on the positional relationship between the face, hands, and mask, this invention combines an improved YOLOv5 model with prior behavioral knowledge to detect mask removal, resulting in higher accuracy and greater versatility. Attached Figure Description

[0083] Figure 1 This is a flowchart of the method of the present invention;

[0084] Figure 2 A schematic diagram of the unmodified YOLOv5 model structure;

[0085] Figure 3 This is a schematic diagram of the improved YOLOv5 model structure of the present invention;

[0086] Figure 4 This is a schematic diagram of the multi-scale attention module of the present invention;

[0087] Figure 5 This is a schematic diagram of the multi-scale attention module between the teacher network and the student network in the knowledge distillation of this invention. Detailed Implementation

[0088] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0089] Numerous specific details are set forth in the following description to provide a thorough understanding of the invention. The described embodiments are merely some, not all, of the embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0090] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0091] The terms "first," "second," and "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects and not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, is intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0092] Example 1

[0093] In Embodiment 1, the present invention provides a mask detection method for an elevator scenario based on edge computing, comprising the following steps:

[0094] Step 100: Construct an improved YOLOv5 network, perform image prediction using the improved YOLOv5 network, and output the predicted behavior and related target labels, confidence scores, and location information.

[0095] like Figure 2 The image shows a traditional Yolov5 network, which mainly consists of three parts: a backbone network for feature extraction, a neck network for feature fusion, and a head network for detection. It adopts an FPN+PANeT structure, in which the Feature Pyramid Network (FPN) transmits semantic information from the high-level network to the low-level network, while the PANET structure transmits localization information from the low-level network to the high-level network, thereby fusing semantic and localization information.

[0096] like Figure 3As shown, the improved Yolov5 network includes a feature extraction backbone network, a feature fusion neck network, and a feature detection head network. It is an improvement on the traditional Yolov5 network using an FPN+PANeT structure. The original 3×3 convolutional modules in the feature extraction backbone network are modified into repeatable convolutional neural networks (RepVGG modules) with multi-branch structures and different convolutional kernels. These repeatable convolutional neural networks use different convolutional kernels to obtain different receptive fields, and the information obtained from different receptive fields is added together. Simultaneously, a sigmoid-weighted linear combination is used instead of the original activation function. The PANet module in the original feature fusion neck network is replaced with a weighted bidirectional feature pyramid network (BiFPN module). This weighted bidirectional feature pyramid network transmits high-level semantic information and low-level feature location information through a bottom-up channel to enhance target classification and localization capabilities. This bottom-up channel connects the input and output nodes of the same layer of the backbone network, achieving bidirectional... The system employs cross-layer and weighted feature map fusion. Each 3* convolutional layer of the neck network for feature fusion and the head network for feature detection includes a convolutional attention module. This module, used in the feedforward convolutional neural network, allows intermediate feature maps to sequentially infer attention maps along two independent dimensions: channel and spatial. The attention maps are then multiplied by the input feature maps for adaptive feature optimization to acquire more features, enabling the backbone network for feature extraction to quickly learn both channel and spatial features. The head network for feature detection outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. These maps predict the target category and bounding box position. The 80×80 feature map detects small targets, the 40×40 map detects medium-sized targets, and the 20×20 map detects large targets. Each feature map layer contains three predicted bounding boxes with different aspect ratios, each containing the target's location information and confidence level. Finally, a non-maximum suppression algorithm discards overlapping predicted bounding boxes with an intersection-union ratio exceeding a set threshold.

[0097] Step 200: Construct a mask-wearing detection module

[0098] (201) Create a mask-wearing detection dataset

[0099] Use a 2-megapixel, 2.8mm wide-angle infrared camera commonly used in elevators to collect images of people entering and exiting the elevator when the elevator doors open and images of the elevator interior during different time periods. The images should include both daytime and nighttime scenes, as well as images from different angles and heights, to ensure that the model has good generalization ability and robustness.

[0100] The collected images are filtered and cleaned to select clear and distinct images, thereby obtaining a dataset consisting of color and depth images;

[0101] The collected dataset is augmented using data augmentation techniques, including but not limited to combinations of various data augmentation techniques such as spatial transformation, flipping and skewing, random transformation, and color transformation.

[0102] For images of people entering elevators, a labeling tool was used to annotate the faces of people in color images to create a mask-wearing detection dataset. The dataset was divided into three categories: not wearing a mask, wearing a mask, and wearing a mask improperly. The labeled mask-wearing detection dataset was then divided into a training set and a validation set according to the proportions.

[0103] (202) A teacher network is obtained by training the mask-wearing detection dataset on the training set of the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is then deployed on a low-cost edge device to obtain the mask-wearing detection module.

[0104] like Figure 5 As shown, to avoid the problem of different convolutional channel numbers between the teacher and student networks and to better perform knowledge distillation of feature maps, this invention uses an attention transfer method based on activation function values ​​between the teacher and student networks. The absolute value statistic of the output function is used to measure the degree of attention a neuron pays to the input. To reduce computational cost, this invention uses the calculation of the feature map vector V... i The spatial attention feature map is generated by maximizing the P-square of the absolute value of dimension C, and the formula is as follows:

[0105]

[0106] To enable the improved Yolov5 network of this invention to perform better knowledge distillation, allowing the model to run on low-cost edge devices. For example... Figure 4 As shown, this invention designs a multi-scale attention module, which is inserted at the end of five CSP modules in the YOLOv5 network of both the teacher and student networks. This enables the convolutional networks at different levels of the student network to learn feature extraction capabilities similar to those of the teacher network. This multi-scale attention module uses pooling layers of three different scales—1×1, 3×3, and 5×5—to extract the feature map vector R. W×HxC Based on the features at the corresponding scale, the maximum value of the absolute value of the three feature map vectors in dimension C is calculated using formula (1) to obtain the input feature map vector R. W×H×C Attention feature maps at different scales; finally, the pixels at corresponding positions in the three attention feature maps are summed to obtain a single-channel multi-scale attention feature map R. W×HAfter obtaining the multi-scale attention feature maps of the teacher network and the student network, the L2 distance between them is calculated using the minimum mean square error loss function, as shown in the following formula:

[0107]

[0108] After obtaining the student network, the model needs to be lightweighted. Since the entire feature extraction process of the YOLOv5 model directly uses PyTorch's Torch tensor correlation operators to process the data, there is code in the detection layer that does not use Torch tensor correlation operators. Therefore, when quantizing the model, the code needs to be modified and added to the detection function.

[0109] (203) The mask-wearing detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores and location information.

[0110] Images from the mask-wearing detection dataset are preprocessed using the Mosaic data augmentation method in the YOL0v5 network. The preprocessed images are then input into the feature extraction backbone network. Different receptive fields are obtained through different convolutional kernels in a multi-branch repeatable convolutional neural network, and the information from these different receptive fields is summed. The backbone network outputs primary feature maps of different sizes, which are then fed into the feature fusion neck network. A weighted bidirectional feature pyramid network is used to enhance feature extraction, transmitting high-level semantic information and low-level feature location information via a bottom-up channel, thereby improving target classification and localization capabilities. This bottom-up channel connects the input and output nodes of the same layer of the backbone network. The weighted bidirectional feature pyramid network enables bidirectional, cross-layer, and weighted feature map fusion. The feature fusion neck network then passes through F... The feature maps obtained by the PN module and the weighted bidirectional feature pyramid network are fused with the feature maps obtained by the backbone network to fully utilize the features extracted by the backbone network. The fused feature map obtained by the neck network of feature fusion is input into the head network of feature detection. The head network of feature detection outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. The target category and bounding box position are predicted for the feature maps of different sizes respectively. The 80×80 feature map is responsible for detecting small targets in the image, the 40×40 feature map is responsible for detecting medium targets in the image, and the 20×20 feature map is responsible for detecting large targets in the image. On each feature map layer, three prediction boxes with different aspect ratios are preset. Each prediction box contains the target's position information and confidence level. Finally, the non-maximum suppression algorithm discards overlapping prediction boxes with an intersection-union ratio exceeding a set threshold.

[0111] Step 300: Construct a mask removal behavior detection module

[0112] (301) Create a prior knowledge dataset

[0113] For the images of people entering the elevator and images of the elevator running in the data augmentation dataset, the Labeling tool is used to label the hands, heads and masks worn by people in the color images as a prior knowledge dataset, which is divided into three categories: head, hand and mask. The labeled prior knowledge dataset is divided into training set and validation set according to the proportion.

[0114] (302) The teacher network is trained on the training set of the prior knowledge dataset by the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask removal behavior detection module. The specific method is the same as step 202.

[0115] Because the action of removing a mask is highly similar to other actions, and the trained model depends on the training set, and because the behavior of removing a mask varies greatly in reality, a typical training set is unlikely to cover all situations, leading to low accuracy. Considering that the act of removing a mask is often accompanied by prior conditions, such as constraints on the positional relationship between the face, hands, and mask, this invention combines an improved YOLOv5 with prior behavioral knowledge to detect the act of removing a mask.

[0116] The prior knowledge regarding the act of removing a mask in this invention includes:

[0117] 1. The prediction box for mask removal behavior should include the head area. In cases where multiple people may appear in the image at the same time, the predicted behavior in the prediction box corresponds to the person whose head is included.

[0118] 2. When the act of removing a mask occurs, there are certain constraints on the positional relationship between the head, hands, and mask. When the confidence level of the behavior label predicted by the trained student network is low, or when the actual behavior is missed or falsely detected, these constraints can be used to establish logical reasoning rules based on prior knowledge of behavior, and further determine the behavior.

[0119] (303) The mask removal behavior detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information. The label L represents the mask removal behavior, and T represents the confidence threshold for the occurrence of the mask removal behavior. The specific process is as follows:

[0120] 1. When the detection result predicts label L and the confidence level p0 > T, it is directly determined that the act of removing the mask has occurred;

[0121] 2. When the detection result predicts label L and the confidence level p0≤T, it is necessary to re-determine whether the mask removal behavior has occurred based on the distance relationship between targets related to the mask removal behavior, including the following steps:

[0122] The distances between the center points of each predicted bounding box are calculated to obtain the distance values ​​Dist(head, mask), Dist(hand, mask), and Dist(head, hand). The probability of mask removal in the image is correlated with these distance values. Since the absolute distance between pixels varies with image scale, the minor axis distance Len(face) of the detected head predicted bounding box is used as a reference distance to establish distance-based judgment rules, thereby determining the actual increment of confidence in mask removal.

[0123] Rule 1: When the distance condition 1 is met: Dist(hand, mask) ≤ a × Len(face), the actual increment of the confidence in the occurrence of the mask removal behavior is p1;

[0124] Rule 2: When distance condition 2 is met: Dist(head, mask) ≥ b × Len(face), the actual increment of confidence in the occurrence of mask removal is p2.

[0125] Rule 3: When the distance condition 3 is met: Dist(head, hand) ≤ c × Len(face), the actual increment of the confidence level for the mask removal behavior is p3.

[0126] The behavioral weights a, b, and c are obtained through statistical analysis of the information labeled in the training samples of the prior knowledge dataset. The parameters p1, p2, and p3 are first assigned manually based on experience, according to the contribution of prior knowledge to the occurrence of mask removal behavior. Rule 2 is more important, rule 3 is relatively important, and rule 1 is less important due to the large number of people facing away from the camera in the elevator. Then, p1, p2, and p3 are normalized, that is, when the above rules are satisfied at the same time, p1 + p2 + p3 = 1. After obtaining the normalized parameters p1, p2, and p3, they are validated and adjusted in the validation set of the prior knowledge dataset so that the parameters p1, p2, and p3 achieve good results in detecting mask removal behavior.

[0127] By determining whether the above three distance conditions are met, the actual increase in confidence level for the mask-removal behavior is determined. This is then used to correct the confidence level obtained from the improved YOLOv5 network, resulting in the final confidence level for the mask-removal behavior.

[0128] For whether the above distance condition i is true or not, i = 1, 2, 3, let the actual increase in confidence of the mask removal behavior at this time be:

[0129]

[0130] The final confidence level for the act of removing the mask is then revised as follows:

[0131]

[0132] If the final confidence level p of the mask removal behavior is higher than the threshold T, then the mask removal behavior is determined to have occurred; otherwise, the mask removal behavior is determined not to have occurred.

[0133] 3. When the detection result has no label L and p0 = 0, it is necessary to re-determine whether the mask removal behavior has occurred by using the distance relationship between targets related to the mask removal behavior. The actual increase in confidence that the mask removal behavior has occurred is Δp. i (i = 1, 2, 3), then the final confidence level for the occurrence of the mask removal behavior is revised as follows:

[0134]

[0135] If the final confidence level p of the mask removal action is higher than the threshold T, then the mask removal action is determined to have occurred; otherwise, the mask removal action is determined not to have occurred.

[0136] Step 400, such as Figure 1 As shown, when the elevator starts running, it triggers the start-up control module to control other modules to work, and the camera starts acquiring images. When the elevator door opens, the mask-wearing detection module on the mobile edge computing device acquires images of people entering and exiting the elevator from the camera and detects whether people are wearing masks. When the elevator door closes, the mask removal behavior detection module on the mobile edge computing device acquires images of people in the elevator from the camera and identifies whether mask removal has occurred. If there is no mask, improper mask wearing, or mask removal, the system will remind people in the elevator through the sound output device. At the same time, the identified images and results will be transmitted to the server host through the transmission network. The server host will archive and store the collected images and results and display them visually, which will help the back-end administrator to monitor and review in real time.

[0137] Example 2

[0138] In Embodiment 2, the present invention provides a mask detection system for an elevator scenario based on edge computing, comprising a start control module, an image acquisition module, a mask wearing detection module, and a mask removal behavior detection module, wherein...

[0139] The start-up control module is triggered in response to the opening of the elevator door, controlling all modules to leave standby and enter the working state, and controlling the image acquisition module to collect images of people entering and exiting the elevator and images of the elevator interior during operation from the elevator monitoring camera.

[0140] The image acquisition module enhances the images of people entering and exiting the elevator and the images inside the elevator during operation captured by the elevator monitoring camera. Then, it sends the images of people entering and exiting the elevator to the mask wearing detection module and the images inside the elevator during operation to the mask removal behavior detection module.

[0141] The mask-wearing detection module detects whether people are wearing masks based on images of people entering and exiting the elevator transmitted by the camera when the elevator door opens, using an improved YOL0v5 network. It outputs the predicted behavior and related target labels, confidence scores, and location information to obtain detection results for not wearing masks or wearing masks improperly.

[0142] The mask removal behavior detection module detects whether people have removed their masks based on the images of the elevator inside the elevator transmitted by the camera when the elevator door is closed, using an improved YOL0v5 network. It outputs the predicted behavior and the labels, confidence scores, and location information of related targets to obtain the detection result of the mask removal behavior.

[0143] The improved Yolov5 network, comprising a feature extraction backbone, a feature fusion neck network, and a feature detection head network, is an improvement on the traditional Yolov5 network using an FPN+PANeT structure. The original 3×3 convolutional modules in the feature extraction backbone are replaced with repeatable convolutional neural networks (CNNs) with multi-branch structures and different kernels. These CNNs use different kernels to obtain different receptive fields, summing the information from different receptive fields, and replacing the original activation function with a sigmoid-weighted linear combination. The PANet module in the original feature fusion neck network is replaced with a weighted bidirectional feature pyramid network. This network transmits high-level semantic information and low-level feature location information via a bottom-up channel, enhancing target classification and localization capabilities. This bottom-up channel connects the input and output nodes of the same layer of the backbone network, achieving bidirectional cross-layer feature mapping with weighted features through the weighted bidirectional feature pyramid network. The feature fusion process involves a convolutional attention module in each 3* convolutional layer of the neck network for feature fusion and the head network for feature detection. This module, used for the feedforward convolutional neural network, allows the intermediate feature maps to sequentially infer attention maps along the two independent dimensions of channel and space. The attention maps are then multiplied with the input feature maps for adaptive feature optimization to obtain more features, enabling the backbone network for feature extraction to quickly learn channel and spatial features. The head network for feature detection outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. These feature maps predict the target category and bounding box position, with the 80×80 feature map detecting small targets, the 40×40 feature map detecting medium targets, and the 20×20 feature map detecting large targets. Each feature map layer contains three predicted bounding boxes with different aspect ratios, each containing the target's position information and confidence level. Finally, a non-maximum suppression algorithm discards overlapping predicted bounding boxes with an intersection-union ratio exceeding a set threshold.

[0144] An attention transfer method based on activation function values ​​is used between teacher and student networks, and the degree of attention of neurons to the input is measured by calculating the absolute value statistics of the output function.

[0145] A multi-scale attention module is designed and inserted at the end of five CSP modules in the YOLOv5 networks of both the teacher and student networks. This allows the convolutional networks at different levels of the student network to learn feature extraction capabilities similar to those of the teacher network. This multi-scale attention module uses pooling layers of three different scales (1×1, 3×3, and 5×5) to extract feature map vectors R. W×H×CFor features at the corresponding scale, calculate the maximum P-th power of the absolute value of each of the three feature map vectors in dimension C to obtain the input feature map vector R. W×H×C Attention feature maps at different scales; finally, the pixels at corresponding positions in the three attention feature maps are summed to obtain a single-channel multi-scale attention feature map R. W×H After obtaining the multi-scale attention feature maps of the teacher network and the student network, the L2 distance between them is calculated using the minimum mean square error loss function, as shown in the following formula:

[0146]

[0147] The mask-wearing detection module is constructed and performs detection functions through the following steps:

[0148] (201) Create a mask-wearing detection dataset

[0149] Images of people entering and exiting the elevator when the elevator doors open and images of the elevator interior during operation are collected at different time periods; the collected images are filtered and cleaned to obtain a dataset consisting of color images and depth images; data augmentation preprocessing is performed on the collected dataset;

[0150] For images of people entering elevators, annotation tools are used to annotate the faces of people in color images to create a mask-wearing detection dataset, which is divided into three categories: not wearing a mask, wearing a mask, and wearing a mask improperly. The annotated mask-wearing detection dataset is then divided into training and validation sets according to the proportions.

[0151] (202) A teacher network is obtained by training the mask-wearing detection dataset on the training set of the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is then deployed on a low-cost edge device to obtain the mask-wearing detection module.

[0152] (203) The mask-wearing detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information:

[0153] The preprocessed image is input into the feature extraction backbone network. Different receptive fields are obtained through different convolutional kernels in a multi-branch repeatable convolutional neural network, and the information obtained from different receptive fields is added together. The feature extraction backbone network outputs primary feature maps of different sizes to the feature fusion neck network. By strengthening the feature extraction through a weighted bidirectional feature pyramid network, high-level semantic information and low-level feature location information are transmitted through a bottom-up channel, thereby enhancing the target classification and target localization capabilities. The bottom-up channel connects the input and output nodes of the same layer of the backbone network. The weighted bidirectional feature pyramid network achieves bidirectional cross-level and weighted feature map fusion. The feature fusion neck network combines the feature maps obtained from the FPN module and the weighted bidirectional feature pyramid network. The feature map is fused with the feature map obtained from the backbone network to fully utilize the features extracted by the backbone network. The fused feature map obtained from the neck network is input into the head network of feature detection. The head network of feature detection outputs three feature maps of different sizes: 20×20, 40×40, and 80×80. The target category and bounding box position are predicted for the feature maps of different sizes. The 80×80 feature map is responsible for detecting small targets in the image, the 40×40 feature map is responsible for detecting medium targets in the image, and the 20×20 feature map is responsible for detecting large targets in the image. On each feature map layer, three prediction boxes with different aspect ratios are preset. Each prediction box contains the target's position information and confidence score. Finally, the non-maximum suppression algorithm discards overlapping prediction boxes with an intersection-union ratio exceeding a set threshold.

[0154] The mask removal behavior detection module is constructed and performs detection functions through the following steps:

[0155] (301) Create a prior knowledge dataset

[0156] Images of people entering and exiting the elevator when the elevator doors open and images of the elevator interior during operation are collected at different time periods; the collected images are filtered and cleaned to obtain a dataset consisting of color images and depth images; data augmentation preprocessing is performed on the collected dataset;

[0157] For the images of people entering the elevator and images of the elevator running in the data augmentation dataset, the annotation tool is used to annotate the hands, heads and masks of people in the color images as a prior knowledge dataset, which is divided into three categories: head, hand and mask. The annotated prior knowledge dataset is divided into training set and validation set according to the proportion.

[0158] (306) The teacher network is obtained by training the data of the training set of the prior knowledge dataset through the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask removal behavior detection module.

[0159] (303) The mask removal behavior detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information. The label L represents the mask removal behavior, and T represents the confidence threshold for the occurrence of the mask removal behavior. The specific process is as follows:

[0160] Step 1. When the detection result predicts label L and the confidence level p0 > T, directly determine that the act of removing the mask has occurred;

[0161] Step 2. When the detection result predicts label L and the confidence level p0≤T, it is necessary to re-determine whether the mask removal behavior has occurred by using the distance relationship between targets related to the mask removal behavior, determine the actual increment of the confidence level of the mask removal behavior, and then correct the confidence level obtained by the improved YOLOv5 network to obtain the final confidence level of the mask removal behavior. This includes the following steps:

[0162] The distances between the center points of each predicted bounding box are calculated to obtain the distance values ​​Dist(head, mask), Dist(hand, mask), and Dist(head, hand). The probability of mask removal in the image is correlated with these distance values. Using the minor axis distance Len(face) of the detected head predicted bounding box as a reference distance, a distance-based judgment rule is established to determine the actual confidence increment of mask removal behavior.

[0163] Rule 1: When the distance condition 1 is met: Dist(hand, mask) ≤ a × Len(face), the actual increment of the confidence in the occurrence of the mask removal behavior is p1;

[0164] Rule 2: When distance condition 2 is met: Dist(head, mask) ≥ b × Len(faCe), the actual increment of confidence in the occurrence of mask removal is p2.

[0165] Rule 3: When the distance condition 3 is met: Dist(head, hand) ≤ c × Len(face), the actual increment of the confidence level for the mask removal behavior is p3.

[0166] The behavioral weights a, b, and c are obtained through statistical analysis of the information labeled in the training samples in the prior knowledge dataset. The parameters p1, p2, and p3 are first assigned manually based on the contribution of prior knowledge to the occurrence of mask removal behavior. Then, p1, p2, and p3 are normalized. That is, when the above rules are satisfied at the same time, p1 + p2 + p3 = 1. After obtaining the normalized parameters p1, p2, and p3, they are verified and adjusted in the validation set of the prior knowledge dataset so that the parameters p1, p2, and p3 meet the expectations for mask removal behavior detection.

[0167] By determining whether the above three distance conditions are met, the actual increase in confidence level for the mask-removal behavior is determined. This is then used to correct the confidence level obtained from the improved YOLOv5 network, resulting in the final confidence level for the mask-removal behavior.

[0168] If the final confidence level p of the mask removal behavior is higher than the threshold T, then the mask removal behavior is determined to have occurred; otherwise, the mask removal behavior is determined not to have occurred.

[0169] Step 3. When the detection result has no label L and p0 = 0, it is necessary to re-determine whether the mask removal behavior has occurred by using the distance relationship between the targets related to the mask removal behavior, determine the actual increment of the confidence that the mask removal behavior has occurred, and then correct the confidence obtained by the improved YOLOv5 network to obtain the final confidence p that the mask removal behavior has occurred.

[0170] If the final confidence level p of the mask removal action is higher than the threshold T, then the mask removal action is determined to have occurred; otherwise, the mask removal action is determined not to have occurred.

[0171] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units or modules as needed, that is, the internal structure of the storage device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A mask detection method in an elevator scenario based on edge computing, characterized in that... Includes the following steps: Step 100: Construct an improved Yolov5 network The improved Yolov5 network, comprising a feature extraction backbone network, a feature fusion neck network, and a feature detection head network, is an improvement upon the traditional Yolov5 network employing an FPN+PANeT structure. It incorporates features from the original feature extraction backbone network... The convolutional module is modified into a repeatable convolutional neural network with a multi-branch structure and different convolutional kernels. The multi-branch structure of the repeatable convolutional neural network uses different convolutional kernels to obtain different receptive fields. The information obtained from different receptive fields is added together, and a sigmoid weighted linear combination is used instead of the original activation function. The PANet module in the original feature fusion neck network is replaced with a weighted bidirectional feature pyramid network. The weighted bidirectional feature pyramid network transmits high-level semantic information and low-level feature location information through bottom-up channels. The bottom-up channels connect the input and output nodes of the same backbone network. The weighted bidirectional feature pyramid network realizes bidirectional cross-level and weighted feature map fusion. A convolutional attention module is set in each 3* convolutional layer of the feature fusion neck network and the feature detection head network. The convolutional attention module used for the feedforward convolutional neural network makes the attention map infer sequentially along the two independent dimensions of channel and space. Then, the attention map is multiplied with the input feature map for adaptive feature optimization to obtain more features, so that the feature extraction backbone network can quickly learn channel features and spatial features. Head network output for feature detection , and There are three feature maps of different sizes. The target category and bounding box position are predicted on the feature maps of different sizes respectively. The 80×80 feature map is responsible for detecting small targets in the image, the 40×40 feature map is responsible for detecting medium targets in the image, and the 20×20 feature map is responsible for detecting large targets in the image. On each feature map layer, three prediction boxes with different aspect ratios are preset. Each prediction box contains the target's position information and confidence score. Finally, the non-maximum suppression algorithm discards overlapping prediction boxes with an intersection-union ratio exceeding a set threshold. Step 200: Train the teacher network using the data in the training set of the mask wearing detection dataset through the improved YOLOv5 network. After knowledge distillation, obtain a lightweight student network. Deploy the lightweight student network on a low-cost edge device to obtain the mask wearing detection module. The mask wearing detection module performs image prediction based on the improved YOLOv5 network and outputs the predicted behavior and related target labels, confidence scores and location information. Step 300: The teacher network is trained on the training set of the prior knowledge dataset using the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask removal behavior detection module. The mask removal behavior detection module performs image prediction based on the improved YOLOv5 network and outputs the predicted behavior and the labels, confidence scores and location information of related targets. Step 400: The elevator starts running, triggering the start control module to control other modules to work, and the camera starts to acquire images; when the elevator door opens, the mask wearing detection module on the mobile edge computing device acquires images of people entering and exiting the elevator from the camera and detects whether people are wearing masks; when the elevator door closes, the mask removal behavior detection module on the mobile edge computing device acquires images of people in the elevator from the camera and identifies whether mask removal behavior has occurred. If there is no mask being worn, the mask being worn improperly, or mask removal behavior has occurred, an alarm will be issued to the people in the elevator.

2. The mask detection method in an elevator scenario based on edge computing according to claim 1, characterized in that: An attention transfer method based on activation function values ​​is used between teacher and student networks, and the degree of attention of neurons to the input is measured by calculating the absolute value statistics of the output function. A multi-scale attention module is designed and inserted at the end of five CSP modules in the YOLOv5 network for both the teacher and student networks. This allows the convolutional networks at different levels of the student network to learn feature extraction capabilities similar to those of the teacher network. This multi-scale attention module uses... , and Three pooling layers of different scales extract feature map vectors respectively. For features at the corresponding scale, calculate the maximum P-th power of the absolute value of each of the three feature map vectors in dimension C to obtain the input feature map vector. Attention feature maps at different scales are generated; finally, the pixels at corresponding positions in the three attention feature maps are summed to obtain a single-channel multi-scale attention feature map. After obtaining the multi-scale attention feature maps of the teacher network and the student network, the L2 distance between them is calculated using the minimum mean square error loss function, as shown in the following formula: （2）。 3. A mask detection method in an elevator scenario based on edge computing according to any one of claims 1 or 2, characterized in that... Step 200 specifically involves: (201) Create a mask-wearing detection dataset: Images of people entering and exiting the elevator when the doors open and images of the elevator interior during operation are collected at different time periods; the collected images are filtered and cleaned to obtain a dataset consisting of color and depth images; the collected dataset is then processed. Data augmentation preprocessing; For images of people entering elevators, annotation tools are used to annotate the faces of people in color images to create a mask-wearing detection dataset, which is divided into three categories: not wearing a mask, wearing a mask, and wearing a mask improperly. The annotated mask-wearing detection dataset is then divided into training and validation sets according to the proportions. (202) The teacher network is trained on the training set of the mask wearing detection dataset by the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask wearing detection module. (203) The mask-wearing detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information: The preprocessed image is input into the feature extraction backbone network. Different receptive fields are obtained through different convolutional kernels in a multi-branch repeatable convolutional neural network, and the information obtained from different receptive fields is added together. The feature extraction backbone network outputs primary feature maps of different sizes to the feature fusion neck network. By strengthening the feature extraction through a weighted bidirectional feature pyramid network, high-level semantic information and low-level feature location information are transmitted through a bottom-up channel, thereby enhancing the target classification and target localization capabilities. This bottom-up channel connects the input and output nodes of the same layer of the backbone network. The weighted bidirectional feature pyramid network achieves bidirectional cross-level and weighted feature map fusion. The feature fusion neck network fuses the feature maps obtained by the FPN module and the weighted bidirectional feature pyramid network with the feature maps obtained by the backbone network, realizing full utilization of the features extracted by the backbone network. The fused feature map obtained by the feature fusion neck network is input into the feature detection head network, and the feature detection head network outputs... , and There are three feature maps of different sizes. The target category and bounding box position are predicted for each feature map of different sizes. The 80×80 feature map is responsible for detecting small targets in the image, the 40×40 feature map is responsible for detecting medium targets in the image, and the 20×20 feature map is responsible for detecting large targets in the image. On each feature map layer, three prediction boxes with different aspect ratios are preset. Each prediction box contains the target's position information and confidence score. Finally, the non-maximum suppression algorithm discards overlapping prediction boxes with an intersection-union ratio exceeding a set threshold.

4. A mask detection method in an elevator scenario based on edge computing according to any one of claims 1 or 2, characterized in that... Step 300 specifically involves: (301) Create a prior knowledge dataset For the images of people entering the elevator and images of the elevator running in the data augmentation dataset, the annotation tool is used to annotate the hands, heads and masks of people in the color images as a prior knowledge dataset, which is divided into three categories: head, hand and mask. The annotated prior knowledge dataset is divided into training set and validation set according to the proportion. (302) The teacher network is obtained by training the data of the training set of the prior knowledge dataset through the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask removal behavior detection module. (303) The mask removal behavior detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information, using labels. To indicate the act of removing a mask, use The confidence threshold for indicating the occurrence of mask removal is determined as follows: Step 1. When the detection results predict the label And confidence level At that time, it is directly determined that the act of removing the mask has occurred; Step 2. When the detection results predict the label And confidence level If the mask-removal behavior occurs, it is necessary to re-determine whether the mask-removal behavior has occurred by using the distance relationship between the targets related to the mask-removal behavior, determine the actual increment of the confidence that the mask-removal behavior has occurred, and then correct the confidence obtained by the improved YOLOv5 network to obtain the final confidence that the mask-removal behavior has occurred. This includes the following steps: The distance between the head and the mask is obtained by calculating the distance between the center points of each prediction frame. Distance between hands and mask and the distance between the hand and the head The probability of mask removal in an image is correlated with these distance values, using the minor axis distance of the detected head prediction boxes. As a reference distance, distance-based judgment rules are established to determine the actual increment in confidence that the act of removing a mask has occurred: Rule 1: When distance condition 1 is met: When the confidence level of removing the mask occurs, the actual increment is: ; Rule 2, when distance condition 2 is met: When the confidence level of removing the mask occurs, the actual increment is: ; Rule 3, when distance condition 3 is met: When the confidence level of removing the mask occurs, the actual increment is: ; Among them, behavioral weights The parameters are obtained through statistical analysis of the information labeled in the training samples of the prior knowledge dataset. First, the contribution of prior knowledge to the occurrence of mask removal is artificially assigned, and then... Normalization is performed when all of the above rules are met. After obtaining the normalized parameters Then, the parameters are validated and adjusted on the validation set of the prior knowledge dataset. , so that the parameters The detection of mask-removal behavior met expectations; By determining whether the above three distance conditions are met, the actual increase in confidence level for the mask-removal behavior is determined. This is then used to correct the confidence level obtained from the improved YOLOv5 network, resulting in the final confidence level for the mask-removal behavior. The final confidence level of the act of removing the mask Above the threshold If the mask is removed, it is determined that the act of removing the mask has occurred; otherwise, it is determined that the act of removing the mask has not occurred. Step 3. When the test result has no label and Then, it is necessary to re-determine whether the mask-removal behavior has occurred by analyzing the distance relationship between the targets related to the mask-removal behavior, determine the actual increment of the confidence that the mask-removal behavior has occurred, and then correct the confidence obtained from the improved YOLOv5 network to obtain the final confidence that the mask-removal behavior has occurred. ; If the final confidence level of the mask removal behavior occurs Above the threshold If the mask removal action occurs, it is determined that the mask removal action has occurred; otherwise, it is determined that the mask removal action has not occurred.

5. A mask detection method in an elevator scenario based on edge computing according to claim 2, characterized in that... The aforementioned attention transfer method based on activation function values ​​between the teacher and student networks measures the degree of attention a neuron pays to the input by calculating the absolute value statistic of the output function. Specifically: The feature map vector V is calculated. i The spatial attention feature map is generated by maximizing the P-square of the absolute value of dimension C, and the formula is as follows: .

6. A mask detection system for elevator scenarios based on edge computing, characterized in that: It includes a startup control module, an image acquisition module, a mask wearing detection module, and a mask removal behavior detection module, among which... The start-up control module is triggered in response to the opening of the elevator door, controlling all modules to leave standby and enter the working state, and controlling the image acquisition module to collect images of people entering and exiting the elevator and images of the elevator interior during operation from the elevator monitoring camera. The image acquisition module enhances the images of people entering and exiting the elevator and the images inside the elevator during operation captured by the elevator monitoring camera. Then, it sends the images of people entering and exiting the elevator to the mask wearing detection module and the images inside the elevator during operation to the mask removal behavior detection module. The mask-wearing detection module detects whether people are wearing masks based on images of people entering and exiting the elevator transmitted by the camera when the elevator door opens, using an improved YOLOv5 network. It outputs the predicted behavior and related target labels, confidence scores, and location information to obtain detection results for not wearing masks or wearing masks improperly. The mask removal behavior detection module detects whether people are removing their masks based on the images of the elevator inside the elevator transmitted by the camera when the elevator door is closed, using an improved YOLOv5 network. It outputs the predicted behavior and the labels, confidence scores, and location information of related targets to obtain the detection result of the mask removal behavior. The improved Yolov5 network, comprising a feature extraction backbone network, a feature fusion neck network, and a feature detection head network, is an improvement upon the traditional Yolov5 network employing an FPN+PANeT structure. It incorporates features from the original feature extraction backbone network... The convolutional module was modified into a repeatable convolutional neural network with a multi-branch structure and different convolutional kernels. This repeatable convolutional neural network uses different convolutional kernels to obtain different receptive fields, and the information obtained from different receptive fields is summed. Simultaneously, a sigmoid weighted linear combination is used instead of the original activation function. The PANet module in the original feature fusion neck network was replaced with a weighted bidirectional feature pyramid network. This weighted bidirectional feature pyramid network transmits high-level semantic information and low-level feature location information through bottom-up channels to enhance target classification and target localization capabilities. The bottom-up channels connect the input and output nodes of the same backbone network. A weighted bidirectional feature pyramid network is used to achieve bidirectional cross-layer fusion of weighted feature maps. Each 3* convolutional layer of the neck network for feature fusion and the head network for feature detection has a convolutional attention module. This convolutional attention module, used for the feedforward convolutional neural network, allows the intermediate feature map to infer the attention map sequentially along the two independent dimensions of channel and space. Then, the attention map is multiplied with the input feature map for adaptive feature optimization to obtain more features, enabling the backbone network for feature extraction to quickly learn channel features and spatial features. Head network output for feature detection , and There are three feature maps of different sizes. The target category and bounding box position are predicted on the feature maps of different sizes respectively. The 80×80 feature map is responsible for detecting small targets in the image, the 40×40 feature map is responsible for detecting medium targets in the image, and the 20×20 feature map is responsible for detecting large targets in the image. On each feature map layer, three prediction boxes with different aspect ratios are preset. Each prediction box contains the target's position information and confidence score. Finally, the non-maximum suppression algorithm discards overlapping prediction boxes with an intersection-union ratio exceeding a set threshold.

7. A mask detection system for elevator scenarios based on edge computing according to claim 6, characterized in that: An attention transfer method based on activation function values ​​is used between teacher and student networks, and the degree of attention of neurons to the input is measured by calculating the absolute value statistics of the output function. A multi-scale attention module is designed and inserted at the end of five CSP modules in the YOLOv5 network for both the teacher and student networks. This allows the convolutional networks at different levels of the student network to learn feature extraction capabilities similar to those of the teacher network. This multi-scale attention module uses... , and Three pooling layers of different scales extract feature map vectors respectively. For features at the corresponding scale, calculate the maximum P-th power of the absolute value of each of the three feature map vectors in dimension C to obtain the input feature map vector. Attention feature maps at different scales are generated; finally, the pixels at corresponding positions in the three attention feature maps are summed to obtain a single-channel multi-scale attention feature map. After obtaining the multi-scale attention feature maps of the teacher network and the student network, the L2 distance between them is calculated using the minimum mean square error loss function, as shown in the following formula: （2）。 8. A mask detection system for elevator scenarios based on edge computing according to claim 6, characterized in that... The mask-wearing detection module is constructed and performs detection functions through the following steps: (201) Create a mask wearing detection dataset Images of people entering and exiting the elevator when the doors open and images of the elevator interior during operation are collected at different time periods; the collected images are filtered and cleaned to obtain a dataset consisting of color and depth images; the collected dataset is then processed. Data augmentation preprocessing; For images of people entering elevators, annotation tools are used to annotate the faces of people in color images to create a mask-wearing detection dataset, which is divided into three categories: not wearing a mask, wearing a mask, and wearing a mask improperly. The annotated mask-wearing detection dataset is then divided into training and validation sets according to the proportions. (202) The teacher network is trained on the centralized training set of the mask wearing detection dataset using an improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is then deployed on a low-cost edge device to obtain the mask wearing detection module. (203) The mask-wearing detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information: The preprocessed image is input into the feature extraction backbone network. Different receptive fields are obtained through different convolutional kernels in a multi-branch repeatable convolutional neural network, and the information obtained from different receptive fields is added together. The feature extraction backbone network outputs primary feature maps of different sizes to the feature fusion neck network. By strengthening the feature extraction through a weighted bidirectional feature pyramid network, high-level semantic information and low-level feature location information are transmitted through a bottom-up channel, thereby enhancing the target classification and target localization capabilities. This bottom-up channel connects the input and output nodes of the same layer of the backbone network. The weighted bidirectional feature pyramid network achieves bidirectional cross-level and weighted feature map fusion. The feature fusion neck network fuses the feature maps obtained by the FPN module and the weighted bidirectional feature pyramid network with the feature maps obtained by the backbone network, realizing full utilization of the features extracted by the backbone network. The fused feature map obtained by the feature fusion neck network is input into the feature detection head network, and the feature detection head network outputs... , and There are three feature maps of different sizes. The target category and bounding box position are predicted for each feature map of different sizes. The 80×80 feature map is responsible for detecting small targets in the image, the 40×40 feature map is responsible for detecting medium targets in the image, and the 20×20 feature map is responsible for detecting large targets in the image. On each feature map layer, three prediction boxes with different aspect ratios are preset. Each prediction box contains the target's position information and confidence score. Finally, the non-maximum suppression algorithm discards overlapping prediction boxes with an intersection-union ratio exceeding a set threshold.

9. A mask detection system for elevator scenarios based on edge computing according to claim 6, characterized in that... The mask removal behavior detection module is constructed and performs detection functions through the following steps: (301) Create a prior knowledge dataset Images of people entering and exiting the elevator when the doors open and images of the elevator interior during operation are collected at different time periods; the collected images are filtered and cleaned to obtain a dataset consisting of color and depth images; the collected dataset is then processed. Data augmentation preprocessing; For the images of people entering the elevator and images of the elevator running in the data augmentation dataset, the annotation tool is used to annotate the hands, heads and masks of people in the color images as a prior knowledge dataset, which is divided into three categories: head, hand and mask. The annotated prior knowledge dataset is divided into training set and validation set according to the proportion. (303) The teacher network is obtained by training the data of the training set of the prior knowledge dataset through the improved YOLOv5 network. After knowledge distillation, a lightweight student network is obtained. The lightweight student network is deployed on a low-cost edge device to obtain the mask removal behavior detection module. (303) The mask removal behavior detection module performs image prediction based on the improved YOLOv5 network, and outputs the predicted behavior and related target labels, confidence scores, and location information, using labels. To indicate the act of removing a mask, use The confidence threshold for indicating the occurrence of mask removal is determined as follows: Step 1. When the detection results predict the label And confidence level At that time, it is directly determined that the act of removing the mask has occurred; Step 2. When the detection results predict the label And confidence level If the mask-removal behavior occurs, it is necessary to re-determine whether the mask-removal behavior has occurred by using the distance relationship between the targets related to the mask-removal behavior, determine the actual increment of the confidence that the mask-removal behavior has occurred, and then correct the confidence obtained by the improved YOLOv5 network to obtain the final confidence that the mask-removal behavior has occurred. This includes the following steps: The distance between the head and the mask is obtained by calculating the distance between the center points of each prediction frame. Distance between hands and mask and the distance between the hand and the head The probability of mask removal in an image is correlated with these distance values, using the minor axis distance of the detected head prediction boxes. As a reference distance, distance-based judgment rules are established to determine the actual increment in confidence that the act of removing a mask has occurred: Rule 1: When distance condition 1 is met: When the confidence level of removing the mask occurs, the actual increment is: ; Rule 2, when distance condition 2 is met: When the confidence level of removing the mask occurs, the actual increment is: ; Rule 3, when distance condition 3 is met: When the confidence level of removing the mask occurs, the actual increment is: ; Among them, behavioral weights The parameters are obtained through statistical analysis of the information labeled in the training samples of the prior knowledge dataset. First, the contribution of prior knowledge to the occurrence of mask removal is artificially assigned, and then... Normalization is performed when all of the above rules are met. After obtaining the normalized parameters Then, the parameters are validated and adjusted on the validation set of the prior knowledge dataset. , so that the parameters The detection of mask-removal behavior met expectations; By determining whether the above three distance conditions are met, the actual increase in confidence level for the mask-removal behavior is determined. This is then used to correct the confidence level obtained from the improved YOLOv5 network, resulting in the final confidence level for the mask-removal behavior. The final confidence level of the act of removing the mask Above the threshold If the mask is removed, it is determined that the act of removing the mask has occurred; otherwise, it is determined that the act of removing the mask has not occurred. Step 3. When the test result has no label and Then, it is necessary to re-determine whether the mask-removal behavior has occurred by analyzing the distance relationship between the targets related to the mask-removal behavior, determine the actual increment of the confidence that the mask-removal behavior has occurred, and then correct the confidence obtained from the improved YOLOv5 network to obtain the final confidence that the mask-removal behavior has occurred. ; If the final confidence level of the mask removal behavior occurs Above the threshold If the mask removal action occurs, it is determined that the mask removal action has occurred; otherwise, it is determined that the mask removal action has not occurred.

Images