A face mask recognition method, device, system and storage medium

Abstract

The application provides a face mask recognition method, device and system and a storage medium, and relates to the technical field of face recognition. The face mask recognition method comprises the following steps: constructing a neural network depth model; training the neural network depth model by using an image data set based on corresponding preset candidate boxes matched with the size of a target mask area in the neural network depth model; deploying the neural network depth model and performing recognition to output a recognition result. The neural network depth model is constructed based on a G-GhostNet convolution module, the model is trained by using an image data set, and corresponding preset candidate boxes matched with the size of a target mask area can be obtained in the neural network depth model. The model calculation parameters and the model size can be greatly reduced while the accuracy is improved, so that the model can be deployed in a mobile device in a lightweight manner.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and more specifically, to a method, device, system, and storage medium for facial mask recognition. Background Technology

[0002] In medical institutions, biomedical research and development units, and production workshops, as well as during outbreaks of Class A or Class B infectious diseases, and in financial service institutions serving large numbers of customers, wearing masks has become the norm. Existing facial recognition technology can only recognize faces when they are unobstructed. However, in scenarios where mask-wearing is mandatory, it is necessary to leverage the powerful feature extraction capabilities of deep learning and convolutional neural networks to reduce the operational complexity of image recognition technology and apply computer vision technology for facial mask recognition.

[0003] Manually detecting mask wearing in open, dynamic, and crowded environments, or relying solely on traditional target detection and tracking in complex scenarios, results in low accuracy and slow speed, and also fails to provide real-time, continuous alerts on mobile devices.

[0004] Deploying deep learning models and performing real-time data processing on mobile devices is a major trend in the development of mobile electronic devices. Traditional deep learning models suffer from high computational cost and energy consumption, making them unsuitable for mobile devices with limited computing power and energy storage. For example, AlexNet has 60 million parameters, occupying 228MB of memory on a float32 device, with its convolutional layers consuming 663MB of memory for FLOPS; VGG, on the other hand, has three times the number of parameters of AlexNet. To address the issues of high memory consumption and excessive system resources, methods such as compressing the model size and optimization are often employed, but this inevitably leads to a decrease in recognition accuracy.

[0005] In summary, facial recognition methods cannot accurately identify faces wearing masks, and the deep learning models used for recognition consume a lot of memory, resources, and have low efficiency. Summary of the Invention

[0006] In view of the above, and to address the aforementioned technical problems, this invention provides a face mask recognition method applicable to fintech or other related fields, comprising:

[0007] Construct a deep neural network model based on the G-GhostNet convolutional module;

[0008] Based on the preset candidate boxes corresponding to the size matching of the target mask area in the neural network deep model, the neural network deep model is trained using an image dataset;

[0009] The trained neural network deep model is deployed, and the real-time target image is identified using the neural network deep model, and the identification result is output.

[0010] Preferably, the step of training the neural network deep model using an image dataset based on the preset candidate boxes corresponding to the target mask region size matching in the neural network deep model includes:

[0011] The image dataset is normalized using the ImageNet database to adjust all training set images in the image dataset to the target resolution;

[0012] Based on the size of the target mask region in the training set image, a preset candidate box of corresponding size is matched so that the neural network deep model can be trained using the training set image based on the preset candidate box.

[0013] Preferably, the preset candidate boxes include a first candidate box, a second candidate box, and a third candidate box;

[0014] Wherein, the side length of the first candidate box is twice that of the second candidate box; the side length of the second candidate box is twice that of the third candidate box;

[0015] Each of the preset candidate boxes has 3 featureMap channels.

[0016] Preferably, the resolution size of the target resolution is 640×640;

[0017] The resolution of the first candidate box is 80×80.

[0018] The resolution of the second candidate box is 40×40.

[0019] The resolution size of the third candidate box is 20×20.

[0020] Preferably, training the neural network deep model using the image dataset includes:

[0021] For the training set images in the image dataset, initial cluster centers are randomly set within the range of 0-640;

[0022] Calculate the distance of all samples to the nearest cluster center until the cluster centers corresponding to the nine candidate boxes based on the featureMap channels of the preset candidate boxes are selected;

[0023] Preferably, the probability of a non-cluster center sample point being selected as the next cluster center is proportional to the distance it previously calculated.

[0024] Preferably, the deep features of the G-GhostNet convolutional module include Ghost features and complex features;

[0025] The training based on the neural network deep model further includes:

[0026] In the convolutional network of the deep neural network model employing the G-GhostNet convolutional module, the Ghost features are obtained through a linear transformation of the first convolutional block;

[0027] The features from the second convolutional block to the preset number of convolutional blocks are concatenated, and the mix transform function is used to obtain features in the same domain as the Ghost features, which are the complex features; wherein, the preset number of convolutional blocks exist in the neural network deep model using the G-GhostNet convolutional module;

[0028] The Ghost feature and the complex feature are added together and fused to output the feature result, so as to complete the integration of the original convolution.

[0029] Preferably, the training based on the neural network deep model further includes:

[0030] An attention mechanism is added before the mix transformation function, and the feature result is obtained based on the attention mechanism;

[0031] The feature results are output using the following function:

[0032]

[0033]

[0034]

[0035] In the neural network deep model employing the G-GhostNet convolutional module, there exists a predetermined number of convolutional blocks, where the predetermined number is n, and the output is represented as X∈R. c×h×w The complex features are: The Ghost characteristic is The attention mechanism comprises three tensors: Q, K, and V. Q represents the query vector, K represents the encoder data, and V represents the input to the attention mechanism. C represents the Cheap operation, which is either a 1×1 convolution or a 3×3 convolution. Y n This refers to the characteristic result.

[0036] Preferably, the image dataset of the neural network deep model includes training set images and test set images; the ratio of the number of training set images to the number of test set images is 9:1.

[0037] In the training of the neural network deep model, Avg IOU is used as the evaluation criterion.

[0038] In addition, to solve the above problems, the present invention also provides a face mask recognition device, comprising:

[0039] The building blocks are used to construct deep neural network models based on G-GhostNet convolutional modules;

[0040] The training module is used to train the neural network deep model using an image dataset based on the preset candidate boxes that match the size of the target mask region in the neural network deep model.

[0041] The output module is used to deploy the trained neural network deep model, and to identify real-time target images using the neural network deep model, and output the recognition results.

[0042] In addition, to solve the above problems, the present invention also provides a face mask recognition system, including a memory and a processor. The memory stores a face mask recognition program, and the processor runs the face mask recognition program to enable the face mask recognition system to perform the face mask recognition method as described above.

[0043] In addition, to solve the above problems, the present invention also provides a computer-readable storage medium storing a face mask recognition program, which, when executed by a processor, implements the face mask recognition method as described above.

[0044] This invention provides a method, device, system, and storage medium for facial mask recognition. The method includes: constructing a neural network deep model based on a G-GhostNet convolutional module; training the neural network deep model using an image dataset based on preset candidate boxes matching the size of the target mask region; deploying the trained neural network deep model; and recognizing a real-time target image using the neural network deep model, outputting the recognition result. This invention constructs a neural network deep model based on a G-GhostNet convolutional module and trains the model using an image dataset. The neural network deep model can identify preset candidate boxes matching the size of the target mask region, significantly reducing model computation parameters and model size while improving accuracy, thus enabling lightweight deployment of the model on mobile devices. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the hardware operating environment involved in an embodiment of the face mask recognition method of the present invention;

[0046] Figure 2 This is a flowchart illustrating the first embodiment of the face mask recognition method of the present invention;

[0047] Figure 3 This is a detailed flowchart of step S200 in the second embodiment of the face mask recognition method of the present invention;

[0048] Figure 4 This is a flowchart illustrating the refinement of step S300 (steps S310-S320) in the second embodiment of the face mask recognition method of the present invention.

[0049] Figure 5 This is a flowchart illustrating the refinement of step S300 (steps S330-S350) in the third embodiment of the face mask recognition method of the present invention.

[0050] Figure 6 This is a flowchart illustrating the method for obtaining a specified size feature map in a conventional original convolutional network model in the fourth embodiment of the face mask recognition method of the present invention.

[0051] Figure 7 This is a flowchart illustrating the method of adding and fusing the Ghost features and the complex features to output feature results during the training process of the deep neural network model in the fourth embodiment of the face mask recognition method of the present invention.

[0052] Figure 8 This is a schematic diagram of the process of adding an attention mechanism before the mix transformation function and obtaining the feature result based on the attention mechanism in the fourth embodiment of the face mask recognition method of the present invention.

[0053] Figure 9 This is a schematic diagram of the location distribution of the target to be detected and the length and width distribution of the sample images in the dataset in the fourth embodiment of the face mask recognition method of the present invention;

[0054] Figure 10 This is a schematic diagram of the overall workflow of the face mask recognition method in the fourth embodiment of the present invention;

[0055] Figure 11 This is a schematic diagram of the module connection of the face mask recognition device of the present invention.

[0056] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] The embodiments of the present invention are described in detail below, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0058] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0059] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0060] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0061] like Figure 1 The diagram shown is a structural schematic of the hardware operating environment of the terminal involved in an embodiment of the present invention.

[0062] The facial mask recognition system of this invention can be a PC, or a mobile terminal device such as a smartphone, tablet, or laptop. The facial mask recognition system may include: a processor 1001 (e.g., a CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, an input unit such as a keyboard, or a remote control; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a stable memory, such as a disk storage device. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001. Optionally, the facial mask recognition system may also include RF (Radio Frequency) circuitry, audio circuitry, a Wi-Fi module, etc. In addition, the facial mask recognition system can also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, which will not be elaborated here.

[0063] Those skilled in the art will understand that Figure 1The facial mask recognition system shown is not intended to limit it and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. Figure 1 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a data interface control program, a network connection program, and a face mask recognition program.

[0064] In summary, this invention constructs a neural network deep model based on the G-GhostNet convolutional module and trains the model using an image dataset. The neural network deep model can match the corresponding preset candidate boxes for the size of the target mask area, which can significantly reduce the model's computational parameters and size while improving accuracy, thus enabling the model to be deployed lightweight on mobile devices.

[0065] Example 1:

[0066] Reference Figure 2 The first embodiment of the present invention provides a face mask recognition method, comprising:

[0067] Step S100: Construct a deep neural network model based on the G-GhostNet convolutional module;

[0068] As described above, GhostNet is a lightweight convolutional neural network specifically designed for mobile applications. Its main component is the Ghost module, a novel plug-and-play module. The Ghost module is designed to generate more feature maps using fewer parameters.

[0069] Neural networks using G-GhostNet convolutional modules can solve the problems of large number of parameters and high computational cost in the original network. Furthermore, using lightweight G-GhostNet convolutional modules can retain redundancy at a lower cost.

[0070] Step S200: Based on the preset candidate boxes corresponding to the target mask area size matching in the neural network deep model, train the neural network deep model using the image dataset;

[0071] It should be noted that since target instances of the same category in an image dataset generally have similar aspect ratios, several candidate boxes with higher probabilities can be prepared in advance in the image dataset, and prediction can be made based on them, which are the preset candidate boxes.

[0072] During the training of a deep neural network model, the closer the size parameters of the preset candidate boxes are to the true bounding box of the target mask area, the easier the deep neural network model will be to train, and the more its predicted bounding box will match the true bounding box.

[0073] The quality of the design of the preset candidate box size parameters directly affects the speed of target recognition and the accuracy of the target box position.

[0074] Therefore, in this embodiment, multiple preset candidate frames can be set according to their size, corresponding to different target mask areas.

[0075] Step S300: Deploy the trained neural network deep model, and use the neural network deep model to identify the real-time target image and output the identification result.

[0076] The aforementioned trained deep neural network model can be deployed in the following environments:

[0077] The algorithm model was deployed and trained in a software training environment with Python 3.7.6, torch 1.12.1, CUDA acceleration version 11.3.1, NVIDIA GeForce RTX 3060 GPU with 6GB of GPU core memory, and Ubuntu 16.04 operating system. The training epoch was 200.

[0078] After deploying the aforementioned neural network deep model, the model is used in real time to recognize faces wearing masks. The target data is detected in real time through a camera, and the model inference is completed through a network cable connection to the local area network. The detection results are output, and a real-time judgment is given on whether a mask is being worn.

[0079] This embodiment constructs a neural network deep model based on the G-GhostNet convolutional module. The model is trained using an image dataset. The neural network deep model can match the corresponding preset candidate boxes for the size of the target mask area. This can significantly reduce the model's computational parameters and size while improving accuracy, thus enabling the model to be deployed lightweight on mobile devices.

[0080] Example 2:

[0081] Reference Figure 3 The second embodiment of the present invention provides a face mask recognition method based on the above embodiment 1. Step S200 involves training the neural network deep model using an image dataset, based on the preset candidate boxes corresponding to the target mask region size matching in the neural network deep model.

[0082] Step S210: Normalize the imageNet database of the image dataset to adjust all training set images in the image dataset to the target resolution;

[0083] The target resolution has a resolution size of 640×640;

[0084] The aforementioned preset candidate boxes are calculated based on the input bounding boxes. However, the non-uniqueness of the training input size affects training efficiency. To improve training efficiency, the ImageNet database is normalized so that the images in the input image dataset are adjusted to the same target resolution of 640×640, which also changes the size of the corresponding bounding boxes.

[0085] Step S220: Match a preset candidate box of the corresponding size according to the size of the target mask region in the training set image, so as to train the neural network deep model based on the training set image using the preset candidate box.

[0086] The preset candidate boxes include a first candidate box, a second candidate box, and a third candidate box;

[0087] Furthermore, the side length of the first candidate box is twice that of the second candidate box; the side length of the second candidate box is twice that of the third candidate box.

[0088] For example, the first candidate box has a resolution of 80×80; it detects smaller target mask areas; the second candidate box has a resolution of 40×40; it detects medium-sized target mask areas; and the third candidate box has a resolution of 20×20; it detects larger target mask areas.

[0089] Each candidate box in the preset candidate box has 3 feature map channels of different sizes; therefore, there are a total of 9 candidate boxes in the preset candidate box, namely, candidate boxes of 3 different resolution sizes, each candidate box has 3 feature map channels, for a total of 9, as shown in the table below:

[0090]

[0091] By using nine preset candidate boxes of different sizes and channels to identify the target mask area, the learning of positive and negative samples can be further accelerated.

[0092] Further reference Figure 4 In step S300, training the neural network deep model using the image dataset includes:

[0093] Step S310: For the training set images in the image dataset, randomly set initial cluster centers within the range of 0-640;

[0094] It should be noted that although the traditional K-Means clustering method will cluster the bounding boxes in the image dataset, many images in the dataset have similar resolution and size of the generated bounding boxes, resulting in nine closely spaced boxes, which is not conducive to model training. To improve this problem, this embodiment uses an improved K-Means algorithm to calculate the randomized center point, making it tend to the global optimum.

[0095] As described above, after adjusting the training set images to a uniform target resolution size, based on a resolution size of 640×640, during training, the improved K-Means algorithm is used to regress the anchor points in the images into suitable preset candidate boxes.

[0096] Step S320: Calculate the distance of all samples to the nearest cluster center until the cluster centers corresponding to the 9 candidate boxes based on the featureMap channel are selected.

[0097] Furthermore, the probability that a non-cluster center sample point is selected as the next cluster center is proportional to the distance it was previously calculated.

[0098] As mentioned above, during training, initial cluster centers can be randomly set between 0 and 640 in each training set image, and the distance of each sample to its nearest cluster center can be recorded.

[0099] In addition, to maximize the distance between the selected cluster centers, the probability of selecting any non-cluster center sample point as the next cluster center is proportional to the distance between the previous records, until nine cluster centers are selected.

[0100] In this embodiment, nine preset candidate boxes of different sizes across three channels with different resolutions are used to train the model on target mask regions of different sizes in the image dataset. This enables efficient and rapid detection of cross-scale targets and accelerates the learning of positive and negative samples. At the same time, an improved K-Means algorithm is used to regress feature anchor points into appropriate preset candidate boxes, further improving the training efficiency of the model.

[0101] Example 3:

[0102] Reference Figure 5 The third embodiment of the present invention provides a face mask recognition method. Based on the above embodiment 1, the deep features of the G-GhostNet convolutional module include Ghost features and complex features.

[0103] It should be noted that the excessive number of parameters in the network is a major reason for the difficulty in training deep neural network models.

[0104] To enable a more lightweight deep neural network model and to enhance the network's ability to understand features by utilizing redundant feature maps, this paper adopts the lightweight network model G-GhostNet. That is, the G-GhostNet convolutional module is used in the deep neural network model to replace the original convolutional module in the backbone of the original model, which can achieve the preservation of redundancy at a lower cost.

[0105] refer to Figure 6 In a conventional convolutional network model, the method for obtaining a feature map of a specified size in the output is as follows:

[0106] Generally, it involves two convolution processes (performed sequentially), followed by activation, pooling, and other steps, ultimately outputting a feature map of a specified size.

[0107] In this embodiment, reference Figure 7 By compressing the network ratio and using randomized weight initialization and weight sharing, the number of parameters in the network is reduced, increasing the network's generalization ability. During network model training, it can be trained to detect shapes and edges. The G-GhostNet convolutional module divides deep features into Ghost features and complex features. It integrates the original convolutions, and the output feature result includes the following three steps: (i.e., step S300, based on the training of the neural network deep model, also includes the following:)

[0108] Step S330: In the convolutional network of the neural network deep model employing the G-GhostNet convolutional module, the Ghost feature is obtained through a linear transformation of the first convolutional block.

[0109] Step S340 concatenates the features from the second convolutional block to the preset number of convolutional blocks, and uses the mix transform function to obtain features in the same domain as the Ghost features, which are the complex features; wherein, the preset number of convolutional blocks exist in the neural network deep model using the G-GhostNet convolutional module;

[0110] As mentioned above, the preset number can be n. Complex features are then generated through n convolutional blocks. Specifically, the intermediate features from layers 2 to n in the complex feature branch are concatenated, and the mix transform function is used to obtain features in the same domain as the Ghost features, i.e., the Ghost features.

[0111] The order of steps S330 and S340 is not limited; either step can be executed online or they can be executed simultaneously.

[0112] Step S350: Add and fuse the Ghost feature and the complex feature to output the feature result, so as to complete the integration of the original convolution.

[0113] The two features are added and fused together using the Concat step to obtain the current feature result after the original convolutional integration.

[0114] Compared to using the original convolutional network model, this embodiment switches the backbone network to a convolutional network of a neural network deep model using G-GhostNet convolutional modules. This can solve the problem of the original network having many parameters and high computational cost. Furthermore, using the lightweight G-GhostNet convolution can retain redundancy at a lower cost.

[0115] Further reference Figure 8 In step S300, the training based on the neural network deep model further includes:

[0116] An attention mechanism is added before the mix transformation function, and the feature result is obtained based on the attention mechanism;

[0117] The feature results are output using the following function:

[0118]

[0119]

[0120]

[0121] In the neural network deep model employing the G-GhostNet convolutional module, there exists a predetermined number of convolutional blocks, where the predetermined number is n, and the output is represented as X∈R. c×h×w The complex features are: The Ghost characteristic is The attention mechanism comprises three tensors: Q, K, and V. Q represents the query vector, K represents the encoder data, and V represents the input to the attention mechanism. C represents the Cheap operation, which is either a 1×1 convolution or a 3×3 convolution. Y n This refers to the characteristic result.

[0122] It should be noted that, since computing power remains a bottleneck limiting the development of neural networks, the corresponding models become more complex when there is too much input information. Neural networks, however, have the ability to focus attention on a portion of the input; by introducing attention, the amount of information processed can be reduced, thereby further reducing computing resources.

[0123] In this embodiment, wearing a mask and not wearing a mask are treated as a binary classification task, and a plug-and-play attention mechanism component is introduced into the G-GhostNet convolution module.

[0124] The experimental model diagram is shown in the figure. An attention mechanism is added before the mix transform function. The principle is as follows:

[0125] In each Ghost stage, the experiment generates features through the cheap operation to explore the redundancy between the first and last modules (the nth). Complex features are obtained through a large number of block (convolution) processes. An attention mechanism plugin is added to the complex features to enhance the expressive power of the cheap operation and provide rich information supplementation for the cheap operation.

[0126] Next, the information is aggregated, and Z is subjected to global mean pooling, then transformed into Y using a fully connected layer. n g Within the same domain, information is mixed.

[0127] The above definition, in a deep neural network model, refers to a stage containing n blocks, which is the preset number of convolutional blocks = n, and its output represents X∈R. c×h×w Representing complex features as Ghost characteristics are represented as

[0128] Complex features are enhanced through an attention mechanism; the attention mechanism module has three tensors: Q, K, and V. Q represents the query vector, K represents the area that attention should focus on, i.e., the encoder data, and V represents the input of the attention mechanism model.

[0129] The features are generated using the functions described above, where C represents a Cheap operation, which can be a 1×1 or 3×3 convolution. Combining these two types of features yields the feature result Y of the current stage's output. n .

[0130] In this embodiment, in the deep neural network model, based on the G-GhostNet convolution module, an attention mechanism is added to all G-GhostNet convolutions to enhance the expressive power of features and provide rich supplementary information.

[0131] Furthermore, the image dataset of the neural network deep model includes training set images and test set images; the ratio of the number of training set images to the number of test set images is 9:1.

[0132] In the training of the neural network deep model, Avg IOU is used as the evaluation criterion.

[0133] This model uses Avg IOU as the loss function to evaluate the quality of the candidate box localization parameters. The classification loss cls_loss, localization loss box_loss, and confidence loss obj_loss guide the regression task to obtain the optimal target frame.

[0134] Example 4:

[0135] To better illustrate the face mask recognition method provided in this embodiment, the following test experiment was conducted in this embodiment.

[0136] 1. The constructed neural network deep model is implemented using the PyTorch algorithm framework;

[0137] 2. The software training environment is as follows:

[0138] Python 3.7.6, torch 1.12.1, with CUDA acceleration version 11.3.1, NVIDIA GeForce RTX 3060 GPU, 6GB GPU core memory, Ubuntu 16.04 operating system;

[0139] 3. The hardware environment is as follows:

[0140] This includes a camera and a Jetson TX2. The trained model detects target data in real time through the camera. The target detection model is deployed on the Jetson TX2, and the embedded GPU development platform Xilin FPGA is connected to the camera via USB and connected to the local area network via Ethernet cable to complete model inference and output detection results.

[0141] 4. Image Dataset: The image dataset used in this experiment is a small-scale image dataset containing 2800 face images of people without masks and 2300 face images of people with masks. The image dataset annotation information includes: target type and location information. In order to ensure that the training data is as large as possible and the test set is universal, improve model performance, and prevent information leakage, the training set and the test set are divided in a ratio of 9:1. Among them, the training data images include the following labels: (1) No mask is labeled 0, (2) Mask is labeled 1;

[0142] refer to Figure 9 The distribution of the target location and the aspect ratio of the sample images in the dataset are as follows: most of the target objects are located in the center of the image, but some target objects in other images are located in other positions, which is beneficial to the generalization ability of the model. In addition, the aspect ratios of the training images are different, so the images need to be normalized.

[0143] The problem of mask wearing recognition has many small targets in practical applications. Due to the inconsistent image sizes, increasing the input size can reduce the cropping of large images.

[0144] In this embodiment, the input size of the data is adjusted to a resolution size of 640×640, which can improve the recognition accuracy of small targets such as masks to a certain extent.

[0145] Furthermore, refer to Figure 8 Nine pre-set candidate boxes of varying sizes are used. To ensure the model can efficiently and quickly detect targets across scales while also ensuring that the model's anchors conform to the normalized size, this model uses:

[0146] (1) Small targets are detected using the first candidate bounding box of 80×80.

[0147] (2) The second candidate box of 40×40 detects medium-sized targets.

[0148] (3) Detecting large targets using the third candidate box of 20×20;

[0149] The featureMap is further divided into three channels of different sizes, with a total of nine candidate boxes of varying sizes, which accelerates the learning of positive and negative samples.

[0150] 5. Training methods: Refer to Figure 10 The face mask recognition method provided in Examples 1-3 was used for training.

[0151] 6. Test results:

[0152] (1) In this embodiment, nine preset candidate boxes of different sizes and channels in the face mask recognition method are used. Compared with the ratio of the intersection and union of the predicted rectangle and the real target before and after the improvement, Avg IOU of the improved K-Means is closer to 1, which means that the candidate box parameters obtained by the improved K-Means are better.

[0153] (2) The improved mask recognition method significantly improves the accuracy of mask recognition, reaching 1-2%; the recognition accuracy for both wearing and not wearing masks is greatly improved, with the recognition accuracy for not wearing a mask increasing from 92.1% to 96%, and the recognition accuracy for wearing a mask increasing from 94% to nearly 98.6%.

[0154] (3) Latency was tested on an NVIDIA GeForce RTX 3060 Laptop GPU with 6GB of RAM and 8 batch sizes. Compared to the original Yolov5 model, the model with the G-GhostNet backbone improved mAP@.5 from 91.5% to 93.8% and mAP@.5:.95 from 50.7% to 52.4%. The number of parameters also decreased significantly, by almost two-thirds, to 6.4M. The number of floating-point operations per second remained almost unchanged. The model memory usage decreased by half to 8.4M, and the latency decreased by 1.3ms, which may be related to the reduction in the number of convolutions. After adding attention mechanisms, compared to not adding the model, the number of parameters and model size remained basically unchanged, and the accuracy improved by 0.6%-3.8% for mAP@.5:.95. However, the time consumption increased, with latency increasing by 1.3ms-9.5ms. Among them, the CA attention mechanism achieved the highest accuracy, with 95.6% for mAP@.5 and 56.2% for mAP@.5:.95. The SE attention mechanism had the lowest time consumption at 8.7ms, ranking third in accuracy. Adding ECA increased the latency by 3.8ms, significantly lower than the 17ms latency after adding CBAM, and the accuracy after adding ECA ranked second among the four attention mechanisms. It is evident that SE has the least impact on model latency, while the CA attention mechanism has the best recognition performance. SE and ECA strike a good balance between speed and accuracy.

[0155] (4) In real-world scenarios, comparing the results of mask detection, the addition of the attention mechanism significantly improved the recognition of both masked and unmasked scenes. The best result was achieved with CA (Cognitive Alignment), resulting in a recognition probability of 0.40-0.70 for unmasked scenes and 0.62-0.81 for masked scenes. SE (Self-Alignment) and ECA (Electronic Alignment) showed slightly better results. In actual testing using the pure G-GhostNet skeleton model, one unmasked scene was not detected despite bright lighting. However, the attention mechanism enabled its detection, indicating that it improved the utilization of effective redundant features, focusing target detection on the mouth area.

[0156] In addition, refer to Figure 11 This embodiment also provides a face mask recognition device, including:

[0157] Module 10 is used to build a neural network deep model based on the G-GhostNet convolution module;

[0158] Training module 20 is used to train the neural network deep model using an image dataset based on the preset candidate boxes corresponding to the size matching of the target mask region in the neural network deep model.

[0159] The output module 30 is used to deploy the trained neural network deep model, and to identify real-time target images through the neural network deep model, and output the recognition results.

[0160] In addition, this embodiment also provides a face mask recognition system, including a memory and a processor. The memory stores a face mask recognition program, and the processor runs the face mask recognition program to enable the face mask recognition system to perform the face mask recognition method as described above.

[0161] In addition, this embodiment also provides a computer-readable storage medium storing a face mask recognition program, which, when executed by a processor, implements the face mask recognition method as described above.

[0162] In summary, this invention constructs a neural network deep model based on the G-GhostNet convolutional module and trains the model using an image dataset. The neural network deep model can match the corresponding preset candidate boxes for the size of the target mask area, which can significantly reduce the model's computational parameters and size while improving accuracy, thus enabling the model to be deployed lightweight on mobile devices.

[0163] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0164] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention. The above are only preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for facial mask recognition, characterized in that, include: Construct a deep neural network model based on the G-GhostNet convolutional module; Based on the preset candidate boxes corresponding to the target mask region size matching in the neural network deep model, the neural network deep model is trained using an image dataset; the deep features of the G-GhostNet convolutional module include Ghost features and complex features; The step of training the neural network deep model using an image dataset, based on the preset candidate boxes corresponding to the target mask region size matching in the neural network deep model, further includes: obtaining the Ghost feature through a linear transformation of the first convolutional block in the convolutional network of the neural network deep model using the G-GhostNet convolutional module; concatenating the features from the second convolutional block to the preset number of convolutional blocks, and using a mix transformation function to obtain features in the same domain as the Ghost feature, which is the complex feature; wherein, there are a preset number of convolutional blocks in the neural network deep model using the G-GhostNet convolutional module; adding and fusing the Ghost feature and the complex feature to output the feature result, so as to complete the integration of the original convolution; the step of training the neural network deep model using an image dataset based on the preset candidate boxes corresponding to the target mask region size matching in the neural network deep model further includes: adding an attention mechanism before the mix transformation function, and obtaining the feature result based on the attention mechanism; the feature result is output through the following function: ； ； ； Wherein, the preset quantity is n, and its output is expressed as The complex features are: The Ghost feature is: The attention mechanism comprises three tensors: Q, K, and V. Q represents the query vector, K represents the encoder data, and V represents the input of the attention mechanism. C represents the Cheap operation, which is a 1×1 convolution or a 3×3 convolution. The feature result; The trained neural network deep model is deployed, and the real-time target image is identified using the neural network deep model, and the identification result is output.

2. The face mask recognition method as described in claim 1, characterized in that, The step of training the neural network deep model using an image dataset, based on the preset candidate boxes corresponding to the target mask region size matching in the neural network deep model, includes: The image dataset is normalized using the ImageNet database to adjust all training set images in the image dataset to the target resolution; Based on the size of the target mask region in the training set image, a preset candidate box of corresponding size is matched so that the neural network deep model can be trained using the training set image based on the preset candidate box.

3. The face mask recognition method as described in claim 2, characterized in that, The preset candidate boxes include a first candidate box, a second candidate box, and a third candidate box; Wherein, the side length of the first candidate box is twice that of the second candidate box; the side length of the second candidate box is twice that of the third candidate box; Each of the preset candidate boxes has 3 featureMap channels.

4. The face mask recognition method as described in claim 3, characterized in that, The target resolution has a resolution size of 640×640; The resolution of the first candidate box is 80×80. The resolution of the second candidate box is 40×40. The resolution size of the third candidate box is 20×20.

5. The face mask recognition method as described in claim 3, characterized in that, The training of the neural network deep model using the image dataset includes: For the training set images in the image dataset, initial cluster centers are randomly set within the range of 0-640; Calculate the distance of all samples to the nearest cluster center until the cluster centers corresponding to the nine candidate boxes based on the featureMap channels of the preset candidate boxes are selected.

6. The face mask recognition method as described in claim 5, characterized in that, The probability that a non-cluster center sample point is selected as the next cluster center is proportional to the distance it was previously calculated.

7. The face mask recognition method as described in claim 1, characterized in that, The image dataset of the neural network deep model includes training set images and test set images; the ratio of the number of training set images to the number of test set images is 9:1; In the training of the neural network deep model, Avg IOU is used as the evaluation criterion.

8. A face mask recognition device, characterized in that, include: The building blocks are used to construct deep neural network models based on G-GhostNet convolutional modules; The training module is used to train the neural network deep model using an image dataset based on the corresponding preset candidate boxes that match the size of the target mask region in the neural network deep model; the deep features of the G-GhostNet convolution module include Ghost features and complex features; The step of training the neural network deep model using an image dataset, based on the preset candidate boxes corresponding to the target mask region size matching in the neural network deep model, further includes: obtaining the Ghost feature through a linear transformation of the first convolutional block in the convolutional network of the neural network deep model using the G-GhostNet convolutional module; concatenating the features from the second convolutional block to the preset number of convolutional blocks, and using a mix transformation function to obtain features in the same domain as the Ghost feature, which is the complex feature; wherein, there are a preset number of convolutional blocks in the neural network deep model using the G-GhostNet convolutional module; adding and fusing the Ghost feature and the complex feature to output the feature result, so as to complete the integration of the original convolution; the step of training the neural network deep model using an image dataset based on the preset candidate boxes corresponding to the target mask region size matching in the neural network deep model further includes: adding an attention mechanism before the mix transformation function, and obtaining the feature result based on the attention mechanism; the feature result is output through the following function: ； ； ； Wherein, the preset quantity is n, and its output is expressed as The complex features are: The Ghost feature is: The attention mechanism comprises three tensors: Q, K, and V. Q represents the query vector, K represents the encoder data, and V represents the input of the attention mechanism. C represents the Cheap operation, which is a 1×1 convolution or a 3×3 convolution. The feature result; The output module is used to deploy the trained neural network deep model, and to identify real-time target images using the neural network deep model, and output the recognition results.

9. A facial mask recognition system, characterized in that, The system includes a memory and a processor, wherein the memory stores a face mask recognition program, and the processor runs the face mask recognition program to enable the face mask recognition system to perform the face mask recognition method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a face mask recognition program, which, when executed by a processor, implements the face mask recognition method as described in any one of claims 1-7.

Images