Training method, device, equipment and storage medium for mask recognition model

Abstract

The present invention provides a training method, device, equipment and storage medium for a mask recognition model. The method includes: cutting each picture in the first training set into a plurality of image blocks of the same size, and labeling the image blocks to obtain a second training set; cutting each picture in the third training set into a plurality of image blocks of the same size, and labeling the image blocks to obtain a fourth training set; using the second training set to pre-train the first preset model to obtain a pre-trained model; using the fourth training set and the second preset model to perform formal training to obtain a mask recognition model; wherein the first preset model and the second preset model can adopt a YOLOV4-tiny network model, and the YOLOV4-tiny network model includes a backbone network and a remaining network; the parameters of the backbone network of the model in the pre-training and formal training are frozen. The method improves the running speed of the model and the accuracy of recognizing small targets at a long distance.

Description

Technical Field

[0001] The present invention relates to the field of new generation information technology, and in particular to a training method, device, equipment and storage medium for a mask recognition model. Background Art

[0002] With the development of computer vision technology, using computer vision technology to detect faces from images and identify whether they are wearing masks has very important research significance and application value.

[0003] At present, neural networks are mostly used for mask recognition, and many security monitoring systems have achieved mask recognition through system upgrades. For example, a multi-task convolutional neural network (MTCNN) is used as a network model for mask wearing recognition, marking the region of interest (ROI) on the spectral image, obtaining coordinates and category information, training the support vector machine (SVM) classifier, and then classifying and judging whether a mask is worn.

[0004] The existing recognition methods have a large number of model parameters, resulting in slow operation speed and long recognition cycle, and are unable to accurately recognize small targets at a long distance. Summary of the invention

[0005] The present invention provides a training method, device, equipment and storage medium for a mask recognition model, which are used to improve the accuracy of recognizing small targets at a long distance, and solve the problems of slow running speed and long recognition cycle caused by a large number of model parameters.

[0006] In a first aspect, an embodiment of the present invention provides a method for training a mask recognition model, comprising:

[0007] Cut each picture in the first training set into multiple image blocks, and label the image blocks to obtain a second training set, where the first training set is a first face mask data set, and the labels of the image blocks in the second training set include position information of the image blocks in the picture and category information of whether the face in the image block wears a mask;

[0008] Cut each picture in the third training set into multiple image blocks, and label the image blocks to obtain a fourth training set, where the third training set is a second face mask data set, and the labels of the image blocks in the fourth training set include position information of the image blocks in the picture and category information of whether the face in the image block wears a mask;

[0009] Pre-training a first preset model using the second training set to obtain a pre-trained model, wherein the first preset model includes a backbone network and a remaining network, and parameters of the backbone network of the first preset model are frozen during the pre-training process;

[0010] The fourth training set and the second preset model are used for formal training to obtain a mask recognition model, wherein the second preset model includes a backbone network and a remaining network, the parameters of the backbone network of the second training model are the parameters of the backbone network of the pre-trained model, the initial parameters of the remaining network of the second preset model are the parameters of the remaining network of the pre-trained model, and the parameters of the backbone network of the second preset model are frozen during the formal training process.

[0011] In a possible implementation, the first preset model adopts a YOLOV4-tiny network model, and the first preset model is pre-trained using the second training set to obtain a pre-trained model, including:

[0012] Loading the parameters obtained by training on the ImageNet dataset into the backbone network of the first preset model;

[0013] After the loading is completed, freezing the backbone network of the first preset model;

[0014] The parameters of the remaining network of the first preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the pre-trained model:

[0015] Inputting a first number of image blocks in the second training set into the first preset model for training each time to obtain a training result;

[0016] Determine the YOLO loss value based on the label of the input image block and the training result of the input image block;

[0017] Perform back propagation according to the YOLO loss value to obtain update parameters of the remaining network of the first preset model;

[0018] The update parameters are used to update the parameters of the remaining network of the first preset model.

[0019] In a possible implementation, the second preset model adopts a YOLOV4-tiny network model, and the fourth training set and the second preset model are used for formal training to obtain a mask recognition model, including:

[0020] Loading the parameters of the backbone network of the pre-trained model onto the backbone network of the second preset model;

[0021] After the loading is completed, freezing the backbone network of the second preset model;

[0022] Loading the parameters of the remaining network of the pre-trained model onto the remaining network of the second preset model;

[0023] The parameters of the remaining network of the second preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the mask recognition model:

[0024] Inputting a first number of image blocks in the fourth training set into the second preset model for training each time to obtain a training result;

[0025] Determine the YOLO loss value based on the label of the input image block and the training result of the input image block;

[0026] Back propagation is performed according to the YOLO loss value to obtain update parameters of the remaining network of the preset model;

[0027] The update parameters are used to update the parameters of the remaining network of the second preset model.

[0028] In a possible implementation, cutting each picture in the first training set and the third training set into a plurality of image blocks respectively includes:

[0029] Starting from any corner of the picture, sliding the image frame with preset pixels, cutting the image in the image frame to form the image block, wherein the size of the preset pixels is smaller than the length and width of the image frame;

[0030] The coordinates of the frame of the image block are obtained according to the coordinates of the picture frame marked with the picture and the size of the sliding image frame.

[0031] In a possible implementation manner, before cutting each picture in the first training set and the third training set into a plurality of image blocks, the method further includes:

[0032] Data enhancement processing is performed on the images in the first training set and the third training set, and the data enhancement processing includes one or more of the following processing: randomly adjusting the image size, randomly adjusting the image contrast, randomly adjusting the image hue, randomly adjusting the image brightness, randomly adding noise to the image, randomly changing the image color model, and randomly cropping the image.

[0033] In a possible implementation, the size of the image block is 416*416 pixels, and the backbone network includes 6 cross-stage partial CSP networks connected in series, each of which is used to extract features from the input image;

[0034] The target CSP network in the backbone network is connected to the input of the CAT module, the output of the CAT module is connected to the remaining network, the feature map extracted by the target CSP network is 26*26 pixels and 13*13 pixels, and the CAT module is used to connect the feature map extracted by the target CSP network.

[0035] In a possible implementation, the proportion of face targets in some images in the first face mask data set is greater than a preset threshold, and the proportion of face targets in the remaining images is less than or equal to the preset threshold;

[0036] The proportion of facial targets included in each image in the second face mask data set is greater than the preset threshold.

[0037] In a second aspect, an embodiment of the present invention provides a mask recognition method, which is applied to a mask recognition model trained by the method of the first aspect, and the method includes:

[0038] Cutting a picture to be identified into a plurality of image blocks, and determining position information of each image block in the picture to be identified, wherein the picture to be identified contains at least one face target;

[0039] Inputting the multiple image blocks into the mask recognition model to obtain a first recognition result for each image block, wherein the first recognition result is used to indicate whether the face in the image block wears a mask;

[0040] When the same face target in the image to be identified exists in different image blocks, the confidence of each image block where the face target is located is calculated according to the position information of each image block where the face target is located in the image to be identified and the face target detection frame in each image block, and the first recognition result of the image block with the largest confidence is selected as the recognition result of the face target;

[0041] Output the recognition result of the face target in the image to be recognized.

[0042] In a possible implementation, calculating the confidence of each image block where the face target is located according to the position information of each image block where the face target is located in the picture to be identified and the face target detection frame in each image block includes:

[0043] Restoring each image block to the picture to be identified according to position information of each image block where the face target is located in the picture to be identified;

[0044] For each image block, calculating the ratio of the face target detection frame in the image block to the face target detection frame in the image to be identified;

[0045] The calculated ratio is used as the confidence of the image block.

[0046] In a third aspect, an embodiment of the present invention provides a training device for a mask recognition model, comprising:

[0047] A first cutting module is used to cut each image in the first training set into multiple image blocks, and label the image blocks to obtain a second training set, wherein the first training set is a first face mask data set, and the labels of the image blocks in the second training set include position information of the image blocks in the image and category information of whether the face in the image block wears a mask;

[0048] a second cutting module, used to cut each picture in the third training set into multiple image blocks, and label the image blocks to obtain a fourth training set, wherein the third training set is a second face mask data set, and the labels of the image blocks in the fourth training set include position information of the image blocks in the picture and category information of whether the face in the image block wears a mask;

[0049] A pre-training module, used to pre-train a first preset model using the second training set to obtain a pre-trained model, wherein the first preset model includes a backbone network and a remaining network, and parameters of the backbone network of the first preset model are frozen during the pre-training process;

[0050] A formal training module is used to perform formal training using the fourth training set and the second preset model to obtain a mask recognition model, wherein the second preset model includes a backbone network and a remaining network, the parameters of the backbone network of the second training model are the parameters of the backbone network of the pre-trained model, the initial parameters of the remaining network of the second preset model are the parameters of the remaining network of the pre-trained model, and the parameters of the backbone network of the second preset model are frozen during the formal training process.

[0051] In a possible implementation, the first preset model adopts a YOLOV4-tiny network model, and the pre-training module is specifically used for:

[0052] Loading the parameters obtained by training on the ImageNet dataset into the backbone network of the first preset model;

[0053] After the loading is completed, freezing the backbone network of the first preset model;

[0054] The parameters of the remaining network of the first preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the pre-trained model:

[0055] Inputting a first number of image blocks in the second training set into the first preset model for training each time to obtain a training result;

[0056] Determine the YOLO loss value based on the label of the input image block and the training result of the input image block;

[0057] Perform back propagation according to the YOLO loss value to obtain update parameters of the remaining network of the first preset model;

[0058] The update parameters are used to update the parameters of the remaining network of the first preset model.

[0059] In a possible implementation, the second preset model adopts a YOLOV4-tiny network model, and the formal training module is specifically used for:

[0060] Loading the parameters of the backbone network of the pre-trained model onto the backbone network of the second preset model;

[0061] After the loading is completed, freezing the backbone network of the second preset model;

[0062] Loading the parameters of the remaining network of the pre-trained model onto the remaining network of the second preset model;

[0063] The parameters of the remaining network of the second preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the mask recognition model:

[0064] Inputting a first number of image blocks in the fourth training set into the second preset model for training each time to obtain a training result;

[0065] Determine the YOLO loss value based on the label of the input image block and the training result of the input image block;

[0066] Back propagation is performed according to the YOLO loss value to obtain update parameters of the remaining network of the preset model;

[0067] The update parameters are used to update the parameters of the remaining network of the second preset model.

[0068] In a possible implementation, the first cutting module and the second cutting module are specifically used to:

[0069] Starting from any corner of the picture, sliding the image frame with preset pixels, cutting the image in the image frame to form the image block, wherein the size of the preset pixels is smaller than the length and width of the image frame;

[0070] The coordinates of the frame of the image block are obtained according to the picture frame coordinates marked on the picture and the size of the sliding image frame.

[0071] In a possible implementation, the first cutting module and the second cutting module further include:

[0072] An enhancement unit is used to perform data enhancement processing on the images in the first training set and the third training set, wherein the data enhancement processing includes one or more of the following processing: randomly adjusting the image size, randomly adjusting the image contrast, randomly adjusting the image hue, randomly adjusting the image brightness, randomly adding noise to the image, randomly changing the image color model, and randomly cropping the image.

[0073] In a possible implementation, the size of the image block in the first cutting module or the second cutting module is 416*416 pixels, and the backbone network in the pre-training module or the formal training module includes 6 cross-stage partial CSP networks connected in series, and each of the CSP networks is used to extract features from the input image;

[0074] The target CSP network in the backbone network is connected to the input of the CAT module, the output of the CAT module is connected to the remaining network, the feature map extracted by the target CSP network is 26*26 pixels and 13*13 pixels, and the CAT module is used to connect the feature map extracted by the target CSP network.

[0075] In a possible implementation, the proportion of face targets in some images in the first face mask data set is greater than a preset threshold, and the proportion of face targets in the remaining images is less than or equal to the preset threshold;

[0076] The proportion of facial targets included in each image in the second face mask data set is greater than the preset threshold.

[0077] In a fourth aspect, a mask recognition device is provided, the device comprising:

[0078] A cutting module, used for cutting the image to be identified into a plurality of image blocks, and determining the position information of each image block in the image to be identified, wherein the image to be identified contains at least one human face target;

[0079] An input module, used to input the multiple image blocks into the mask recognition model to obtain a first recognition result for each image block, wherein the first recognition result is used to indicate whether the face in the image block wears a mask;

[0080] A calculation module, when the same face target in the to-be-recognized picture exists in different image blocks, calculates the confidence of each image block where the face target is located according to the position information of each image block where the face target is located in the to-be-recognized picture and the face target detection frame in each image block, and selects the first recognition result of the image block with the largest confidence as the recognition result of the face target;

[0081] The output module is used to output the recognition result of the face target in the image to be recognized.

[0082] In a possible implementation, the calculation module is specifically used to:

[0083] Restoring each image block to the picture to be identified according to position information of each image block where the face target is located in the picture to be identified;

[0084] For each image block, calculating the ratio of the face target detection frame in the image block to the face target detection frame in the image to be identified;

[0085] The calculated ratio is used as the confidence of the image block.

[0086] In a fifth aspect, an electronic device for mask recognition includes:

[0087] at least one processor;

[0088] and a memory communicatively coupled to the at least one processor;

[0089] Among them, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method of the mask recognition model provided in the first aspect of the present invention.

[0090] In a sixth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the training method for the mask model provided in the first aspect of the present invention is implemented.

[0091] The present invention provides a method and device for training a mask recognition model, which reduces the number of model parameters by designing a model structure and improving processing of small targets in training set image blocks, thereby accelerating the running speed, shortening the recognition cycle, and improving the accuracy of recognizing small targets at a long distance. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0093] Figure 1 It is a schematic diagram of the structure of the YOLOV4-tiny network model of the present invention;

[0094] Figure 2 It is a schematic diagram of the structure of the CSP network in the YOLOV4-tiny network model of the present invention;

[0095] Figure 3 A flowchart of a method for training a mask recognition model provided in Embodiment 1 of the present invention;

[0096] Figure 4 A schematic diagram for image cutting;

[0097] Figure 5 is a schematic diagram of image block position information;

[0098] Figure 6 A flowchart of a pre-training method for a mask recognition model provided in Embodiment 2 of the present invention;

[0099] Figure 7 A flowchart of a formal training method for a mouth recognition model provided in Embodiment 3 of the present invention;

[0100] Figure 8 A flowchart of a mask recognition method provided in Embodiment 4 of the present invention;

[0101] Fig. 9 A schematic diagram of the structure of a mask recognition model training device provided in Embodiment 5 of the present invention;

[0102] Fig.10 A structural schematic diagram of a mask recognition device provided in Embodiment 6 of the present invention;

[0103] Fig.11 This is a structural schematic diagram of a mask recognition device provided in Example 7 of the present invention.

[0104] The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0105] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0106] The embodiment of the present invention provides a training method and a use method of a mask recognition model, wherein the mask recognition model is used to recognize whether a face is wearing a mask. The mask recognition model can adopt a YOLOV4-tiny network model. Figure 1 Schematic diagram of the structure of the YOLOV4-tiny network model of the present invention. Figure 1 As shown in the figure, the YOLOV4-tiny network model is mainly composed of a backbone network 107, a Concat connection module 108 and a remaining partial network 109. The backbone network 107 is a series of six cross-stage partial networks (CSP networks for short), as shown in 101 to 106 in the figure. Each CSP network is used to extract features from the input image block. The size of the feature map extracted by each CSP network is different. According to the series connection order, the feature map extracted by the CSP network gradually decreases. Figure 1 In the structure shown, the last two levels of CSP networks are connected to the input end of the Concat connection module 108, and the target feature maps extracted by the two levels of CSP networks enter the remaining network 109 through the output end of the Concat connection module 108. The remaining network 109 is a convolution layer that performs convolution processing on the input target feature map.

[0107] Exemplarily, when an image block of 416*416 pixels is input, the feature maps extracted in sequence by the six cross-stage partial CSP networks connected in series through the backbone network are 208*208 pixels, 104*104 pixels, 52*52 pixels, 26*26 pixels and 13*13 pixels. The feature maps of 26*26 pixels and 13*13 pixels are input into the remaining network through the Concat connection module. After the image block is processed by the remaining network, the recognition result of the image block is finally output.

[0108] The present invention adopts the YOLOV4-tiny network model to identify whether the face target in the image to be identified is wearing a mask. The network model improves the running speed of the model by using multiple CSP networks with a cross-hop connection structure.

[0109] Combine the following Figure 2 The CSP network structure in the YOLOV4-tiny network model is described in detail.

[0110] Figure 2 Schematic diagram of the structure of the CSP network in the YOLOV4-tiny network model of the present invention. Figure 2 As shown in the figure, the CSP network consists of an input module 201, an output module 206, convolutional layers 202, 203, 204, 205, and CAT modules 25 and 26. Among them, the convolution kernel size of convolutional layer 202, convolutional layer 203 and convolutional layer 204 is 3×3, and the convolution kernel size of convolutional layer 205 is 1×1. The function of CAT modules 25 and 26 is to connect two arrays without changing the array characteristics. The CSP network contains two skip connection structures, as shown in 207 and 208 in the figure, which are used to divide the input feature map into two on the channel. After the input feature map passes through the skip connection structure, only the feature map on one channel needs to be convolved, thereby reducing the calculation amount of the model and speeding up the computer operation.

[0111] Specifically, Figure 2 As shown, the feature map output by the convolution layer 202 is divided into path 21 and path 22 in the channel when passing through the skip connection structure 207. The feature map output by path 21 is directly connected with the feature map output by the convolution layer 205 in the CAT module 26 and enters the output module 206. The feature map output by path 22 enters the convolution layer 203. The feature map output by the convolution layer 203 after convolution calculation is divided into path 23 and path 24 in the channel when passing through the skip connection structure 208. The feature map output by path 24 enters the convolution layer 204. The feature map output by path 23 is connected with the feature map output by the convolution layer 204 in the CAT module 25 and enters the convolution layer 205.

[0112] Exemplarily, the input module 201 inputs a feature map of 64@104*104 pixels, enters the first convolution layer 202, performs convolution calculation processing on the feature map by the convolution function Conv3×3, and outputs the feature map. When the output feature map passes through the skip connection structure 207, the feature map is converted into a channel. Figure 1 The feature map is divided into two so that the channels of the feature map are halved. Then one part of the feature map with 32@104*104 pixels enters the CAT module 26 via path 21, and the other part of the feature map with 32@104*104 pixels enters the convolution layer 203 via path 22. The feature map is processed by the convolution function Conv3×3 and then output. When the output feature map passes through the skip connection structure 208, the feature map is converted on the channel. Figure 1The feature map is divided into two so that the channels of the feature map are halved. Then, a part of the feature map with 16@104*104 pixels enters the CAT module 25 via path 23, and the other part of the feature map with 16@104*104 pixels enters the convolution layer 204 via path 24. The feature map is processed by convolution function Conv3×3 and then output. The output feature map enters the CAT module 25 and is combined with the feature map input via path 23. The feature map is then input into the convolution layer 205. The feature map is processed by convolution function Conv1×1 and then input into the CAT module 26. The feature map is combined with the feature map input via path 21 in the CAT module 26, and then the feature map is output via output module 206.

[0113] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the invention will be described below in conjunction with the accompanying drawings.

[0114] It should be noted that the network model used in the following specific embodiments adopts Figure 1 The YOLOV4-tiny network model shown.

[0115] Before using the YOLOV4-tiny network model to identify whether the face target in the test image is wearing a mask, it is necessary to train the parameters in the YOLOV4-tiny network model to obtain a YOLOV4-tiny network model with optimal recognition efficiency. Figure 3 The training of the YOLOV4-tiny network model is explained in detail.

[0116] Figure 3 This is a flow chart of a method for training a mask recognition model provided in Embodiment 1 of the present invention. Figure 3 As shown, the training method of the mask recognition model includes the following steps.

[0117] S301, cutting each image in the first training set into a plurality of image blocks, and labeling the image blocks to obtain a second training set.

[0118] The first training set is a first face mask data set. Specifically, the proportion of face targets in some pictures in the first face mask data set is greater than a preset threshold, and the proportion of face targets in the remaining pictures is less than or equal to the preset threshold. Exemplarily, the data set can be a mask face data set (FaceMask_CelebA), or it can be one or more of a real mask face recognition data set, a simulated mask face recognition data set, and a real mask face verification data set in a mask-occluded face data set (Real-Word Masked Face Dataset, RMFD for short).

[0119] Specifically, the data set contains a large number of images and labels of the images, and the labels are category information of whether the face is wearing a mask. For example, when the face in the image is wearing a mask, its category information is 1, and when the face is not wearing a mask, its category information is 0. Of course, the category information can also be represented in other forms, which is not limited in this embodiment.

[0120] Each image in the first training set is cut into multiple image blocks, and the sizes of the image blocks can be the same or different, which is not limited in this embodiment. Labels are annotated on the cut image blocks, and specifically, the labels include the position information of the image blocks in each image and the category information of whether the face in the image block wears a mask.

[0121] The current mask wearing recognition method has poor detection effect on small targets at a long distance. In order to improve the detection effect of small targets at a long distance, the present invention adopts a method of first magnifying and then detecting, that is, cutting each picture in the training set into multiple image blocks, and then detecting the image blocks.

[0122] Figure 4 This is a schematic diagram of image cutting. Figure 4 As shown, a certain size image frame 40 is used to slide on the picture in sequence according to preset pixels, and the area in each image frame is cut off to obtain an image block, wherein the size of the preset pixel is smaller than the length and width of the certain size image frame. For example, the size of the image frame 40 can be 416*416, and the preset pixel size can be 300.

[0123] like Figure 4 As shown, in a possible implementation, starting from the upper left corner of the picture, a fixed-size image frame is used to slide from left to right in sequence according to preset pixels, and the image in the image frame is cut to form multiple image blocks. When the image frame slides to the rightmost end, the image frame is slid down by one step according to the sliding step length, and then, it is slid from left to right or from right to left in sequence according to the preset pixels, and the image block is cut in the above manner. Among them, the sliding step length is the size of multiple pixels, and the sliding step length and the preset pixels can be the same or different.

[0124] In another possible implementation, starting from the upper left corner of the image, a frame of a certain size is used to slide from top to bottom on the image in sequence according to the size of preset pixels, and the image in the image frame is cut to form multiple image blocks. When the image frame slides to the bottom, the image frame is slid to the right by one step according to the sliding step length, and then, the image frame is slid from top to bottom or from bottom to top in sequence according to the preset pixels, and the image block is cut in the above manner. The sliding step length is the size of multiple pixels, and the sliding step length and the preset pixels can be the same or different.

[0125] The cut image blocks are labeled to obtain a second training set. The labels of the image blocks in the second training set include position information of the image blocks in the picture and category information of whether the face in the image blocks is wearing a mask. The category information of the image blocks is the same as the category information of the picture before cutting, that is, the category information of the face in the picture before cutting is wearing a mask, then the category information of the face in each image block after cutting is also wearing a mask. Among them, after the cutting is completed, some image blocks may not have faces, then the category information of the image blocks without faces is not wearing masks.

[0126] The position information of the image block may be the position information of the cutting point of the image block in the picture, and may be obtained according to the coordinates of the picture before cutting. Figure 5 is a schematic diagram of the image block position. Figure 5 As shown, point A (0, 0) of the image is defined as the coordinate origin, and point B and point C are the position information of the first image block and the second image block in the image, respectively. It can be understood that the position of the image block here is not the physical coordinate of the image block, but the pixel position of the image block. Exemplarily, when the size of the cutting box is 416*416 and the sliding pixel is 300, the position information of the first image block is B (416, 416), and the position information of the second image block is C (716, 416). By analogy, according to the coordinates of the image, the size of the image box during cutting, and the sliding pixel translation operation, the position information of each image box in the image can be obtained.

[0127] Optionally, before cutting each image in the first training set into a plurality of image blocks, each image in the first training set needs to be subjected to data enhancement processing. Specifically, the data enhancement processing method may be one or more of the following processing methods: randomly adjusting the image size, randomly adjusting the image contrast, randomly adjusting the image hue, randomly adjusting the image brightness, randomly adding noise to the image, randomly changing the image color model, and randomly cropping the image.

[0128] For example, randomly adjusting the brightness and contrast of an image can change the quality of the image so that it is consistent with the different imaging qualities caused by environmental factors such as air quality in real shooting scenes; randomly cropping an image can change the position of the face target in the image so that it is consistent with the depth of field change of the foreground and background caused by the change of the face target position in the real scene; it can be seen that by enhancing the images in the data set, the images in the final training set include various problems that may affect the imaging quality under conventional framing conditions.

[0129] S302: Cut each image in the third training set into a plurality of image blocks, and label the image blocks to obtain a fourth training set.

[0130] The third training set is the second face mask data set. Specifically, the proportion of face targets included in each picture in the second face mask data set is greater than a preset threshold. The pictures in the data set are obtained by the staff through daily shooting. Compared with the pictures in the first face mask data set, the pictures in the data set are closer to pictures taken under a surveillance camera, that is, the face targets in the pictures are closer to small targets. The present invention can improve the accuracy of small target detection by training with pictures in the non-public face training set.

[0131] Specifically, the data set contains a large number of images and labels of the images, and the labels are category information of whether the face is wearing a mask. For example, when the face in the image is wearing a mask, its category information is 1, and when the face is not wearing a mask, its category information is 0. Of course, the category information can also be represented in other forms, which is not limited in this embodiment.

[0132] The cut image blocks are labeled to obtain the fourth training set. The labels of the image blocks in the fourth training set include position information of the image blocks in the picture and category information of whether the face in the image blocks is wearing a mask. The category information of the image blocks is the same as the category information of the picture before cutting, that is, the category information of the face in the picture before cutting is wearing a mask, then the category information of the face in each image block after cutting is also wearing a mask. Among them, after the cutting is completed, some image blocks may not have faces, then the category information of the image blocks without faces is not wearing masks.

[0133] Specifically, the method for cutting the images in the third training set and the method for marking the image block position information are the same as step S301, and will not be repeated here.

[0134] Optionally, before cutting each image in the third training set into multiple image blocks, it is necessary to perform data enhancement processing on each image in the third training set. The enhancement processing method and its effect are the same as those in step S301, and will not be repeated here.

[0135] S303: Pre-train the first preset model using the second training set to obtain a pre-trained model.

[0136] The first preset model uses the YOLOV4-tiny network model. The YOLOV4-tiny network model can be used as follows Figure 1 As shown in the structure, it can be understood that the network structure of the YOLOV4-tiny network model can be transformed, for example, the number of CSP networks is greater or less than Figure 1 The network model shown. The image blocks in the second training set are input into the first preset model in a certain number each time for model pre-training. For example, the number of image blocks input each time can be 16. Correspondingly, the output results of 16 image blocks will be obtained, and the output results are used to indicate whether the face in the image block is wearing a mask. The YOLO loss value is obtained according to the output result of the model training and the label of the image block. The YOLO loss value is back-propagated, and the model parameters are iteratively updated using the gradient descent method until the iteration is completed to obtain the pre-trained model.

[0137] The first preset model includes a backbone network and a remaining network, wherein the parameters of the backbone network of the first preset model are frozen during the pre-training process. The parameters of the backbone network are frozen, which means that the parameters of the backbone network are not updated during the iterative training process and remain unchanged, and only the parameters of the remaining network are updated. By freezing the parameters of the backbone network, the training process can be accelerated and the training time can be reduced.

[0138] S304: Use the fourth training set and the second preset model to perform formal training to obtain a mask recognition model.

[0139] The second preset model uses the YOLOV4-tiny network model. The YOLOV4-tiny network model can be used Figure 1 The structure shown. The image blocks in the fourth training set are input into the first preset model in a certain number each time for model pre-training. For example, the number of image blocks input each time can be 16. Correspondingly, the output results of 16 image blocks will be obtained, and the output results are used to indicate whether the face in the image block is wearing a mask. The YOLO loss value is obtained according to the output result of the model training and the label of the image block. The YOLO loss value is back-propagated, and the model parameters are iteratively updated using the gradient descent method until the iteration is completed to obtain the mask recognition model.

[0140] The second preset model includes a backbone network and a remaining network. The parameters of the backbone network of the second preset model are the parameters of the backbone network of the pre-trained model, and the initial parameters of the remaining network of the second preset model are the parameters of the remaining network of the pre-trained model. By pre-training, the parameters of the remaining network are set to a better state, which can accelerate the convergence of the model in formal training.

[0141] In the process of formal training, the parameters of the backbone network are frozen, just like the pre-training process, and only the parameters of the remaining network are updated.

[0142] In this embodiment, the second training set and the fourth training set are obtained by cutting the images in the first training set and the third training set, respectively, and the image blocks in the second training set are input into the first preset model for pre-training to obtain a pre-trained model, and on the basis of the pre-trained model, the image blocks in the fourth training set are input into the second preset model for formal training to obtain a mask recognition model. This method improves the accuracy of recognizing small targets at a long distance by cutting the image into multiple image blocks and training and recognizing small image blocks.

[0143] Figure 6 Flow chart of a pre-training method for a mask recognition model provided in Embodiment 2 of the present invention. This embodiment is a detailed description of step S303 in Embodiment 1. Figure 6 As shown, the pre-training method of the mask recognition model includes the following steps.

[0144] S601, loading the parameters obtained by training on the ImageNet dataset into the backbone network of the first preset model.

[0145] The image blocks in the ImageNet dataset are input into the backbone network of the YOLOV4-tiny network model, and the parameters of the model are trained on the backbone network. Finally, the trained parameters are filled into the corresponding positions of the backbone network in turn.

[0146] Exemplarily, when the model of the backbone network is n=Ax+By+Cz, the ImageNet dataset is input into the backbone network and trained, and the values ​​of parameters A, B, and C can be obtained and filled into the corresponding positions of the backbone network.

[0147] S602: After loading is completed, freeze the backbone network of the first preset model.

[0148] According to step S601, after filling each parameter into the corresponding position of the backbone network in sequence, freezing the backbone network means that the parameters of the backbone network part will not be changed during pre-training.

[0149] S603: Input a first number of image blocks in the second training set into a first preset model for training each time to obtain training results.

[0150] Input the first number of image blocks in the second training set into the YOLOV4-tiny network model. Exemplarily, the first number may be 16 image blocks. Input the first number of image blocks each time until all the image blocks in the second training set are input into the YOLOV4-tiny network model. Train the model according to the first number of image blocks input each time to obtain each training result, and output the training result.

[0151] S604, determining a YOLO loss value according to the label of the input image block and the training result of the input image block.

[0152] The label of the input image block is the category information of whether the face is wearing a mask. For example, the category information of wearing a mask is 1, and the category information of not wearing a mask is 0. The training result of the image block is a real number between 0 and 1. For example, it can be 0.12 or 0.9. When the value is close to 1, it means that the face target of the image block is wearing a mask. When the value is close to 0, it means that the face target of the image block is not wearing a mask. The difference between the label of the input image block and the training result of the input image block is used as the YOLO loss value.

[0153] S605, performing back propagation according to the YOLO loss value to obtain update parameters of the remaining network of the first preset model.

[0154] Back propagation is performed based on the YOLO loss value, that is, the gradient of the YOLO loss value to each model parameter is calculated, and then the gradient is used to update the parameters of the remaining network model in the first preset model through the gradient descent method. Gradient descent is a kind of iterative method. In simple terms, it is a method for finding the minimum objective function. That is, when solving the minimum value of the loss function, the gradient descent method can be used to iterate step by step to obtain the minimized loss function and model parameter value.

[0155] S606: Use the update parameters to update the parameters of the remaining network of the first preset model.

[0156] S607, judging whether the iteration condition is satisfied.

[0157] After each update is completed, it is determined whether the iteration condition is met. When the iteration condition is met, step S608 is executed. When the iteration condition is not met, the process returns to step S603.

[0158] The iteration condition is, for example, completing a preset number of iteration trainings, which may be 100 or 120, etc., wherein one iteration training means training all the image blocks in the second training set once.

[0159] S608: Use the trained model as a pre-trained model.

[0160] The parameters of the remaining network of the first preset model are iteratively updated through steps S603-608 until the iteration condition is met, and the trained model is used as the pre-trained model.

[0161] The model parameters output after the pre-training are sequentially filled into the corresponding positions of the remaining network of the first preset network model for formal training of the second preset network model.

[0162] In this embodiment, the parameters obtained by training on the ImageNet data set are loaded onto the backbone network of the first preset network model, and the backbone network is frozen. The first preset network model is pre-trained using the data of the second training set, and the YOLO loss value is obtained based on the difference between the label of the input image block and the training result of the input image block. The YOLO loss value is back-propagated to obtain the gradient of the YOLO loss value to the model parameters, and then the gradient is used to update the model parameters through the gradient descent method, and the pre-trained model is obtained through iteration. This method can set the parameters of the first preset network model to a better state by pre-training the first preset network model, thereby accelerating the convergence of the second preset network model in formal training.

[0163] Figure 7 This is a flow chart of a formal training method for a mask recognition model provided in Embodiment 3 of the present invention. This embodiment is a detailed description of step S304 in Embodiment 1. Figure 7 As shown, the formal training method of the mask recognition model includes the following steps.

[0164] S701, loading the parameters of the backbone network in the pre-trained model into the backbone network of the second preset model.

[0165] The parameters on the backbone network in the pre-trained model obtained in Example 2 are sequentially filled into the corresponding positions of the backbone network in the second preset model.

[0166] S702, after loading is completed, freezing the backbone network of the second preset model.

[0167] Freezing the backbone network in the second preset model means ensuring that the parameters of the backbone network are not changed when training the remaining network of the second preset model.

[0168] S703, loading the parameters of the remaining network of the pre-trained model onto the remaining network of the second preset model.

[0169] The parameters of the remaining network of the pre-trained model obtained in Example 2 are filled in the corresponding positions of the remaining network in the second preset model in sequence, and the second preset model is formally trained based on the parameters obtained in the pre-training, which is conducive to accelerating the convergence of the second preset network model in the formal training, thereby improving the efficiency of training.

[0170] S704: Input a first number of image blocks in the fourth training set into a second preset model for training each time to obtain training results.

[0171] A first number of image blocks in the fourth training set are input into the second preset network model. The specific first number and method of inputting image blocks are the same as step S603 in Example 2, and will not be repeated here.

[0172] S705, determining a YOLO loss value according to the label of the input image block and the training result of the input image block.

[0173] The label information of the input image block is the same as the information of the image block in step S604 in the second embodiment, and will not be repeated here. The difference between the label of the input image block and the training result of the input image block is used as the YOLO loss value.

[0174] S706, performing back propagation according to the YOLO loss value to obtain update parameters of the remaining network of the second preset model.

[0175] Back propagation is performed according to the YOLO loss value, that is, the gradient of the YOLO loss value to each model parameter is calculated, and then the gradient descent method is used to iterate and update the model parameters. In the process of formal model training, at this stage, the relevant parameters of the training stage need to be set according to the specific training index requirements. These parameters include learning rate, number of iterations, and attenuation strategy. By manually adjusting these relevant parameters, the model training can be accelerated, and the model parameters obtained by training can be optimized, which can further improve the accuracy of model recognition.

[0176] S707: Use the update parameters to update the parameters of the remaining network of the second preset model.

[0177] S708, determining whether the iteration condition is satisfied.

[0178] After each update is completed, it is determined whether the iteration condition is met. When the iteration condition is met, step S709 is executed. When the iteration condition is not met, the process returns to step S704.

[0179] The iteration condition is, for example, that the YOLO loss value is reduced to a preset YOLO loss value, and the YOLO loss value no longer changes significantly during multiple consecutive trainings, wherein the change in the YOLO loss value can be determined by the variance. Specifically, when the gradient descent method is used for iterative training, the YOLO loss value decreases continuously during training. When the YOLO loss value decreases to a lower level and no longer decreases significantly, the minimum YOLO loss value is obtained, at which point the iteration is completed and the training is stopped.

[0180] S709, fill in the corresponding positions of the remaining network in the second preset model with the model parameters obtained in the formal training in sequence to obtain a mask recognition model.

[0181] In the present embodiment, on the basis of the pre-trained model, the backbone network in the second preset model is frozen, wherein the parameters of the backbone network are the same as those in the pre-trained model. The YOLOV4-tiny network model is formally trained using the data of the fourth training set, and the YOLO loss value is obtained according to the difference between the label of the input image block and the training result of the input image block, and the YOLO loss value is back-propagated to obtain the gradient of the YOLO loss value to the model parameters, and then iterated by the gradient descent method, and the model parameters are updated using the gradient. During the training process, the YOLO loss value continues to decrease, and when the YOLO loss value is reduced to a lower level and no longer significantly reduced, that is, the minimum YOLO loss value is obtained, the training is stopped to obtain the mask recognition model. This method can improve the recognition accuracy of the target detection object by formally training the YOLOV4-tiny network model.

[0182] Figure 8 A mask recognition method is provided in the fourth embodiment of the present invention. The method of this embodiment uses the mask recognition model trained in the first embodiment, the second embodiment, and the third embodiment. Figure 8 As shown, the mask recognition method includes the following steps.

[0183] S801, cutting a picture to be identified into a plurality of image blocks, and determining position information of each image block in the picture to be identified, wherein the picture to be identified contains at least one face target.

[0184] The image to be identified is an image obtained from a surveillance video. Specifically, the surveillance video obtained in real time is converted into a frame-by-frame image by a frame segmentation method. An image obtained from the surveillance video contains at least one face target, which can be 1 face target or 3 face targets.

[0185] The image to be identified is cut into multiple image blocks of the same size. The specific cutting method is the same as the cutting method in the first embodiment and will not be described again here.

[0186] The method for determining the position information of each image block in the image to be identified is the same as the method in the first embodiment and will not be described in detail here.

[0187] S802, input multiple image blocks into a mask recognition model to obtain a first recognition result for each image block, where the first recognition result is used to indicate whether the face in the image block wears a mask.

[0188] A plurality of image blocks are input into the mask recognition model, where the number of the image blocks is the number of image blocks obtained by cutting the image to be recognized.

[0189] The first recognition result of each image block obtained through the mask recognition model is the category information of whether the face in the image block is wearing a mask. The first recognition result is a real number between 0 and 1. When the first recognition result value is close to 1, it indicates that the face target in the image block is wearing a mask. When the first recognition result value is close to 0, it indicates that the face target in the image block is not wearing a mask.

[0190] S803, when the same face target in the image to be recognized exists in different image blocks, the confidence of each image block where the face target is located is calculated, and the first recognition result of the image block with the largest confidence is selected as the recognition result of the face target.

[0191] When the same face target in the picture to be identified exists in different image blocks, the confidence of each image block is calculated according to the position information of each image block where the face target is located in the picture to be identified and the face target detection frame in each image block. Specifically, according to the position information of each image block where the face target is located in the picture to be identified, each image block is restored to the picture to be identified. For each image block, the ratio of the face target detection frame in the image block to the face target detection frame in the picture to be identified is calculated, and the calculated ratio is used as the confidence of each image block.

[0192] The confidence of each image block having the same face target is calculated, and the first recognition result of the image block with the largest confidence is used as the recognition result of the face target.

[0193] S804: output the recognition result of the face target in the image to be recognized.

[0194] The recognition result of the face target in the image to be recognized is the first recognition result of the image block corresponding to the face target. When there are multiple face targets in the image to be recognized, the recognition result of each face target can be obtained separately according to the first recognition result of the image block where the face target is located.

[0195] In this embodiment, an image obtained from a surveillance video is used as a picture to be identified, and the picture to be identified is cut into multiple image blocks of the same size, and then the image blocks are input into a mask recognition model to obtain a first recognition result for each image block, and a recognition result of whether the face target in the picture to be identified is wearing a mask is obtained based on the first recognition result of the image block. Among them, when the same face target in the picture to be identified exists in different image blocks, the confidence of each image block is calculated and the first recognition result of the image block with the largest confidence is selected to indicate whether the face in the image block is wearing a mask. This method uses the mask recognition model of the present invention to identify and check whether the target object is wearing a mask, and further verifies the accuracy of the recognition model based on the recognition result.

[0196] Fig. 9 This is a schematic diagram of the structure of a mask recognition model training device provided in Embodiment 5 of the present invention. Fig. 9 As shown, the training device 90 of the mask recognition model includes: a first cutting module 901, a second cutting module 902, a pre-training module 903, and a formal training module 904.

[0197] Among them, the first cutting module 901 is used to cut each picture in the first training set into multiple image blocks, and label the image blocks to obtain a second training set, the first training set is a first face mask data set, and the labels of the image blocks in the second training set include position information of the image blocks in the picture and category information of whether the face in the image block wears a mask;

[0198] The second cutting module 902 is used to cut each image in the third training set into multiple image blocks, and label the image blocks to obtain a fourth training set, where the third training set is a second face mask data set, and the labels of the image blocks in the fourth training set include position information of the image blocks in the image and category information of whether the face in the image block wears a mask;

[0199] A pre-training module 903 is used to pre-train the first preset model using the second training set to obtain a pre-trained model, where the first preset model includes a backbone network and a remaining network, and the parameters of the backbone network of the first preset model are frozen during the pre-training process;

[0200] The formal training module 904 is used to perform formal training using the fourth training set and the second preset model to obtain a mask recognition model. The second preset model includes a backbone network and a remaining network. The parameters of the backbone network of the second training model are the parameters of the backbone network of the pre-trained model. The initial parameters of the remaining network of the second preset model are the parameters of the remaining network of the pre-trained model. During the formal training process, the parameters of the backbone network of the second preset model are frozen.

[0201] In a possible implementation, the first preset model uses a YOLOV4-tiny network model, and the pre-training module 903 is specifically used for:

[0202] Load the parameters obtained by training on the ImageNet dataset into the backbone network of the first preset model;

[0203] After loading is complete, freeze the backbone network of the first preset model;

[0204] The parameters of the remaining network of the first preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the pre-trained model:

[0205] Inputting a first number of image blocks in the second training set into the first preset model for training each time to obtain training results;

[0206] Determine the YOLO loss value based on the label of the input image block and the training result of the input image block;

[0207] Back propagation is performed according to the YOLO loss value to obtain update parameters of the remaining network of the first preset model;

[0208] The parameters of the remaining network of the first preset model are updated using the update parameters.

[0209] In a possible implementation, the second preset model adopts the YOLOV4-tiny network model, and the formal training module 904 is specifically used for:

[0210] Loading the parameters of the backbone network of the pre-trained model onto the backbone network of the second preset model;

[0211] After loading is complete, freeze the backbone network of the second preset model;

[0212] Loading the parameters of the remaining network of the pre-trained model onto the remaining network of the second preset model;

[0213] Use the following iterative process to update the parameters of the remaining network of the second preset model until the iteration condition is met, and then use the trained model as the mask recognition model:

[0214] Inputting a first number of image blocks in a fourth training set into a second preset model for training each time to obtain a training result;

[0215] Determine the YOLO loss value based on the label of the input image block and the training result of the input image block;

[0216] Back propagation is performed based on the YOLO loss value to obtain the update parameters of the remaining network of the preset model;

[0217] The parameters of the remaining network of the second preset model are updated using the update parameters.

[0218] In a possible implementation, the first cutting module 901 and the second cutting module 902 are specifically used to:

[0219] Starting from any corner of the image, slide the image frame with preset pixels to cut the image in the image frame to form an image block, wherein the size of the preset pixels is smaller than the length and width of the image frame;

[0220] The coordinates of the frame of the image block are obtained according to the picture frame coordinates marked on the picture and the size of the sliding image frame.

[0221] In a possible implementation, the first cutting module 901 and the second cutting module 902 further include:

[0222] The enhancement unit is used to perform data enhancement processing on the images in the first training set and the third training set, and the data enhancement processing includes one or more of the following processing: randomly adjusting the image size, randomly adjusting the image contrast, randomly adjusting the image hue, randomly adjusting the image brightness, randomly adding noise to the image, randomly changing the image color model, and randomly cropping the image.

[0223] In a possible implementation, the size of the image block in the first cutting module 901 or the second cutting module 902 is 416*416 pixels, and the backbone network in the pre-training module or the formal training module includes 6 cross-stage partial CSP networks connected in series, and each CSP network is used to extract features from the input image;

[0224] The target CSP network in the backbone network is connected to the input of the CAT module, and the output of the CAT module is connected to the remaining network. The feature map extracted by the target CSP network is 26*26 pixels and 13*13 pixels. The CAT module is used to connect the feature map extracted by the target CSP network.

[0225] In a possible implementation, the proportion of face targets in some images in the first face mask data set is greater than a preset threshold, and the proportion of face targets in the remaining images is less than or equal to the preset threshold;

[0226] The proportion of face targets included in each image in the second face mask data set is greater than a preset threshold.

[0227] The device provided in this embodiment can be used to execute the method steps of the above-mentioned embodiment one, embodiment two or embodiment three. The specific implementation method and technical effects are similar and will not be repeated here.

[0228] Fig.10This is a structural diagram of a mask recognition device provided by Embodiment 6 of the present invention. Fig.10 As shown, the mask recognition device 10 includes: a cutting module 110, an input module 120, a calculation module 130, and an output module 140.

[0229] The cutting module 110 is used to cut the image to be identified into multiple image blocks and determine the position information of each image block in the image to be identified, where the image to be identified contains at least one face target;

[0230] An input module 120, used to input multiple image blocks into a mask recognition model to obtain a first recognition result for each image block, where the first recognition result is used to indicate whether the face in the image block is wearing a mask;

[0231] The calculation module 130 calculates the confidence of each image block where the face target is located according to the position information of each image block where the face target is located in the image to be identified and the face target detection frame in each image block, when the same face target in the image to be identified exists in different image blocks, and selects the first recognition result of the image block with the largest confidence as the recognition result of the face target;

[0232] The output module 140 is used to output the recognition result of the face target in the image to be recognized.

[0233] The device provided in this embodiment can be used to execute the method steps of the above-mentioned embodiment 4. The specific implementation method and technical effects are similar and will not be repeated here.

[0234] Fig.11 An electronic device 11 for mask recognition provided in Embodiment 7 of the present invention includes:

[0235] at least one processor 111; and

[0236] A memory 112 is communicatively connected to at least one processor 111; wherein,

[0237] The memory 112 stores instructions that can be executed by at least one processor 111, and the instructions are executed by at least one processor 111 so that the at least one processor 111 can execute the training method of the mask recognition model as described above.

[0238] The specific implementation process of the processor 111 can refer to the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0239] Embodiment 8 of the present invention provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the method steps in the method embodiment described above. The specific implementation methods and technical effects are similar and will not be repeated here.

[0240] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention is indicated by the following claims.

[0241] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A training method for a mask recognition model, characterized in that: include: Cut each picture in the first training set into multiple image blocks, and label the image blocks to obtain a second training set, where the first training set is a first face mask data set, and the labels of the image blocks in the second training set include position information of the image blocks in the picture and category information of whether the face in the image block wears a mask; Cut each picture in the third training set into multiple image blocks, and label the image blocks to obtain a fourth training set, where the third training set is a second face mask data set, and the labels of the image blocks in the fourth training set include position information of the image blocks in the picture and category information of whether the face in the image block wears a mask; Pre-training a first preset model using the second training set to obtain a pre-trained model, wherein the first preset model includes a backbone network and a remaining network, and during the pre-training process, parameters of the backbone network of the first preset model are frozen, and the first preset model adopts a YOLOV4-tiny network model; The fourth training set and the second preset model are used for formal training to obtain a mask recognition model, wherein the second preset model includes a backbone network and a remaining network, the parameters of the backbone network of the second preset model are the parameters of the backbone network of the pre-trained model, the initial parameters of the remaining network of the second preset model are the parameters of the remaining network of the pre-trained model, and the parameters of the backbone network of the second preset model are frozen during the formal training process. The second preset model adopts the YOLOV4-tiny network model.

2. The method according to claim 1, characterized in that The using the second training set to pre-train the first preset model to obtain the pre-trained model includes: Loading the parameters obtained by training on the ImageNet dataset onto the backbone network of the first preset model; After the loading is completed, freezing the backbone network of the first preset model; The parameters of the remaining network of the first preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the pre-trained model: Inputting a first number of image blocks in the second training set into the first preset model for training each time to obtain a training result; Determine the YOLO loss value based on the label of the input image block and the training result of the input image block; Perform back propagation according to the YOLO loss value to obtain update parameters of the remaining network of the first preset model; The update parameters are used to update the parameters of the remaining network of the first preset model.

3. The method according to claim 2, characterized in that The method of using the fourth training set and the second preset model to perform formal training to obtain a mask recognition model includes: Loading the parameters of the backbone network of the pre-trained model onto the backbone network of the second preset model; After the loading is completed, freezing the backbone network of the second preset model; Loading the parameters of the remaining network of the pre-trained model onto the remaining network of the second preset model; The parameters of the remaining network of the second preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the mask recognition model: Inputting a first number of image blocks in the fourth training set into the second preset model for training each time to obtain a training result; Determine the YOLO loss value based on the label of the input image block and the training result of the input image block; Back propagation is performed according to the YOLO loss value to obtain update parameters of the remaining network of the preset model; The update parameters are used to update the parameters of the remaining network of the second preset model.

4. The method according to any one of claims 1 to 3, characterized in that: Cutting each picture in the first training set and the third training set into a plurality of image blocks respectively includes: Starting from any corner of the picture, sliding the image frame with preset pixels, cutting the image in the image frame to form the image block, wherein the size of the preset pixels is smaller than the length and width of the image frame; The coordinates of the frame of the image block are obtained according to the coordinates of the picture frame marked with the picture and the size of the sliding image frame.

5. The method according to claim 4, characterized in that Before cutting each picture in the first training set and the third training set into a plurality of image blocks respectively, the method further includes: Data enhancement processing is performed on the images in the first training set and the third training set, and the data enhancement processing includes one or more of the following processing: randomly adjusting the image size, randomly adjusting the image contrast, randomly adjusting the image hue, randomly adjusting the image brightness, randomly adding noise to the image, randomly changing the image color model, and randomly cropping the image.

6. The method according to any one of claims 2 or 3, characterized in that: The size of the image block is 416 416 pixels, the backbone network includes 6 cross-stage partial CSP networks connected in series, each of the CSP networks is used to extract features from the input image; The target CSP network in the backbone network is connected to the input of the CAT module, the output of the CAT module is connected to the remaining network, and the feature map extracted by the target CSP network is 26 26 pixels and 13 13 pixels, the CAT module is used to connect the feature map extracted by the target CSP network.

7. The method according to any one of claims 1 to 3, characterized in that: The proportion of face targets in some images in the first face mask data set is greater than a preset threshold, and the proportion of face targets in the remaining images is less than or equal to the preset threshold; The proportion of facial targets included in each image in the second face mask data set is greater than the preset threshold.

8. A mask recognition method, characterized in that: A mask recognition model trained by the method according to any one of claims 1 to 7, the method comprising: Cutting a picture to be identified into a plurality of image blocks, and determining position information of each image block in the picture to be identified, wherein the picture to be identified contains at least one face target; Inputting the multiple image blocks into the mask recognition model to obtain a first recognition result for each image block, wherein the first recognition result is used to indicate whether the face in the image block wears a mask; When the same face target in the image to be identified exists in different image blocks, the confidence of each image block where the face target is located is calculated according to the position information of each image block where the face target is located in the image to be identified and the face target detection frame in each image block, and the first recognition result of the image block with the largest confidence is selected as the recognition result of the face target; Output the recognition result of the face target in the image to be recognized.

9. The method according to claim 8, characterized in that The calculating the confidence of each of the image blocks where the face target is located according to the position information of each of the image blocks where the face target is located in the picture to be identified and the face target detection frame in each of the image blocks comprises: Restoring each image block to the picture to be identified according to position information of each image block where the face target is located in the picture to be identified; For each image block, calculating the ratio of the face target detection frame in the image block to the face target detection frame in the image to be identified; The calculated ratio is used as the confidence of the image block.

10. A training device for a mask recognition model, characterized in that: include: A first cutting module is used to cut each picture in the first training set into multiple image blocks, and label the image blocks to obtain a second training set, wherein the first training set is a first face mask data set, the pictures are labeled with the coordinates of the picture frame and the category information of whether the face wears a mask, and the labels of the image blocks in the second training set are used to indicate the coordinates of the image block frame and the category information of whether the face wears a mask; a second cutting module, used to cut each picture in the third training set into multiple image blocks, and label the image blocks to obtain a fourth training set, wherein the third training set is a second face mask data set, the pictures are labeled with the coordinates of the picture frame and the category information of whether the face wears a mask, and the labels of the image blocks in the fourth training set are used to indicate the coordinates of the image block frame and the category information of whether the face wears a mask; A pre-training module, used to pre-train a first preset model using the second training set to obtain a pre-trained model, wherein the first preset model includes a backbone network and a remaining network, and during the pre-training process, parameters of the backbone network of the first preset model are frozen, and the first preset model adopts a YOLOV4-tiny network model; A formal training module is used to perform formal training using the fourth training set and the second preset model to obtain a mask recognition model, wherein the second preset model includes a backbone network and a remaining network, the parameters of the backbone network of the second preset model are the parameters of the backbone network of the pre-trained model, the initial parameters of the remaining network of the second preset model are the parameters of the remaining network of the pre-trained model, and the parameters of the backbone network of the second preset model are frozen during the formal training process, and the second preset model adopts the YOLOV4-tiny network model.

11. The device according to claim 10, characterized in that The pre-training module is specifically used for: Loading the parameters obtained by training on the ImageNet dataset onto the backbone network of the first preset model; After the loading is completed, freezing the backbone network of the first preset model; The parameters of the remaining network of the first preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the pre-trained model: Inputting a first number of image blocks in the second training set into the first preset model for training each time to obtain a training result; Determine the YOLO loss value based on the label of the input image block and the training result of the input image block; Perform back propagation according to the YOLO loss value to obtain update parameters of the remaining network of the first preset model; The update parameters are used to update the parameters of the remaining network of the first preset model.

12. The device according to claim 11, characterized in that The formal training module is specifically used for: Loading the parameters of the backbone network of the pre-trained model onto the backbone network of the second preset model; After the loading is completed, freezing the backbone network of the second preset model; Loading the parameters of the remaining network of the pre-trained model onto the remaining network of the second preset model; The parameters of the remaining network of the second preset model are updated using the following iterative process until the iteration condition is met, and the trained model is used as the mask recognition model: Inputting a first number of image blocks in the fourth training set into the second preset model for training each time to obtain a training result; Determine the YOLO loss value based on the label of the input image block and the training result of the input image block; Back propagation is performed according to the YOLO loss value to obtain update parameters of the remaining network of the preset model; The update parameters are used to update the parameters of the remaining network of the second preset model.

13. The device according to claim 10, characterized in that The first cutting module and the second cutting module are specifically used for: Starting from any corner of the picture, sliding the image frame with preset pixels, cutting the image in the image frame to form the image block, wherein the size of the preset pixels is smaller than the length and width of the image frame; The coordinates of the frame of the image block are obtained according to the coordinates of the picture frame marked with the picture and the size of the sliding image frame.

14. A mask recognition device, characterized in that: The mask recognition model trained by the method according to any one of claims 1 to 7 is applied, and the device comprises: A cutting module, used for cutting the image to be identified into a plurality of image blocks, and determining the position information of each image block in the image to be identified, wherein the image to be identified contains at least one human face target; An input module, used to input the multiple image blocks into the mask recognition model to obtain a first recognition result for each image block, wherein the first recognition result is used to indicate whether the face in the image block wears a mask; A calculation module, configured to, when the same face target in the to-be-recognized picture exists in different image blocks, calculate the confidence of each image block where the face target is located according to the position information of each image block where the face target is located in the to-be-recognized picture and the face target detection frame in each image block, and select the first recognition result of the image block with the largest confidence as the recognition result of the face target; The output module is used to output the recognition result of the face target in the image to be recognized.

15. An electronic device for mask recognition, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

Images