Method, device, electronic device and storage medium for detecting face wearing mask

Abstract

The present invention provides a method, device, electronic device, and storage medium for detecting faces wearing masks. The method comprises: generating a facial heat map for each frame of a target video, and obtaining image feature information for each frame of the image; inputting the facial heat map and image feature information for each frame of the image into a face detection model to obtain facial border information output by the face detection model, wherein the facial border information is used to mark the face area in the target video. The method, device, electronic device, and storage medium for detecting faces wearing masks provided by the present invention generate corresponding facial border information based on the feature information starting from the center position of the facial heat map, using the facial heat map and image feature information for each frame of the image. This can meet the dual requirements of accuracy and speed for detecting faces wearing masks, and can be lightweight and deployed on the device side for security control.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, electronic device and storage medium for detecting a face of a person wearing a mask. Background Art

[0002] Current face detectors simplify target detection into an image classification problem of a large number of potential object bounding boxes, displaying each target in the form of an axisymmetric bounding box tightly surrounding the target. For the bounding box, the classifier determines whether the image content is a specific target or background. The one-stage detector slides a series of complexly arranged possible bounding boxes (called anchor boxes) across the image and performs background and foreground classification without specifying the content of the box. The two-stage detector recalculates the image features of each detection box and then classifies these features. Finally, the non-maximum suppression algorithm is used to delete repeated detections of the same instance by calculating the intersection of the bounding boxes to obtain the final detection and recognition result. Although this idea has achieved good success in the past five years. However, the two-stage approach of using a region proposal network based on anchor boxes has some drawbacks. First, sliding window-based object detectors are somewhat time-consuming because they need to enumerate all possible object positions and sizes. In addition, in order to improve the overlap between anchor boxes and ground truth, face detectors usually require a large number of dense anchors to achieve a good recall rate. The more anchor boxes there are, the slower the inference speed will be, which cannot meet the requirements of edge deployment. Second, anchor points are hyperparameter designs based on statistical calculations of specific data sets. Therefore, face detection in multi-scale dense scenes is not always feasible, and its universal performance will be poor. Third, during network training, this method of relying on feature extraction to classify anchor box positive and negative samples is not very robust to situations such as face occlusion, and the post-processing of non-maximum suppression is difficult to differentiate and train. Therefore, most existing detectors cannot meet the dual requirements of accuracy and speed for detecting faces wearing masks. Summary of the Invention

[0003] In response to the problems existing in the prior art, the present invention provides a method, device, electronic device and storage medium for detecting faces wearing masks.

[0004] In a first aspect, the present invention provides a method for detecting a face wearing a mask, comprising:

[0005] Generate a facial heat map for each frame of the target video and obtain image feature information for each frame;

[0006] Inputting the facial heat map and image feature information of each frame into a face detection model to obtain facial bounding box information output by the face detection model, and the facial bounding box information is used to mark the face area in the target video;

[0007] The face detection model is a model trained by machine learning using the face bounding box, face heat map, and image feature information determined based on the sample image, as well as the face frame information of the sample image, for annotating faces in the video.

[0008] The face detection model includes a center position judgment layer and a face frame generation layer. The center position judgment layer is used to determine the center position of the face in the face heat map; the face frame generation layer is used to generate face frame information based on image feature information located at the center position.

[0009] In one embodiment, the facial heat map and image feature information of each frame of image are input into a face detection model to obtain facial bounding box information output by the face detection model, including:

[0010] Inputting the facial heat map of each frame image into the center position determination layer, and the center position determination layer outputs the center position of the face in each frame image;

[0011] The center position of the face in each frame image and the image feature information of each frame image are input into the face frame generation layer. The face frame generation layer determines the image feature information of the first area based on the center position, and generates face frame information based on the image feature information of the first area; wherein the center of the first area is the center position.

[0012] In one embodiment, the center position determination layer outputs the center position of the face in each frame of the image, including:

[0013] The center position determination layer determines the peak value in the facial heat map according to the facial heat map of each frame image, and outputs the position of the peak value as the center position of the face.

[0014] In one embodiment, the method further includes the step of constructing a face detection model, including:

[0015] Acquire a sample image, wherein the sample image includes a facial bounding box, a facial heat map, and image feature information;

[0016] Constructing a preset neural network, wherein the loss function of the preset neural network includes a center position loss function, a center offset loss function, and a face width and height loss function; wherein the center position loss function is used to learn to determine the center position based on the face heat map, the center offset loss function is used to learn to determine the offset of the center position based on the face heat map and the face bounding box, and the face width and height loss function is used to determine the facial border information based on the image feature information and the center position;

[0017] Input the sample image into the preset neural network for model training;

[0018] If the model converges during the training process, a face detection model is generated.

[0019] In a second aspect, the present invention provides a face detection device for a person wearing a mask, comprising:

[0020] An acquisition module is used to generate a facial heat map for each frame of the target video and obtain image feature information for each frame of the image;

[0021] A detection module, configured to input the facial heat map and image feature information of each frame into a face detection model, and obtain facial bounding box information output by the face detection model, wherein the facial bounding box information is used to mark the face area in the target video;

[0022] The face detection model is a model trained by machine learning using the face bounding box, face heat map, and image feature information determined based on the sample image, as well as the face frame information of the sample image, for annotating faces in the video.

[0023] The face detection model includes a center position judgment layer and a face frame generation layer. The center position judgment layer is used to determine the center position of the face in the face heat map; the face frame generation layer is used to generate face frame information based on image feature information located at the center position.

[0024] In a third aspect, the present invention provides an electronic device comprising a memory and a memory storing a computer program, wherein when the processor executes the program, the steps of the method for detecting a face wearing a mask described in the first aspect are implemented.

[0025] In a fourth aspect, the present invention provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the steps of the mask-wearing face detection method described in the first aspect.

[0026] The method, device, electronic device and storage medium for detecting faces wearing masks provided by the present invention use the facial heat map and image feature information of each frame image, starting from the center position of the facial heat map, and generating corresponding facial border information based on the feature information. This can meet the dual requirements of accuracy and speed for detecting faces wearing masks, and can be lightweight deployed on the device side for security control. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 Schematic diagram of the process of detecting a face wearing a mask provided by the present invention;

[0029] Figure 2 It is a structural schematic diagram of the face detection device for people wearing masks provided by the present invention;

[0030] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention; DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0032] The following combination Figure 1-Figure 3 The present invention describes a method, device, electronic device and storage medium for detecting a face wearing a mask.

[0033] Figure 1 A schematic diagram of a method for detecting a face wearing a mask according to the present invention is shown. Figure 1 , the method comprising:

[0034] 11. Generate a facial heat map for each frame of the target video, and obtain image feature information for each frame of the image;

[0035] 12. Input the facial heat map and image feature information of each frame into the face detection model to obtain the facial bounding box information output by the face detection model. The facial bounding box information is used to mark the face area in the target video;

[0036] The face detection model is a model trained through machine learning that uses the face bounding box, face heat map, and image feature information determined based on the sample image, as well as the face border information of the sample image, to annotate faces in the video.

[0037] The face detection model includes a center position judgment layer and a face frame generation layer. The center position judgment layer is used to determine the center position of the face based on the face heat map; the face frame generation layer is used to generate face frame information based on the image feature information located at the center position.

[0038] Regarding steps 11 and 12, it should be noted that, in the present invention, the method is used to identify faces wearing masks in the target video and mark them with frames. Video is a variation of multiple frames of continuous images. To this end, the video needs to be decomposed to decompose the image of each frame. A heat map is an analytical method that uses different markers to mark and present areas in an image according to the degree of attention they receive. The marking method generally adopts the form of color depth, density of points, and presentation weight. Since this method is for face detection, each frame needs to be processed to generate a corresponding face heat map. The heat map is generated based on the degree of attention received by the face in the image. The generation of heat maps belongs to the prior art, and the generation process will not be elaborated on here.

[0039] In the present invention, any image contains image feature information. This image feature information can include color features, texture features, shape features, and spatial relationship features. Image features can be used to make judgments during the detection process. Therefore, this method also requires obtaining image feature information for each frame of the image.

[0040] Then, the facial heat map and image feature information of each frame image are input into the face detection model, so that the face detection model analyzes and processes the facial heat map and image feature information of each frame image, and outputs the facial border information corresponding to each frame image. The facial border information is used to mark the face area in the target video, that is, to frame the face in each frame image.

[0041] Since the face area is marked for each frame of the image, in the target video, as the person moves, the facial frame information also changes accordingly, forming a scene in which the frame in the video tracks the face.

[0042] In the present invention, the face detection model is a model obtained through machine learning training that takes the face bounding box, face heat map and image feature information determined based on the sample image, as well as the face frame information of the sample image as input, and is used to label faces in the video.

[0043] It should be noted that the facial bounding box is the facial bounding box obtained by the method of annotating faces in videos in the prior art. This annotation method cannot effectively annotate faces wearing masks in videos. In the process of training the model of this method, since the facial bounding box has a center position, the center position indicates the center of the face. The facial heat map can also reflect a center position. To this end, it is necessary to conduct overall training on the correspondence between the facial bounding box in the sample image and the center position in the heat map (whether an offset occurs), and then generate a facial border information in the heat map that is relatively adaptable to faces wearing masks and not wearing masks based on the image feature information in the sample image, that is, a frame that is different from the facial bounding box generated in the prior art.

[0044] In the present invention, after the face detection model is trained, it can directly analyze and process the facial heat map and image feature information of each frame of image, and output the facial frame information for each frame of image. Therefore, the face detection model includes a center position determination layer and a facial frame generation layer. The center position determination layer is used to determine the center position of the face from the facial heat map; the facial frame generation layer is used to generate facial frame information based on the image feature information located at the center position. Since the input image feature information is all the image feature information in the entire image, but the detection of the face mainly focuses on the features in the facial area, after determining the center position of the face, the image feature information of the portion of the area located at the center position must be filtered out, and then the facial frame information is generated based on the image feature information located at the center position. Here, based on the facial frame information located at the center position, a frame with specific width and height values ​​is actually generated.

[0045] The mask-wearing face detection method provided by the present invention uses the facial heat map of each frame image and the image feature information of each frame image, starting from the center position of the facial heat map, and generates corresponding facial frame information based on the feature information. It can meet the dual requirements of accuracy and speed of mask-wearing face detection, and can be lightweight deployed on the device side for security control.

[0046] In the further description of the above method, the processing of inputting the facial heat map and image feature information of each frame image into the face detection model and obtaining the facial bounding box information output by the face detection model is mainly explained as follows:

[0047] The facial heat map of each frame image is input to the center position judgment layer, and the center position judgment layer outputs the center position of the face in each frame image;

[0048] The center position of the face in each frame image and the image feature information of each frame image are input into the face frame generation layer. The face frame generation layer determines the image feature information of the first area based on the center position, and generates the face frame information based on the image feature information of the first area; wherein the center of the first area is the center position.

[0049] In this regard, it should be noted that, in the present invention, the center position determination layer needs to analyze the facial heat map of each input frame image to determine the center position in the facial heat map. In further explanation, the center position determination layer determines the peak value in the facial heat map based on the facial heat map of each frame image, and outputs the position of the peak value as the center position of the face.

[0050] After determining the center of the face, the center of the face and the image feature information for each frame are input into the facial frame generation layer. The facial frame generation layer then filters out the image feature information for a portion of the area (i.e., a first area) located at the center. This first area can be selected using a preset size or based on a hotspot value near the center.

[0051] Then, facial frame information is generated based on the image feature information at the center. Here, a frame with specific width and height values ​​is actually generated based on the facial frame information at the center.

[0052] A further method of the present invention uses a center position determination layer and a face frame generation layer to perform division of labor and cooperation on each frame of image, thereby accelerating the processing speed.

[0053] In the further explanation of the above method, the construction process of the face detection model is mainly explained, as follows:

[0054] Obtain a sample image, which includes a facial bounding box, a facial heat map, and image feature information;

[0055] Constructing a preset neural network, the preset neural network loss function includes a center position loss function, a center offset loss function, and a face width and height loss function; wherein the center position loss function is used to learn to determine the center position based on the face heat map, the center offset loss function is used to learn to determine the offset of the center position based on the face heat map and the face bounding box, and the face width and height loss function is used to determine the facial border information based on image feature information and the center position;

[0056] Input the sample image into the preset neural network for model training;

[0057] If the model converges during the training process, a face detection model is generated.

[0058] It should be noted that, in this invention, training the model requires sample data (i.e., image data). For example, this training data can include over 10,000 images, including faces of varying scales, angles, expressions, occlusions, and brightness. To ensure accurate training, sample data enhancement can be performed using methods such as random flipping, random scaling, color jittering, and random cropping of the original images. By processing the sample data, facial bounding boxes, facial heat maps, and image feature information can be obtained for each sample.

[0059] Then the initial neural network is constructed, and the loss function of the neural network includes the center position loss function, the center offset loss function and the face width and height loss function.

[0060] The center position loss function is as follows:

[0061]

[0062] Where [x1, y1, x2, y2] is the face bounding box of the face, then the center position of the face is c = [(x1+x2) / 2, (y1+y2) / 2], generating a heat map, Y∈[0,1] W / R×H / R , [W,H,R] are the width, height and output stride of the input image respectively. Represents a human face, represents the background, α and β are parameters and can be set to 2 and 4.

[0063] When remapping the center position from the heat map to the input image, some pixels may be misaligned, which affects the accuracy of face detection. To solve this problem, we calculate whether the center position coordinates are within the image corresponding to the feature information, and predict the position offset to slightly adjust the center position before remapping it to the input resolution. The center offset loss function is as follows:

[0064]

[0065] Among them, O k , They represent the predicted value of the offset and the true value of the center position deviation respectively, and N is the number of center positions.

[0066] The face width and height loss function is as follows:

[0067]

[0068] in, S k The width and height prediction values ​​and the true width and height values ​​of the face.

[0069] From the above, we can see that the overall loss is L c +λ1L size +λ2Loff .

[0070] After the neural network is constructed, the sample image is input into the preset neural network for model training. If the model converges during the training process, a face detection model is generated.

[0071] A further method of the present invention uses a neural network to train a model using a facial bounding box and a facial heat map, establishes a loss relationship between the facial bounding box and the facial heat map, and obtains a model for face detection in an image based on the facial heat map and feature information, thereby improving the accuracy of face detection in the image.

[0072] The following describes the face detection device for people wearing masks provided by the present invention. The face detection device for people wearing masks described below and the face detection method for people wearing masks described above can be referenced to each other.

[0073] Figure 2 The present invention provides a schematic diagram of the structure of a face detection device for people wearing masks, see Figure 2 The device includes an acquisition module 21 and a detection module 22, wherein:

[0074] An acquisition module 21 is used to generate a facial heat map of each frame of the target video and obtain image feature information of each frame of the image;

[0075] The detection module 22 is used to input the facial heat map and image feature information of each frame into the face detection model to obtain the facial bounding box information output by the face detection model. The facial bounding box information is used to mark the face area in the target video;

[0076] The face detection model is a model trained through machine learning that uses the face bounding box, face heat map, and image feature information determined based on the sample image, as well as the face border information of the sample image, to annotate faces in the video.

[0077] The face detection model includes a center position judgment layer and a face frame generation layer. The center position judgment layer is used to determine the center position of the face based on the face heat map; the face frame generation layer is used to generate face frame information based on the image feature information located at the center position.

[0078] In further description of the above device, the detection module is specifically used to:

[0079] The facial heat map of each frame image is input to the center position judgment layer, and the center position judgment layer outputs the center position of the face in each frame image;

[0080] The center position of the face in each frame image and the image feature information of each frame image are input into the face frame generation layer. The face frame generation layer determines the image feature information of the first area based on the center position, and generates the face frame information based on the image feature information of the first area; wherein the center of the first area is the center position.

[0081] In a further description of the above device, the detection module is specifically used to:

[0082] The center position determination layer determines the peak value in the facial heat map according to the facial heat map of each frame image, and outputs the position of the peak value as the center position of the face.

[0083] In a further description of the above apparatus, the apparatus further comprises a building block for:

[0084] Obtain a sample image, which includes a facial bounding box, a facial heat map, and image feature information;

[0085] Constructing a preset neural network, the preset neural network loss function includes a center position loss function, a center offset loss function, and a face width and height loss function; wherein the center position loss function is used to learn to determine the center position based on the face heat map, the center offset loss function is used to learn to determine the offset of the center position based on the face heat map and the face bounding box, and the face width and height loss function is used to determine the facial border information based on image feature information and the center position;

[0086] Input the sample image into the preset neural network for model training;

[0087] If the model converges during the training process, a face detection model is generated.

[0088] Since the principles of the apparatus described in the embodiment of the present invention are the same as those of the method described in the above embodiment, more detailed explanations are omitted here.

[0089] It should be noted that, in the embodiment of the present invention, relevant functional modules can be implemented by a hardware processor.

[0090] The masked face detection device provided by the present invention uses the facial heat map of each frame image and the image feature information of each frame image, starting from the center position of the facial heat map, and generates corresponding facial frame information based on the feature information. It can meet the dual requirements of accuracy and speed in masked face detection, and can be lightweight deployed on the device side for security control.

[0091] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 31 , a communication interface 32 , a memory 33 and a communication bus 34 , wherein the processor 31 , the communication interface 32 and the memory 33 communicate with each other via the communication bus 34 . The processor 31 can call the computer program in the memory 33 to execute the steps of the user off-grid prediction method, for example, including: generating a facial heat map for each frame of the target video, and obtaining image feature information of each frame of the target video; inputting the facial heat map and image feature information of each frame of the target video into the face detection model to obtain facial bounding box information output by the face detection model, and the facial bounding box information is used to mark the face area in the target video; wherein the face detection model is a model obtained by machine learning training using the facial bounding box, facial heat map and image feature information determined according to the sample image, and the facial bounding box information of the sample image as input, and is used to mark the faces in the video; the face detection model includes a center position judgment layer and a face bounding box generation layer, the center position judgment layer is used to determine the center position of the face on the facial heat map; the face bounding box generation layer is used to generate facial bounding box information based on the image feature information located at the center position.

[0092] In addition, the logic instructions in the above-mentioned memory 33 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0093] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the mask-wearing face detection method provided by the above-mentioned methods, the method including: generating a facial heat map for each frame image according to the target video, and obtaining image feature information of each frame image; inputting the facial heat map and image feature information of each frame image into a face detection model to obtain facial border information output by the face detection model, and the facial border information is used to mark the face area in the target video; wherein the face detection model is a model obtained by machine learning training using the facial bounding box, facial heat map and image feature information determined according to the sample image, and the facial border information of the sample image as input; the face detection model includes a center position judgment layer and a face border generation layer, the center position judgment layer is used to determine the center position of the face on the facial heat map; the face border generation layer is used to generate facial border information based on the image feature information located at the center position.

[0094] On the other hand, an embodiment of the present application also provides a processor-readable storage medium, which stores a computer program, and the computer program is used to enable the processor to execute the mask-wearing face detection method provided by the above-mentioned embodiments, for example, including: generating a facial heat map for each frame image according to the target video, and obtaining image feature information of each frame image; inputting the facial heat map and image feature information of each frame image into a face detection model to obtain facial border information output by the face detection model, and the facial border information is used to mark the face area in the target video; wherein the face detection model is a model obtained by machine learning training using the facial bounding box, facial heat map and image feature information determined according to the sample image, and the facial border information of the sample image as input; the face detection model includes a center position judgment layer and a face border generation layer, the center position judgment layer is used to determine the center position of the face on the facial heat map; the face border generation layer is used to generate facial border information based on the image feature information located at the center position.

[0095] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NANDFLASH), solid-state drives (SSDs)), etc.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0097] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting faces wearing masks, characterized in that: include: Generate a facial heat map for each frame of the target video and obtain image feature information for each frame; Image feature information includes color features, texture features, shape features and spatial relationship features; Inputting the facial heat map and image feature information of each frame into a face detection model to obtain facial bounding box information output by the face detection model, and the facial bounding box information is used to mark the face area in the target video; The face detection model is a model trained by machine learning using the face bounding box, face heat map, and image feature information determined based on the sample image, as well as the face frame information of the sample image, for annotating faces in the video. The face detection model includes a center position judgment layer and a face frame generation layer. The center position judgment layer is used to determine the center position of the face in the face heat map; the face frame generation layer is used to generate face frame information based on image feature information located at the center position.

2. The method for detecting a face wearing a mask according to claim 1, wherein: The step of inputting the facial heat map and image feature information of each frame of image into a face detection model to obtain facial frame information output by the face detection model includes: Inputting the facial heat map of each frame image into the center position determination layer, and the center position determination layer outputs the center position of the face in each frame image; The center position of the face in each frame image and the image feature information of each frame image are input into the face frame generation layer. The face frame generation layer determines the image feature information of the first area based on the center position, and generates face frame information based on the image feature information of the first area; wherein the center of the first area is the center position.

3. The method for detecting a face wearing a mask according to claim 2, wherein: The center position determination layer outputs the center position of the face in each frame of the image, including: The center position determination layer determines the peak value in the facial heat map according to the facial heat map of each frame image, and outputs the position of the peak value as the center position of the face.

4. The method for detecting a face wearing a mask according to claim 1, wherein: The method further includes a step of constructing a face detection model, including: Acquire a sample image, wherein the sample image includes a facial bounding box, a facial heat map, and image feature information; Constructing a preset neural network, wherein the loss function of the preset neural network includes a center position loss function, a center offset loss function, and a face width and height loss function; wherein the center position loss function is used to learn to determine the center position based on the face heat map, the center offset loss function is used to learn to determine the offset of the center position based on the face heat map and the face bounding box, and the face width and height loss function is used to determine the facial border information based on the image feature information and the center position; Input the sample image into the preset neural network for model training; If the model converges during the training process, a face detection model is generated.

5. A face detection device for people wearing masks, characterized in that: include: An acquisition module is used to generate a facial heat map for each frame of the target video and obtain image feature information for each frame of the image; Image feature information includes color features, texture features, shape features and spatial relationship features; A detection module, configured to input the facial heat map and image feature information of each frame into a face detection model, and obtain facial bounding box information output by the face detection model, wherein the facial bounding box information is used to mark the face area in the target video; The face detection model is a model trained by machine learning using the face bounding box, face heat map, and image feature information determined based on the sample image, as well as the face frame information of the sample image, for annotating faces in the video. The face detection model includes a center position judgment layer and a face frame generation layer. The center position judgment layer is used to determine the center position of the face in the face heat map; the face frame generation layer is used to generate face frame information based on image feature information located at the center position.

6. The face detection device for people wearing masks according to claim 5, characterized in that: The detection module is specifically used for: Inputting the facial heat map of each frame image into the center position determination layer, and the center position determination layer outputs the center position of the face in each frame image; The center position of the face in each frame image and the image feature information of each frame image are input into the face frame generation layer. The face frame generation layer determines the image feature information of the first area based on the center position, and generates face frame information based on the image feature information of the first area; wherein the center of the first area is the center position.

7. The face detection device for people wearing masks according to claim 6, characterized in that: The detection module is specifically used to: The center position determination layer determines the peak value in the facial heat map according to the facial heat map of each frame image, and outputs the position of the peak value as the center position of the face.

8. The face detection device for people wearing masks according to claim 5, characterized in that: The apparatus further comprises a building block for: Acquire a sample image, wherein the sample image includes a facial bounding box, a facial heat map, and image feature information; Constructing a preset neural network, wherein the loss function of the preset neural network includes a center position loss function, a center offset loss function, and a face width and height loss function; wherein the center position loss function is used to learn to determine the center position based on the face heat map, the center offset loss function is used to learn to determine the offset of the center position based on the face heat map and the face bounding box, and the face width and height loss function is used to determine the facial border information based on the image feature information and the center position; Input the sample image into the preset neural network for model training; If the model converges during the training process, a face detection model is generated.

9. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of the method for detecting a face wearing a mask according to any one of claims 1 to 4 are implemented.

10. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, which is used to enable the processor to execute the steps of the method for detecting a face wearing a mask according to any one of claims 1 to 4.

Images