Face detection method and system
By constructing low-resolution occluded face detection models and dense face detection models, the problem of missing facial features caused by wearing masks is solved, improving the accuracy and efficiency of face detection. It is applicable to multi-scale and dense face detection in large-channel crowd scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UESTC (SHENZHEN) ADVANCED RES INST
- Filing Date
- 2023-04-04
- Publication Date
- 2026-05-29
AI Technical Summary
Existing algorithms cannot effectively handle occlusion caused by wearing masks in face detection, resulting in missing facial features and poor detection results.
We employ MTCNN and Fast-SCNN semantic segmentation networks for face semantic segmentation, and combine residual modules and SDD networks for keypoint localization to construct a low-resolution occluded face detection model to supplement missing facial features. For dense face detection, we use an improved YOLO algorithm and K-means clustering algorithm to construct a dense face detection model.
It improves the detection rate and accuracy of faces wearing masks, reduces missed detections and false detections, and is suitable for multi-scale and dense face detection in crowded scenarios with large passageways.
Smart Images
Figure CN116486454B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of target detection technology, specifically to a face detection method and system. Background Technology
[0002] With the continuous development of deep learning, the field of object detection is also rapidly evolving. Face detection is a crucial area within this field. Because users often wear masks in public places, large numbers of people wearing masks are present in crowded public areas such as train stations, airports, and shopping malls. Public place management agencies, based on safety requirements, conduct face detection on masked individuals in crowded passageways. Face detection refers to using a specific strategy to search for faces in any given image, and if found, returning the face's location and size.
[0003] Currently, most object detection algorithms are designed for detecting large objects sparsely distributed in natural scenes. Two common methods are used in existing algorithms: the first uses an R-CNN-based model that generates candidate bounding boxes beforehand; the second uses a YOLO model that does not generate bounding boxes in advance. Existing algorithms perform well for detecting people in clear, unobstructed images. However, when the distance between the face and the camera varies, resulting in different face sizes, or when people wear masks causing facial occlusion, existing algorithms, not specifically designed for occlusion, suffer from missing facial features during face detection.
[0004] Therefore, there is an urgent need for a face detection method and system to solve the above-mentioned technical problems. Summary of the Invention
[0005] This application provides a face detection method and system. The method can detect faces wearing masks in large passageway scenarios. By using the face detection method, it solves the problem that the facial features of people cannot be identified due to the obstruction caused by wearing masks.
[0006] Firstly, this application provides a face detection method applied in a server. The method includes: acquiring a surveillance image of a preset scene, the surveillance image including a face image and a background image; filtering the surveillance image using a face recognition algorithm to obtain a face image; determining whether the number of facial features is consistent with a preset number of facial features; the number of facial features is the number of facial features corresponding to any face image; if the number of facial features is inconsistent with the preset number of facial features, then inputting the face image into a preset occluded face detection model for detection, the preset occluded face detection model supplementing the missing facial features in the face image to obtain the face detection result corresponding to the person, and outputting the face detection result.
[0007] By adopting the above technical solution, when a person's facial features are occluded, the face image can be input into a preset occluded face detection model, which can help detect occluded faces, supplement the missing facial features in the face image, and improve the detection rate of occluded faces.
[0008] Optionally, the MTCNN network is used to process the surveillance image to obtain face detection boxes; the backbone network is used as a real-time semantic segmentation model to perform face semantic segmentation and extract face boxes from the face detection boxes; the residual module in the backbone network is combined with the SDD network to extract face parts, and the face parts are trained for key point localization to obtain a preset occluded face detection model.
[0009] By adopting the above technical solution and using the MTCNN network in a hierarchical cascade manner, it is possible to detect faces at different scales, obtain the position and size of face bounding boxes, and thus achieve face detection.
[0010] Optionally, key point positioning includes the left eye, right eye, nose, left corner of the mouth, and right corner of the mouth.
[0011] By adopting the above technical solution, face prediction can be improved based on key point localization, and missing facial features in face images can be supplemented, thereby further improving the accuracy of face detection.
[0012] Optionally, the backbone network uses Fast-SCNN as a real-time semantic segmentation model to perform face semantic segmentation, obtaining images labeled with each pixel category; and uses a depth-first traversal algorithm to extract face bounding boxes from the face detection boxes.
[0013] By adopting the above technical solution, the Fast-SCNN semantic segmentation network is used to assist in the detection of occluded faces, thereby reducing the occurrence of situations where face detection cannot be performed due to occlusion.
[0014] Optionally, if the number of facial features is consistent with the preset number of facial features, the face image is input into the preset dense face detection model for detection to obtain the face detection result corresponding to the person, and the face detection result is output.
[0015] By adopting the above technical solution, when detecting a large number of small-scale face images in a large passageway scenario, a preset dense face detection model can be used for detection, which can reduce the occurrence of missed detections and false detections when detecting faces.
[0016] Optionally, the K-means clustering algorithm is used to perform cluster analysis on the standard reference boxes to obtain predicted boxes; the predicted boxes are used to process the monitoring images; the processed monitoring images are input into the improved YOLO dense face detection model for training to obtain the preset dense face detection model.
[0017] By adopting the above technical solution and using the K-means clustering algorithm, more representative prediction boxes are obtained, reducing the deviation and variance between the prediction boxes and the real boxes, accelerating the convergence of the preset dense face detection model, and helping the preset dense face detection model to fine-tune the position information of the prediction boxes more quickly during the training process.
[0018] Optional, the improved YOLO dense face detection model includes a feature extraction network and a prediction layer.
[0019] By adopting the above technical solutions, the feature extraction network is improved based on the original YOLO algorithm, resulting in feature maps with richer small-scale facial features, which helps to improve detection accuracy; the prediction layer structure is also improved to further enhance detection accuracy.
[0020] A second aspect of this application provides a face detection system. The system is a server, which includes a receiving unit, a processing unit, and an output unit: The receiving unit acquires a monitoring image of a preset scene, the monitoring image including a face image and a background image; The processing unit uses a face recognition algorithm to filter the monitoring image to obtain a face image; It determines whether the number of facial features is consistent with the preset number of facial features; The number of facial features is the number of facial features corresponding to any face image; The output unit, if the number of facial features is inconsistent with the preset number of facial features, inputs the face image into a preset occluded face detection model for detection. The preset occluded face detection model supplements the missing facial features in the face image to obtain the face detection result corresponding to the person, and outputs the face detection result.
[0021] Optionally, the receiving unit is used to process the monitoring image using the MTCNN network to obtain a face detection box; the processing unit is used to use the backbone network as a real-time semantic segmentation model to perform face semantic segmentation and extract the face box from the face detection box; the residual module in the backbone network is combined with the SDD network to extract face parts, and the face parts are trained for key point localization to obtain a preset occluded face detection model.
[0022] Optionally, key point positioning includes the left eye, right eye, nose, left corner of the mouth, and right corner of the mouth.
[0023] Optionally, the processing unit is used to perform face semantic segmentation using the Fast-SCNN network as a real-time semantic segmentation model in the backbone network, obtaining images labeled with each pixel category; and to extract face boxes from the face detection boxes using a depth-first traversal algorithm.
[0024] Optionally, the output unit is used to input the face image into a preset dense face detection model for detection if the number of facial features is consistent with the preset number of facial features, to obtain the face detection result corresponding to the person, and to output the face detection result.
[0025] Optionally, the receiving unit is used to perform cluster analysis on the standard reference boxes using the K-means clustering algorithm to obtain predicted boxes; the processing unit is used to process the monitoring image using the predicted boxes; and the processed monitoring image is input into the improved YOLO dense face detection model for training to obtain the preset dense face detection model.
[0026] Optionally, the processing unit for the improved YOLO dense face detection model includes a feature extraction network and a prediction layer.
[0027] In a third aspect, this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory, causing the electronic device to perform any of the methods described above in this application.
[0028] In a fourth aspect, this application provides a computer-readable storage medium storing instructions that, when executed, perform any of the methods described above in this application.
[0029] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:
[0030] 1. When a person's facial features are obscured, the face image can be input into a preset occluded face detection model to assist in detecting occluded faces, supplement the missing facial features in the face image, and improve the detection rate of occluded faces.
[0031] 2. Using the MTCNN network in a hierarchical cascaded manner, it can detect faces at different scales and obtain the position and size of face bounding boxes, thereby achieving face detection. The Fast-SCNN semantic segmentation network is used to assist in detecting occluded faces, reducing the number of cases where face detection fails due to occlusion.
[0032] 3. Key point localization can better predict faces, supplement missing facial features in face images, and further improve face detection accuracy.
[0033] 4. When detecting a large number of small-scale face images in a high-traffic area, a pre-set dense face detection model can be used to reduce the occurrence of missed and false detections. Attached Figure Description
[0034] Figure 1 This is a first flowchart illustrating a face detection method provided in an embodiment of this application;
[0035] Figure 2 This is a schematic diagram of the second process of a face detection method provided in an embodiment of this application;
[0036] Figure 3 This is a schematic diagram of the third process of a face detection method provided in an embodiment of this application;
[0037] Figure 4 This is a schematic diagram of the prediction layer structure of a face detection method provided in an embodiment of this application;
[0038] Figure 5 This is a schematic diagram of the structure of a face detection system provided in an embodiment of this application;
[0039] Figure 6 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.
[0040] Explanation of reference numerals in the attached drawings: 501, receiving unit; 502, processing unit; 503, output unit; 600, electronic device; 601, processor; 602, communication bus; 603, user interface; 604, network interface; 605, memory. Detailed Implementation
[0041] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0042] In the description of the embodiments in this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.
[0043] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.
[0044] In this application, the facial features include four parts: the left eye, the right eye, the nose, and the mouth.
[0045] With the continuous development of deep learning, the field of object detection is also rapidly evolving. Face detection is a crucial area within this field. Because users often wear masks in public places, large numbers of people wearing masks are present in crowded public areas such as train stations, airports, and shopping malls. Public venue management agencies, based on safety requirements, conduct face detection on masked individuals in crowded passageways. Face detection refers to searching any given image using a specific strategy to determine if it contains a face, and if so, returning the face's location and size. Face detection requires a front-end camera to capture raw information and perform masked face detection. Currently, most object detection algorithms are designed for detecting large, sparsely distributed objects in natural scenes. Existing algorithms commonly employ two methods for object image detection: the first uses an R-CNN-based model that pre-generates candidate bounding boxes; the second uses a YOLO model that does not pre-generate bounding boxes. Existing algorithms only perform exceptionally well for images with clear, unobstructed images of people. However, when the distance between the face and the camera varies, resulting in different face sizes, and when people wear masks that obscure their faces, the existing algorithms are not specifically designed for occlusion. Therefore, when performing face detection on people, it can lead to the loss of people's features.
[0046] Therefore, how to detect faces when they are occluded, and how to supplement facial features during the detection process to reduce the inability to identify facial features due to occlusion, are key issues. This application provides a face detection method applied to a server. The server in this application can be a platform providing services to airports, train stations, shopping malls, and other similar locations. Figure 1 This is a schematic diagram of the first process of a face detection method provided in an embodiment of this application. (Refer to...) Figure 1 The method includes the following steps S101-S104.
[0047] Step S101: Obtain a monitoring image of a preset scene. The monitoring image includes a face image and a background image.
[0048] In the above steps, the preset scenarios include any one of the following: airports, train stations, and shopping malls, primarily targeting public places with high pedestrian traffic. This application uses a train station as an example of a preset scenario. In large passageways such as airports, train stations, and shopping malls, surveillance cameras are installed to monitor whether overcrowding or other safety incidents occur, ensuring the safety of people passing through. To ensure there are no safety hazards in these passageways, individuals entering the monitoring range are identified. The server acquires the station's surveillance video, obtaining the surveillance images from the video. These images include facial images and background images, with the background images including buildings, vehicles, luggage, and animals, etc.
[0049] Step S102: Use a face recognition algorithm to filter the surveillance images to obtain face images.
[0050] In the above steps, the server extracts facial images from the surveillance image using a facial recognition algorithm and deletes the background image. In this application, the facial recognition algorithm refers to detecting a face in the surveillance image and locating key facial feature points, then cropping the main facial area, preprocessing it, and finally feeding it back to the server for recognition.
[0051] Step S103: Determine whether the number of facial features is consistent with the preset number of facial features; the number of facial features is the number of facial features corresponding to any face image.
[0052] In the above steps, after the server obtains the number of facial features corresponding to the face image, the server determines whether the number of facial features is consistent with the preset number of facial features. The preset number of facial features is the key points of the face, including the left eye, right eye, nose, and mouth.
[0053] Step S104: If the number of facial features is inconsistent with the preset number of facial features, the face image is input into the preset occluded face detection model for detection. The preset occluded face detection model supplements the missing facial features in the face image to obtain the face detection result corresponding to the person, and outputs the face detection result.
[0054] In the above steps, when the number of facial features is inconsistent with the preset number of facial features, it is confirmed that the facial features of the current face image are occluded and the person information corresponding to the face image cannot be identified.
[0055] For example, the server retrieves a facial image of user A. The facial image contains 2 facial features, while the default number of facial features is 4. If the number of facial features for user A does not match the default number, it is confirmed that user A's facial image is occluded and the person corresponding to the facial image cannot be identified. The default number of facial features can be set according to the data conditions and is not limited here.
[0056] If the server cannot recognize the current face image, it inputs the current face image into a preset occluded face detection model for detection. Before inputting the face image into the preset occluded face detection model, a preset occluded face detection model is constructed. The preset occluded face detection model is constructed based on a low-resolution occluded face detection model. In subsequent embodiments, the preset occluded face detection model is represented by the low-resolution occluded face detection model.
[0057] Inputting images into a low-resolution occluded face detection model facilitates face detection for people wearing masks in crowded passageways. How to construct a low-resolution occluded face detection model is a key question. Figure 2 As shown, Figure 2 This is a schematic diagram of the second process of a face detection method provided in an embodiment of this application, with reference to... Figure 2 The method includes the following steps S201-S203.
[0058] Step S201: Use the MTCNN network to process the surveillance image to obtain face detection boxes.
[0059] In the above steps, the MTCNN network is used to filter the image to obtain face bounding boxes, that is, to scale the faces to be detected to obtain an image pyramid. MTCNN contains three sub-networks, namely P-Net, R-Net and O-Net. The first stage is to quickly generate a series of candidate windows through a shallow CNN network, that is, to input the candidate windows into the P-Net network to generate candidate windows and to fine-tune the generated candidate boxes. The second stage is to filter out most of the non-face candidate windows through a more powerful CNN network, that is, to input the non-face boxes into the R-Net network to delete non-face boxes and to fine-tune the positive samples. At this time, the positive samples refer to face boxes. The third stage is to find five marker points on the face through a more powerful network feature, that is, to input the O-Net network to further filter the subsequent windows and obtain accurate face detection boxes.
[0060] The total loss is calculated based on the loss functions of the three sub-networks in MTCNN. MTCNN comprises three sub-networks, each with its own corresponding loss function. Due to the different characteristics of the three sub-networks, different loss functions are used: P-Net uses the cross-entropy loss function, while R-Net and O-Net use the 2L loss function. Finally, the losses of these three parts are multiplied by their respective weights and summed to form the final total loss. The weights at this point can be set based on historical face detection data; no specific restrictions are imposed here. The purpose of obtaining the total loss is to minimize the error between the trained loss function and the true value, thereby obtaining accurate face detection boxes.
[0061] Step S202: Use the backbone network as a real-time semantic segmentation model to perform face semantic segmentation and extract face boxes from the face detection boxes.
[0062] In the above steps, the backbone network uses a Fast-SCNN network as a real-time semantic segmentation model to perform semantic segmentation of faces, obtaining images labeled with each pixel category. Then, a depth-first search algorithm is used to extract face bounding boxes to assist in detecting occluded faces. The Fast-SCNN network employs four modules: a learning downsampling module, a global feature extractor module, a feature fusion module, and a standard classifier. The learning downsampling module uses a three-layer structure, ensuring the effectiveness and efficiency of low-level feature sharing. The global feature extractor module captures the global context for image segmentation, i.e., segmenting the overall information of the image. The feature fusion module ensures effectiveness by simply fusing features from different branches. The standard classifier uses two depthwise separable convolutions (DSConv) and a pointwise convolution (Conv2D) to determine whether a face is present. All of these modules are constructed using depthwise separable convolutions.
[0063] Step S203: Use the residual module in the backbone network in combination with the SDD network to extract face parts, and train the face parts by key point localization to obtain the preset occluded face detection model.
[0064] In the above steps, a residual module is used as a front-end network connected to the SSD network to extract face components. After obtaining the face components, a key point localization task is added to locate the five key points of the face. After extracting the face bounding box, face component localization is performed, and the face is divided into five categories: left eye, right eye, nose, left corner of mouth, and right corner of mouth.
[0065] The residual module addresses the issue of increasing training error as network layers deepen. It's connected as a front-end network to the SSD network, which runs only a single convolutional neural network but uses multi-scale feature maps as input for detection. Furthermore, it employs various data augmentation methods to fuse high-level semantic features into lower-level semantics. These data augmentation methods include image cropping, rotation, and distortion. When acquiring images from surveillance cameras, different angles are used to capture images suitable for various surveillance perspectives.
[0066] A dataset was collected and organized to construct and train a low-resolution occluded face detection model. The MTCNN network, operating in a hierarchical cascade manner, can detect faces at different scales. The Fast-SCNN semantic segmentation network is used to assist in detecting occluded faces, improving the detection rate. After extracting the face bounding boxes, a keypoint localization task is added to better predict faces and further improve detection accuracy. Processing the entire face directly would reduce the size of the face bounding boxes, and the offset of the face bounding boxes would affect detection.
[0067] A low-resolution occluded face detection model is constructed and trained through the above process. The trained low-resolution occluded face detection model is then used to process surveillance images in a large passageway scenario, that is, to identify faces occluded by masks.
[0068] In one possible implementation, most object detection algorithms are designed for large object images that are sparsely distributed in natural scenes. When detecting small objects, i.e. when faced with a large number of small-scale face surveillance images in a preset scene, there will be cases of missed detection and false detection, resulting in low detection accuracy of small-scale objects.
[0069] At this point, the specific scenario for large-channel pedestrian traffic refers to a scenario where the channel width is ≥6m, the pedestrian density is ≥1000 people / minute, and the pedestrian speed is approximately 1.5m / s.
[0070] Therefore, how to reduce false positives and false negatives, and how to confirm that the facial features of the current face image are not occluded when the number of facial features matches the preset number, are all questions that need to be addressed. This requires acquiring dense face images and performing multi-size detection. A preset dense face detection model can be used for detection. Before using the preset dense face detection model, it is necessary to construct the preset dense face detection model.
[0071] By inputting images into a pre-defined dense face detection model, it is possible to conveniently detect faces among densely packed people in scenarios involving large passageways. How to construct a pre-defined dense face detection model is as follows: Figure 3 As shown, Figure 3 This is a schematic diagram of the third process of a face detection method provided in an embodiment of this application, with reference to... Figure 3 The method includes the following steps S301-S303.
[0072] Step S301: Use the K-means clustering algorithm to perform cluster analysis on the standard reference boxes to obtain the predicted boxes.
[0073] In the above steps, the Intersection over Union (IOU) between the predicted bounding box and the standard reference box is used as the loss function. The standard reference box refers to the ground truth bounding box. The K-means clustering algorithm is improved, and the improved distance function is defined as:
[0074] min∑i∑j1-IOU(Box i Truth j )
[0075] Where i represents the index of the predicted bounding box, j represents the index of the standard reference box, and Box i Truth represents the location information of the prediction box. j The first term represents the positional information of the standard reference box; the second term represents the Intersection over Union (IoU) between the predicted box and the standard reference box, an indicator of the degree of overlap between the two boxes. The meaning of this formula is: the larger the IoU between the predicted box and the standard reference box, the closer the predicted box is to the standard reference box, and the smaller the loss value; conversely, the smaller the IoU, the greater the difference between the predicted box and the standard reference box, and the larger the loss value. Therefore, the purpose of this loss function is to allow the model to learn more accurate predicted box positional information, thereby improving the accuracy of object detection.
[0076] When a pre-defined dense face detection model performs object detection tasks, in order to obtain feature maps with rich small-scale face features, standard reference boxes are generated for prediction. These standard reference boxes refer to a set of baseline boxes generated in the image, used for detecting and locating targets of different sizes and proportions. The improved K-means algorithm can be used to cluster these standard reference boxes, thereby obtaining prediction box scales suitable for the current dataset.
[0077] An improved K-means algorithm can be used to cluster standard reference boxes to obtain prediction box scales suitable for the current dataset. The specific steps are as follows: First, collect the width and height information of all objects in the training set as input data for the K-means algorithm. Randomly initialize K cluster centers, using either a uniform or Gaussian distribution. Next, for each input data point, calculate the distance between it and each cluster center, and select the cluster represented by the nearest cluster center. Update the positions of the cluster centers, moving them to the center of their respective clusters. Repeatedly calculate the distance between the input data and each cluster center, and update the positions of the cluster centers, until the positions of the cluster centers no longer change or a predetermined number of iterations is reached. Finally, based on the clustering results, select the box scale represented by each cluster center as the prediction box scale parameter for network initialization. By using the improved K-means algorithm, prediction box scale parameters suitable for the current dataset can be obtained, thereby improving the accuracy and efficiency of object detection.
[0078] Step S302: Process the monitoring image using the prediction box.
[0079] In the above steps, based on the predicted bounding box scale of the dataset, the monitoring image is acquired according to the initial predicted bounding box scale, thereby identifying the presence of personnel information in the monitoring image.
[0080] Step S303: Input the processed surveillance image into the improved YOLO dense face detection model for training to obtain the preset dense face detection model.
[0081] In the steps described above, the improved YOLO dense face detection model includes a feature extraction network and a prediction layer. The original YOLO algorithm uses a single-stage network with 16 convolutional layers and 4 pooling layers. The original YOLO algorithm divides the input image into an S×S grid, predicts B bounding boxes for each grid, and predicts C class scores, as well as the bounding box's position and size. Simultaneously, a convolutional neural network is used to extract features from the entire image, resulting in a feature map. This feature map is then combined with the grid information for object detection. Based on this, the structure of the feature extraction network and prediction layer is improved.
[0082] Improving the feature extraction network specifically involves: using a single-machine network to extract features to obtain a feature map; using spatial filters to reconstruct features from the feature map to obtain sub-feature maps; having multiple sub-feature maps; and then fusing the feature map with multiple sub-feature maps to obtain a multi-level feature map. Obtaining a multi-level feature map ensures that feature representations of different granularities participate in detection. After improving the feature extraction network, the resulting feature map contains richer small-scale facial features, which helps improve detection accuracy.
[0083] Feature extraction using a single-machine network yields a 28*28*512 resolution feature map. Four spatial filters with a stride of 2 are then used to reconstruct the feature map, resulting in four 14*14*512 resolution sub-feature maps. Fusing these multi-level feature maps allows feature representations of different granularities to participate in detection, resulting in a final feature output dimension of 14*14*3072. The extracted feature maps exhibit significant spatial redundancy; dimensionality reduction reduces the redundancy between adjacent pixels in the receptive field. Dimensionality reduction is an operation that transforms a single image into a high-dimensional dataset. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of the prediction layer structure of a face detection method provided in an embodiment of this application.
[0084] The improvement to the prediction layer structure specifically involves increasing the output layer resolution from 7*7 to 14*14, and refining the prediction box structure for each grid cell, resulting in an output tensor dimension of 14*14*18. The 14*14 grid divides the original image into 14*14 grids, each composed of an 18-dimensional vector, corresponding to three prediction boxes, each responsible for predicting face targets at different scales independently. In the improved prediction layer structure, a 1*1 convolutional kernel is used to compress the feature map, integrating information from different receptive fields to enhance the feature representation of small-scale faces. In a Convolutional Neural Network (CNN), the size of the input layer region corresponding to an element in the output of a given layer is called the receptive field. This improvement to the prediction layer results in 588 prediction boxes for the entire model, compared to only 98 for the original YOLO algorithm, further improving detection accuracy and addressing YOLO's less-than-ideal performance in detecting small targets.
[0085] We collected and organized the dataset, built and trained a pre-defined dense face detection model. Using an improved K-means clustering algorithm, we obtained more representative initialization parameters, reducing the bias and variance between the initial and optimal parameters, accelerating model convergence, and helping the model fine-tune the location information of the predicted bounding boxes more quickly during training.
[0086] The above steps are used to construct and train a pre-defined dense face detection model, and then the trained pre-defined dense face detection model is used to process surveillance images in scenarios where large crowds pass through a passageway.
[0087] In one possible implementation, a preset dense face detection model and a low-resolution occluded face detection model are fused to obtain a preset model. This preset model can detect masked faces in scenarios with large crowds passing through, capable of capturing dense face images and adapting to various sizes and masked face images. Alternatively, the preset dense face detection model and the low-resolution occluded face detection model can be merged to obtain the final preset model.
[0088] This application also provides a face detection system. Figure 5 This is a schematic diagram of the structure of a face detection system provided in an embodiment of this application, with reference to... Figure 5 The server includes a receiving unit 501, a processing unit 502, and an output unit 503.
[0089] Receiving unit 501: Acquires monitoring images of a preset scene, including face images and background images.
[0090] Processing unit 502: Uses a face recognition algorithm to filter the monitoring images to obtain face images; determines whether the number of facial features is consistent with the preset number of facial features; the number of facial features is the number of facial features corresponding to any face image.
[0091] Output unit 503: If the number of facial features is inconsistent with the preset number of facial features, the face image is input into the preset occluded face detection model for detection. The preset occluded face detection model supplements the missing facial features in the face image to obtain the face detection result corresponding to the person, and outputs the face detection result.
[0092] In one possible implementation, the receiving unit 501 is used to process the monitoring image using the MTCNN network to obtain a face detection box; the processing unit 502 is used to use the backbone network as a real-time semantic segmentation model to perform face semantic segmentation and extract the face box from the face detection box; the residual module in the backbone network is combined with the SDD network to extract face parts, and the face parts are trained for key point localization to obtain a preset occluded face detection model.
[0093] In one possible implementation, key point localization includes the left eye, right eye, nose, left corner of the mouth, and right corner of the mouth.
[0094] In one possible implementation, the processing unit 502 is used to perform face semantic segmentation using the Fast-SCNN network as a real-time semantic segmentation model in the backbone network to obtain images labeled with each pixel category; and to extract face boxes from the face detection boxes using a depth-first traversal algorithm.
[0095] In one possible implementation, the output unit 503 is used to input the face image into a preset dense face detection model for detection if the number of facial features is consistent with the preset number of facial features, to obtain the face detection result corresponding to the person, and to output the face detection result.
[0096] In one possible implementation, the receiving unit 501 is used to perform cluster analysis on the standard reference boxes using the K-means clustering algorithm to obtain predicted boxes; the processing unit 502 is used to process the monitoring image using the predicted boxes; and the processed monitoring image is input into the improved YOLO dense face detection model for training to obtain a preset dense face detection model.
[0097] In one possible implementation, the processing unit 502 for the improved YOLO dense face detection model includes a feature extraction network and a prediction layer.
[0098] It should be noted that the system provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0099] This application also discloses an electronic device. (See reference...) Figure 6 , Figure 6 This application provides a schematic diagram of the structure of an electronic device. The electronic device 600 may include: at least one processor 601, at least one network interface 604, a user interface 603, a memory 605, and at least one communication bus 602.
[0100] The communication bus 602 is used to enable communication between these components.
[0101] The user interface 603 may include a display screen and a camera. Optionally, the user interface 603 may also include a standard wired interface and a wireless interface.
[0102] The network interface 604 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).
[0103] The processor 601 may include one or more processing cores. The processor 601 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 605, and by calling data stored in the memory 605. Optionally, the processor 601 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 601 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and application requests; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 601 and may be implemented as a separate chip.
[0104] The memory 605 may include random access memory (RAM) or read-only memory. Optionally, the memory 605 may include a non-transitory computer-readable storage medium. The memory 605 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 605 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 605 may also be at least one storage device located remotely from the aforementioned processor 601.
[0105] like Figure 6 As shown, the memory 605, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a face detection application.
[0106] exist Figure 6In the electronic device 600 shown, the user interface 603 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 601 can be used to call the face detection application stored in the memory 605. When executed by one or more processors, the electronic device performs one or more of the methods described in the above embodiments.
[0107] An electronic device readable storage medium stores instructions that, when executed by one or more processors, cause the electronic device to perform one or more of the methods described in the above embodiments.
[0108] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0109] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0110] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some service interfaces; indirect couplings or communication connections between devices or units may be electrical or other forms.
[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.
[0114] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.
Claims
1. A face detection method, characterized in that, When applied to a server, the method includes: Acquire surveillance images of a preset scene, the surveillance images including face images and background images; The surveillance images are filtered using a facial recognition algorithm to obtain the facial images; Determine whether the number of facial features is consistent with the preset number of facial features; the number of facial features is the number of facial features corresponding to any one of the face images. If the number of facial features is inconsistent with the preset number of facial features, the face image is input into a preset occluded face detection model for detection. Before inputting the face image into the preset occluded face detection model for detection, the preset occluded face detection model is constructed, specifically including: processing the monitoring image using an MTCNN network to obtain a face detection box; using a backbone network as a real-time semantic segmentation model to perform face semantic segmentation and extracting face boxes from the face detection boxes; using a residual module combined with an SDD network in the backbone network to extract face components, and training the face components for key point localization to obtain the preset occluded face detection model. The preset occluded face detection model supplements the missing facial features in the face image to obtain the face detection result corresponding to the person, and outputs the face detection result; if the number of facial features is consistent with the preset number of facial features, the face image is input into the preset dense face detection model for detection to obtain the face detection result corresponding to the person, and outputs the face detection result. The preset dense face detection model is used to perform face detection on densely packed people in a large passageway scene.
2. The method according to claim 1, characterized in that, The key points for location include the left eye, right eye, nose, left corner of the mouth, and right corner of the mouth.
3. The method according to claim 1, characterized in that, The process of using a backbone network as a real-time semantic segmentation model for face semantic segmentation, and extracting face bounding boxes from the face detection boxes, specifically includes: The backbone network uses Fast-SCNN as a real-time semantic segmentation model to perform face semantic segmentation, and obtains images labeled with each pixel category. The face bounding box is extracted from the face detection box using a depth-first traversal algorithm.
4. The method according to claim 1, characterized in that, Before inputting the face image into a preset dense face detection model for detection, the method further includes: constructing the preset dense face detection model, specifically including: The K-means clustering algorithm was used to perform cluster analysis on the standard reference boxes to obtain the predicted boxes; The monitoring image is processed using the prediction box; The processed surveillance image is input into the improved YOLO dense face detection model for training to obtain the preset dense face detection model.
5. The method according to claim 4, characterized in that, The improved YOLO dense face detection model includes a feature extraction network and a prediction layer.
6. A face detection system, characterized in that, The system is a server, which includes a receiving unit, a processing unit, and an output unit. The receiving unit is used to acquire a monitoring image of a preset scene, the monitoring image including a face image and a background image; The processing unit is used to filter the surveillance images using a face recognition algorithm to obtain the face images; Determine whether the number of facial features matches the preset number of facial features; The number of facial features is the number of facial features corresponding to any one of the facial images; The output unit is used to input the face image into a preset occluded face detection model for detection if the number of facial features is inconsistent with the preset number of facial features. Before inputting the face image into the preset occluded face detection model for detection, the preset occluded face detection model is constructed, specifically including: processing the monitoring image using an MTCNN network to obtain face detection boxes; using a backbone network as a real-time semantic segmentation model to perform face semantic segmentation and extract face boxes from the face detection boxes; and using a residual module combined with an SDD network in the backbone network to extract face components. The face component is trained using key point localization to obtain the preset occluded face detection model. The preset occluded face detection model supplements the missing facial features in the face image to obtain the face detection result corresponding to the person, and outputs the face detection result. If the number of facial features is consistent with the preset number of facial features, the face image is input into the preset dense face detection model for detection to obtain the face detection result corresponding to the person, and outputs the face detection result. The preset dense face detection model is used to detect faces of densely packed people in a large passageway scene.
7. An electronic device, characterized in that, The device includes a processor (601), a memory (605), a user interface (603), and a network interface (604). The memory (605) is used to store instructions. The user interface (603) and the network interface (604) are used to communicate with other devices. The processor (601) is used to execute the instructions stored in the memory (605) to cause the electronic device (600) to perform the method as described in any one of claims 1-5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-5.