Fast and accurate face detection system for long-distance detection

By using lower resolution images and multi-threshold evaluation methods on low-end devices, combined with head recognition technology, the shortcomings in the computing resources and resolution of facial detection in the prior art are solved, and efficient facial detection effect on low-end devices is achieved.

CN115769278BActive Publication Date: 2025-04-25HP (CHONGQING) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202080099843.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-05-06
Publication Date
2025-04-25
Estimated Expiration
2040-05-06

AI Technical Summary

Technical Problem

The existing CNN-based facial detection technology has high requirements in computing resources and is difficult to operate effectively on low-end devices, especially when the resolution is low and the individual distance is far away, it is difficult to achieve high confidence facial detection.

Method used

Lower resolution images are used for face detection, and the detection results are evaluated by setting high thresholds and low thresholds, accepting results above high thresholds, discarding results below low thresholds, and additional processing is performed on results between the two thresholds. For areas where the head size does not meet the criteria, head recognition technology is used to process it with higher resolution images to improve detection accuracy.

Benefits of technology

Without significantly increasing the use of computing resources, the effect of facial detection is improved, allowing low-end devices to provide a better facial detection experience at longer distances and at lower resolutions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115769278B_ABST
    Figure CN115769278B_ABST
Patent Text Reader

Abstract

Face detection is performed using the lower resolution image and the results are evaluated against a high threshold and a low threshold. Results above the high threshold are accepted and results below the low threshold are discarded. Results between the two thresholds are subjected to additional processing. For candidates, instead of using face detection, head recognition is used. If the head size is above a limit, head recognition is performed. If the head size is below the limit, a higher resolution version of the region is used for head recognition. Results of the head recognition are evaluated against a threshold and for results above the threshold, the original face detection result is accepted and for results below the threshold, the original face detection result is discarded.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to face detection in video streams. Background Art

[0002] Room-based video conferencing systems often use face detection for group composition and active speaker focus. With the advent of convolutional neural networks (CNNs), much effort has been expended to develop CNN-based solutions for face detection. One drawback of CNNs is the need for significant computational power, especially as the networks get deeper. For many low-end products, the computational power does not exist or is unavailable due to competing demands. Therefore, simpler CNNs must be used along with lower resolutions that are lower than the actual camera resolution, including the results. The lower resolution makes the face size smaller (especially when the person is far from the camera), increasing the difficulty for the CNN to perform a high-confidence determination. Summary of the invention

[0003] The preferred embodiment performs face detection with a lower resolution image and evaluates the results against a high threshold and a low threshold. Results above the high threshold are accepted and results below the low threshold are discarded. Results between the two thresholds receive additional processing. For candidates, face detection is not used, but head recognition is used instead. If the head size is above a limit, head recognition is performed. If the head size is below a limit, a higher resolution version of the region is used for head recognition. The results of head recognition are evaluated against the thresholds, and for results above the threshold, the original face detection results are accepted, and for results below the threshold, the original face detection results are discarded.

[0004] This two-step process improves facial detection without using significantly more computing resources, enabling lower-end devices to provide a better overall experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a block diagram illustrating components of a video conferencing terminal according to one embodiment of the present invention.

[0006] Figure 2 is a flow chart of the operation according to one embodiment of the present invention.

[0007] Figure 3 is an illustration of a CNN for performing face detection in one embodiment according to the present invention.

[0008] Figure 3A yes Figure 3 Illustration of a convolutional layer of a CNN with rectified linear units.

[0009] Figure 3B yes Figure 3 Illustration of the Inception layer of a CNN.

[0010] Figure 3C is with Figure 3 Illustration of anchor densification used with CNN.

[0011] Figure 4 is a diagram of a CNN for performing head recognition in one embodiment according to the present invention.

[0012] Figure 5 is based on Figure 2 Flowchart and use Figure 3 Picture of the result of CNN face detection.

[0013] Figure 6 is based on Figure 2 Flowchart and use Figure 4 Picture of the result of CNN head recognition. Figure 4 Picture of the result of CNN face detection. DETAILED DESCRIPTION

[0014] Figure 1 1 is a block diagram of a video conferencing terminal 100 according to one embodiment. A processing unit 102 (such as a DSP or a central processing unit (CPU) or a combination thereof) is present to perform desired audio operations and video operations. The facial detection technology and head recognition technology described below allow the processing unit 102 to be a low processing power unit. In one embodiment, the processing unit 102 is a Qualcomm Snapdragon processor. 625CPU. (SNAPDRAGON is a registered trademark of Qualcomm Incorporated) Memory 104 is connected to processing unit 102 to allow processing unit 102 to access modules and execute various instructions to provide the described functions, and the memory includes a non-temporary area for storing programs or instructions to execute the desired modules 106 (such as face detection modules, head recognition modules, motion detection modules, and audio processing modules and various video modules). Memory 104 further contains image memory 107 to store images to be processed by processing unit 102, which are stored in a temporary portion of memory 104. Although shown as a bus type connection, any type of connection may be used, including a point-to-point connection.

[0015] A network interface 108, such as an Ethernet interface, is connected to the processing unit 102 to allow communication with one or more remote terminals 140 in a video conference. An input / output (I / O) interface 110 is connected to the processing unit 102 to perform any required I / O operations. An A / D converter block 112 is connected to the processing unit 102 and to a microphone 114. The microphone 114 includes an omnidirectional microphone and one or more directional microphones. Similarly, at least two speakers 122 are connected to the processing unit 102 via an A / D converter 113. A camera 116 is connected to the processing unit 102 and the memory 104 to provide near-end video and store video images in the image memory 107 for processing by the processing unit 102. High Definition Multimedia Interface An HDMI or other type of video interface 118 is connected to the processing unit 102 and to an external display 120 to provide video and / or audio output. (HDMI is a registered trademark of HDMI Licensing, LLC) Figure 1 is a very simplified diagram of a video conferencing system terminal, and many other designs are possible, all of which can perform the face detection operations and head recognition operations described below.

[0016] The video conferencing terminal 100 communicates with one or more remote terminals 140 across one or more networks 130 .

[0017] After capturing audio and video with microphone 114 and camera 116, terminal 100 encodes the audio and video using any common coding standard, such as MPEG-1, MPEG-2, MPEG-4, H.261, H.263, H.264, G729 and G711. Then, network interface 108 outputs the encoded audio and video to remote terminal 140 via network 130 using any appropriate protocol. Similarly, network interface 108 receives conference audio and video from remote terminal 140 via network 130, and sends these conference audio and video to codec for processing. Finally, speaker 122 outputs conference audio, and display 120 outputs conference video. In one embodiment, display 120 is an external display, such as a monitoring display, coupled to video conferencing terminal 100 via HDMI or other wired or wireless technology. Many of these modules and other components can operate in a conventional manner known in the art, so further details are not provided here.

[0018] refer to Figure 2, flow chart 200 illustrates face detection according to one embodiment of the present invention. In step 202, camera data is received at two resolutions: a lower resolution and a higher resolution. Although the camera 116 may be a 4k camera, the processing for face detection is done using a lower resolution in order to save computational bandwidth for other operations in the terminal 100. The program contained in the memory 104 includes instructions for the processing unit 102 to downsample the video image to a desired lower resolution image. In this embodiment, 720p resolution and 1080p resolution are used for face detection, 720p being the lower resolution and 1080p being the higher resolution. In step 204, the low resolution video is ready to be submitted to the CNN. In step 206, the CNN performs face detection. In this embodiment, the CNN is Figure 3 2 and described in more detail below. In step 208, the output of the face detector CNN 300 is received. The output is in the form of bounding boxes around the candidate faces and confidence scores for each bounding box. In step 210, each of the confidence scores is checked. If the confidence score is below a lower face detection threshold, such as 0.4, then in step 212, the result is considered negative and discarded. If the confidence score is above a higher face detection threshold, such as 0.6, then in step 214, the result is considered positive and the bounding box is provided as the result in step 216.

[0019] If the confidence score is between the lower face detection threshold and the upper face detection threshold, the operation proceeds to step 218 for further processing. Confidence scores in the middle range are typically caused by high-angle head positions, small-sized images, and blurred facial details. All of these situations are more suitable for head recognition that is subsequently performed. In step 218, since the head is larger than the face, the size of the bounding box is increased, such as by a factor of 1.2. In step 220, it is determined whether the image within the larger bounding box is greater than or equal to a given or predetermined network input size (NIS), which in this embodiment is 48×48 pixels. The larger the image, the better, but in this embodiment, 48×48 is the input size of the CNN used for head recognition, so any larger images are cropped to 48×48 pixels. If greater than or equal to the NIS, then in step 222, the image is provided to a head recognition CNN, such as Figure 42 and described in more detail below. The output of the head recognizer CNN 400 is a confidence score for head recognition. In step 224, the output of the head recognizer CNN 400 is evaluated. If the confidence score output is above a head recognition threshold, such as 0.7, then in step 226, a face detection bounding box is provided as the result. If it is below the head recognition threshold, then in step 228, the result is set to negative and the bounding box is discarded.

[0020] If the image size is less than NIS in step 220, a higher resolution image is obtained in step 230. In step 232, the equivalent position of the lower resolution bounding box is determined, and in step 234, the relevant pixels representing a possible head in the higher resolution image are sampled and provided to the head recognizer CNN 400. In step 236, the output of the head recognizer CNN 400 is evaluated. If it is below the head recognition threshold, the operation proceeds to step 228 and the bounding box is discarded. If it is above the head recognition threshold, then in step 238, the bounding box of the lower resolution image is provided as the result.

[0021] The results of flowchart 200 are provided to the video module to direct camera 116 as needed, such as to frame a group or a particular individual who is speaking.

[0022] Thus, a lower resolution image is initially used to determine the presence of a face. If the result is above a higher confidence threshold, the location is accepted. If the result is below a lower confidence threshold, the location is rejected. For intermediate values, head recognition is performed. If the enlarged bounding box produces an image that exceeds the expected size, head recognition is performed. If not, a higher resolution image is used to obtain the relevant area and head recognition is performed. Head recognition is used to provide a second check on the face detection output. As the image size becomes smaller, higher resolution images are used to provide a second check because the individuals are farther away. Therefore, higher resolution images are only used in certain small image conditions where existing systems would have determined that a face was not present, primarily due to lack of resolution.

[0023] By setting multiple thresholds and performing additional processing only in edge cases, overall face detection is improved with very little additional computational resources. In one embodiment, face detection distance is improved from 4 meters to 7 meters, and in another embodiment, face detection distance is improved from 4-5 meters to 8-10 meters. This improved detection distance is achieved with a minor increase in computational effort since additional computation is only performed in uncertain cases, and even in these cases, the higher resolution processing is limited to specific longer distance cases. The huge improvement in face detection distance easily justifies the slight additional computational effort.

[0024] Figure 3 is an illustration of a face detector CNN 300 used in one embodiment. The architecture is from the paper by S. Zhang, X. Zhu, Z. Lei, H. Shi, X. Wang, and S. Z. Li, "FaceBoxes: A CPU real-time face detector with high accuracy," 2017 IEEE International Joint Conference on Biometrics (IJCB), Denver, Colorado, 2017, pages 1-9, which is hereby incorporated by reference herein. The paper is available from IEEE or via the URL arxiv.org / pdf / 1708.05234.pdf. It is recommended to refer to the paper for further explanation of the face detector. The input image is provided to a fast digest convolutional layer (RDCL) block 302 and then to a multi-scale convolutional layer (MSCL) block 304. The RDCL block 302 has a first layer 306 that receives the image. The first layer 306 is a combination convolution and rectified linear unit (ReLU), called a C.ReLU module, such as Figure 3A 306 is 7×7×24 and has a stride of 4. The second layer 308 is a pooling layer and is 3×3 with a stride of 2. The third layer 310 is a second C.ReLU module, although this time it is 5×5×64 with a stride of 2. The fourth layer 312 is a second pooling layer and is also 3×3 with a stride of 2.

[0025] The output of the fourth layer 312 is provided to the fifth layer 314, which is an Inception module, such as Figure 3B 31 and described below. The sixth layer 316 (second Inception module) follows the fifth layer 314. The seventh layer 318 (third Inception module) follows the sixth layer 316. The seventh layer 318 is followed by the eighth layer 320, which is a 1×1×128 convolutional layer with a stride of 1. The ninth layer 322 (another convolutional layer) follows the eighth layer 320. The ninth layer 322 is a 3×3×256 with a stride of 2. The tenth layer 324 follows the ninth layer 322, which is another 1×1×128 convolutional layer with a stride of 1. The eleventh layer 326 follows the tenth layer 324, which is another 3×3×256 convolutional layer with a stride of 2.

[0026] The outputs of the seventh layer 318 (third Inception module), the ninth layer 322 (convolutional layer), and the eleventh layer 326 (convolutional layer) are provided to a multi-task loss module 328, which performs a SoftmaxLoss function and a SmoothL1Loss function on the three outputs. Each of the three outputs is a reduced size from the previous output to allow detection of faces of different sizes. Since faces may appear at different locations in the image, Figure 3C The anchor densification shown and described below is used to increase the probability that an anchor overlaps a face. The output of the multi-task loss module 328 is the input image with bounding boxes located on possible faces and confidence scores for each bounding box. These are the confidence scores evaluated in step 210.

[0027] Figure 3A A C.ReLU module 340 is shown as used in the RDCL block 302. The first layer 342 is a convolutional layer. The output of the first layer 342 is provided to the second layer 344, which is a BatchNorm layer. The output of the second layer 344 is provided to the third layer 346, which is a negation layer that simply multiplies the output of the second layer 344 by -1. The output of the second layer 344 and the output of the third layer 346 are provided to the fourth layer 348, which is a concatenation layer. The output of the fourth layer 348 is provided to the fifth layer 350, which is a scaling layer. The output of the fifth layer 350 is provided to the final sixth layer 352, which is a ReLU layer.

[0028] Figure 3B The Inception module 370 used in the MSCL block 304 is shown. The first layer 372 represents the layer before the Inception module 370. Figure 2In the case of , for the fifth layer 314 (the first Inception module), the layer is the fourth layer 312 (the second pooling layer). The output of the first layer 372 is provided to: the second layer 374, which is a 1×1×32 convolutional layer; the third layer 376, which is a 3×3 pooling layer; the fourth layer 378, which is a 1×1×24 convolutional layer; and the fifth layer 380, which is a 1×1×24 convolutional layer. The output of the third layer 376 is provided to the sixth layer 382, ​​which is a 1×1×32 convolutional layer. The output of the fourth layer 378 is provided to the seventh layer 384, which is a 3×3×32 convolutional layer. The output of the fifth layer 380 is provided to the eighth layer 386, which is a 3×3×32 convolutional layer. The output of the eighth layer 386 is provided to the ninth layer 388, which is a 3×3×32 convolutional layer. The outputs of the second layer 374, the sixth layer 382, ​​the seventh layer 384, and the ninth layer 388 are provided to the tenth layer 390, which is a concatenation layer. The tenth layer 390 provides the output of the Inception module 370.

[0029] Figure 3C Figure 2 shows the anchor densification as described in the FaceBoxes paper cited above. Reference to that paper is recommended for further explanation. 密度 = 1 shows the base case without anchor densification. 密度 = 2 shows a double densification, where four copies of the anchor for density = 1 are used, as shown by the different line types. 密度 = 3 shows a threefold densification, where nine copies of the anchor are used for density = 1, shown by the different line types. This anchor densification solves the problem of smaller faces appearing in different locations in the larger image, which require smaller anchors.

[0030] The face detector CNN 300 is trained using the WIDER FACE dataset available at shuoyang1213.me / WIDERFACE / index.html. The WIDER FACE dataset is a face detection benchmark dataset, in which images are selected from the publicly available WIDER dataset (available at yjxiong.me / event_recog / WIDER / ). The WIDER FACE dataset has 32,203 images and 393,703 labeled faces, with high variability in scale, pose, and occlusion as depicted in the sample images. The WIDER FACE dataset is organized based on 61 event categories. For each event category, data is randomly selected as training, validation, and test sets at 40% / 10% / 50%. As an optimizer, the learning rate is 0.004 and the momentum is 0.9. Quantization is performed after training to improve latency.

[0031] Figure 4A head landmark identifier CNN 400 used in one embodiment is shown. The O-Net architecture used is from the paper "Joint Face Detection and Alignment Usingmultitask Cascaded convolutionary Networks" by K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, IEEE Signal Processing Letters, Vol. 23, No. 10, pp. 1499-1503, October 2016, which is hereby incorporated by reference. The paper can be obtained through IEEE or at the URL arxiv.org / ftp / arxiv / papers / 1604 / 1604.02878.pdf. It is recommended to refer to the paper for further explanation of the detector. An input image 402 with a size of 48×48 pixels is provided to the first convolution / max pooling stage 404. In the first convolution / max pooling stage 404 of 23×32×32, the convolution layer is 3×3 with a stride of 1, and the max pooling layer is 3×3 with a stride of 2. The output of the first convolution / max pooling stage 404 is provided to a second convolution / max pooling stage 406 of 10×10×64, the convolution layer is 3×3 with a stride of 1, and the max pooling layer is 3×3 with a stride of 2. The output of the second convolution / max pooling stage 406 is provided to a third convolution / max pooling stage 408 of 4×4×64, the convolution layer is 3×3 with a stride of 1, and the max pooling layer is 2×2 with a stride of 2. The output of the third convolution / max pooling stage 408 is provided to a 2×2 convolution layer 410 of 3x3x128 with a stride of 1. The output of the convolutional layer 410 is provided to a fully connected layer 412 having 256 nodes and a binary output vector. The two values ​​of the output vector represent the head recognition confidence score and are used in decision steps 224 and 236.

[0032] The head recognizer CNN400 is trained using the CrowdHuman dataset available at www.crowdhuman.org. The CrowdHuman dataset contains 15,000 images, 4,370 images, and 5,000 images for training, validation, and testing, respectively. There are a total of 470,000 human instances in the training and validation subsets, and an average of 23 people per image, with various types of occlusions in the dataset. Each human instance is annotated with a head bounding box, a human visible area bounding box, and a human full body bounding box. The learning rate is 0.001, and the Adam algorithm is used as the optimizer.

[0033] exist Figure 5 and Figure 6An example of the operation of an embodiment is provided in . In this example, the lower threshold is set to 0.4 and the upper threshold is set to 0.6. Processing the 720p image through the face detector CNN 300 results in four faces being labeled.

[0034] refer to Figure 5 , face ① has a confidence score of 0.85, which is greater than the higher threshold, and is therefore considered to be a frontal face.

[0035] Face ② has a confidence score of 0.45, which is less than the upper threshold but greater than the lower threshold, so that face ② is processed by the head recognizer CNN 400.

[0036] Face ③ has a confidence score of 0.72, which is greater than the upper threshold, and is therefore considered to be a frontal face.

[0037] Face ④ has a confidence score of 0.53, which is less than the upper threshold but greater than the lower threshold, so that face ④ is also processed by the head recognizer CNN 400.

[0038] Go to Figure 6 , two uncertain face candidates ② and ④ are passed from the face detector CNN 300. The head position is obtained by enlarging the bounding box with a scaling factor of 1.2. Face ② has a head size smaller than NIS (48x48), so the head region is sampled from the corresponding high-resolution 1080p image, resized to NIS, and then processed by the head recognizer CNN 400. The score of head recognition is 0.8, which is greater than the head recognition threshold of 0.7, indicating successful head recognition. Therefore, face ② is considered to be a frontal face.

[0039] Face ④ has a head size larger than NIS, so the head image is cropped according to the head bounding box directly from the 720p image, resized to match NIS, and then processed by the head recognizer CNN 400. The score of head recognition is 0.85, which is greater than the head recognition threshold of 0.7, indicating successful head recognition. Therefore, face ④ is considered to be a frontal face.

[0040] It should be understood that the above-mentioned CNN, FaceBoxes and O-Net can be replaced by other face detection and head recognition networks. In addition, head recognition can be replaced by torso / upper body recognition.

[0041] While the above description has a CNN executed on processing unit 102 , it should be understood that auxiliary neural network hardware or hardware modules may be used to increase operating speed and offload processing unit 102 .

[0042] Using two different thresholds, where images below a lower threshold are rejected as faces and images above a higher threshold are accepted, and processing images between the lower and higher thresholds for head recognition (including replacing the image with a higher resolution image when necessary and then evaluating the head recognition results), faces can be reliably detected at greater distances with only a slight increase in computing power utilization compared to the case where only a single threshold is used.

[0043] Although the above description is based on the operation of the video conferencing terminal and the images from the camera 116, it should be understood that Figure 2 The operation can be performed on any image stored in any of the temporary memory or non-temporary memory, and is not just performed on real-time video images. In fact, it can be understood that the training of CNN is performed using stored and non-real-time images.

[0044] The above description is illustrative and non-restrictive. For those skilled in the art, many variations of the present invention will become apparent after reviewing this disclosure. Therefore, the scope of the present invention should not be determined with reference to the above description, but with reference to the full scope of the attached claims and their equivalents.

Claims

1. A method for face detection, the method comprising: receiving an image at a first resolution; performing face detection on the image and providing face detection confidence scores for candidate faces; evaluating the provided face detection confidence score relative to a lower threshold and an upper threshold; discarding candidate faces having face detection confidence scores below the lower threshold; accepting candidate faces having a face detection confidence score above the upper threshold; performing head recognition on candidate faces having face detection confidence scores above the lower threshold and below the upper threshold, and providing a head recognition confidence score for each candidate face; evaluating the provided head identification confidence score relative to a third threshold; discarding candidate faces having head recognition confidence scores below the third threshold; and Candidate faces having head recognition confidence scores above the third threshold are accepted.

2. The method of claim 1, wherein performing the face detection further provides a bounding box for each candidate face, the method further comprising: Before performing head recognition, the bounding box of each candidate face is enlarged.

3. The method according to claim 2, further comprising: Before performing head recognition, determining the size of the image in each enlarged bounding box; comparing the size of the image within each enlarged bounding box to a predetermined size; When the size exceeds the predetermined size, cropping the image in the enlarged bounding box to the predetermined size; and Provides cropped images for head recognition.

4. The method according to claim 3, further comprising: receiving the image at a second higher resolution; When the size of the image in the enlarged bounding box is below the predetermined size, sampling the image in the enlarged bounding box from a second higher resolution image; as well as The sampled images are provided for head recognition. The method according to claim 1 , wherein the method is performed in a video conferencing terminal.

6. A non-transitory computer readable medium storing instructions which, when executed by a processor, cause the processor to perform a method of face detection, the method comprising: receiving an image at a first resolution; performing face detection on the image and providing face detection confidence scores for candidate faces; evaluating the provided face detection confidence score relative to a lower threshold and an upper threshold; discarding candidate faces having face detection confidence scores below the lower threshold; accepting candidate faces having a face detection confidence score above the upper threshold; performing head recognition on candidate faces having face detection confidence scores above the lower threshold and below the upper threshold, and providing a head recognition confidence score for each candidate face; evaluating the provided head identification confidence score relative to a third threshold; discarding candidate faces having head recognition confidence scores below the third threshold; and Candidate faces having head recognition confidence scores above the third threshold are accepted.

7. The non-transitory computer-readable medium of claim 6, wherein the performing the face detection further provides a bounding box for each candidate face, the method further comprising: Before performing head recognition, the bounding box of each candidate face is enlarged.

8. The non-transitory computer readable medium of claim 7, the method further comprising: Before performing head recognition, determining the size of the image in each enlarged bounding box; comparing the size of the image within each enlarged bounding box to a predetermined size; When the size exceeds the predetermined size, cropping the image in the enlarged bounding box to the predetermined size; and Provides cropped images for head recognition.

9. The non-transitory computer readable medium of claim 8, the method further comprising: receiving the image at a second higher resolution; When the size of the image in the enlarged bounding box is below the predetermined size, sampling the image in the enlarged bounding box from a second higher resolution image; as well as The sampled images are provided for head recognition.

10. The non-transitory computer-readable medium of claim 6, wherein the non-transitory computer-readable medium and the processor are used in a video conferencing terminal.

11. A computing device, comprising: an image memory for storing images at a first resolution; a processor coupled to the image memory; A non-transitory memory coupled to the processor for storing program instructions, the instructions causing the processor to perform a method of face detection, the method comprising: performing face detection on the stored images and providing face detection confidence scores for candidate faces; evaluating the provided face detection confidence score relative to a lower threshold and an upper threshold; discarding candidate faces having face detection confidence scores below the lower threshold; accepting candidate faces having a face detection confidence score above the upper threshold; performing head recognition on candidate faces having face detection confidence scores above the lower threshold and below the upper threshold, and providing a head recognition confidence score for each candidate face; evaluating the provided head identification confidence score relative to a third threshold; discarding candidate faces having head recognition confidence scores below the third threshold; and Candidate faces having head recognition confidence scores above the third threshold are accepted.

12. The computing device of claim 11, wherein the performing the face detection further provides a bounding box for each candidate face, the method further comprising: Before performing head recognition, the bounding box of each candidate face is enlarged.

13. The computing device of claim 12, the method further comprising: Before performing head recognition, determining the size of the image in each enlarged bounding box; comparing the size of the image within each enlarged bounding box to a predetermined size; When the size exceeds the predetermined size, cropping the image in the enlarged bounding box to the predetermined size; and Provides cropped images for head recognition.

14. The computing device of claim 13, the memory for storing images at a first resolution further for storing the images at a second, higher resolution, the method further comprising: When the size of the image in the enlarged bounding box is below the predetermined size, sampling the image in the enlarged bounding box from a second higher resolution image; as well as The sampled images are provided for head recognition.

15. The computing device according to claim 14, wherein the computing device is a video conferencing terminal, and the computing device further comprises: a camera coupled to the image memory for storing the image at a third resolution higher than the second higher resolution, and Wherein the method further comprises: From the image stored at the third resolution, the image is developed at a first higher resolution and a second higher resolution, and the first resolution image and the second resolution image are stored in the image memory.

16. The computing device according to claim 14, wherein the computing device is a video conferencing terminal, and the computing device further comprises: a camera coupled to the image memory for storing the image at a third resolution higher than the first resolution, and Wherein the method further comprises: From the image stored at the third resolution, the image is developed at the first resolution and the first resolution image is stored in the image memory.

17. A computer program product comprising a computer program, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Human face recognition method and device

    CN108875502A

  • A face detection method based on cascade convolution neural network structure

    CN109145854A