A lightweight face detection method and system based on a smart TV camera

Abstract

This invention discloses a lightweight face detection method and system based on a smart TV camera. The method includes: scaling up the original image captured by the smart TV camera and using it as input to a face detection model; performing ordinary convolution processing on the scaled original image to obtain a first feature map; performing several single-loop wide-field-of-view depth-separable convolution operations and several double-loop wide-field-of-view depth-separable convolution operations on the first feature map to obtain a dense feature map; selecting two dense feature maps from the obtained dense feature map, selecting anchor points, and setting pre-selected bounding boxes for regression to obtain the face location information. This invention employs two wide-field-of-view convolution operations, depth convolution, which can accelerate network feature extraction and greatly reduce the computational cost of regressing face bounding boxes, thereby improving the efficiency of face location recognition.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a lightweight face detection method and system based on a smart TV camera. Background Technology

[0002] With the development of science and technology, more and more traditional home appliances are evolving towards intelligentization. New features such as seamless interaction, intelligent recommendations, and intelligent playback are being incorporated into more and more smart TVs. Face detection, as a technological prerequisite, provides a guarantee for subsequent user profiling, key point localization, and other algorithms, ensuring accurate and fast face detection.

[0003] In existing technologies, face detection algorithms are mainly divided into two types: feature-based image classification and deep learning based on convolutional networks. Feature-based image classification algorithms have high requirements for the detection environment and relatively low accuracy, failing to meet the detection requirements of diverse and distant family scenes. Deep learning methods based on convolutional neural networks, relying on massive amounts of training data, can achieve extremely high accuracy in complex scenes. However, traditional deep learning methods suffer from numerous model parameters and slow inference speeds, making it difficult to meet real-time requirements on television devices with relatively limited computing power.

[0004] Therefore, there is an urgent need for a method that can quickly detect faces on smart TVs. Summary of the Invention

[0005] The purpose of this invention is to provide a lightweight face detection method based on a smart TV camera, in order to solve the problem mentioned in the background art that due to the limited computing power of smart TVs, traditional deep learning methods consume too much computing resources and cannot meet the real-time requirements of actual projects.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A lightweight face detection method based on a smart TV camera includes:

[0008] S1. Scale the original image captured by the smart TV camera and use it as input to the face detection model;

[0009] S2. Perform ordinary convolution processing on the original image after scaling in S1 to process the scaled original image into the first feature map.

[0010] S3. Perform several single-loop wide-field depth-separable convolution operations on the first feature map obtained in S2 to locate the features of some distant faces and obtain the second feature map.

[0011] S4. Perform several double-loop wide-field depth-separable convolution operations on the second feature map obtained in S3 to capture facial features when the distance is close and the target is large, and obtain a dense feature map.

[0012] S5. Select two dense feature maps from the dense feature maps obtained in S4, select anchor points, and set pre-selection boxes for regression to obtain the location information of the face.

[0013] A further technical solution is as follows: In S2, two ordinary convolution processes are performed, and the size of the first feature map is 64x64x48.

[0014] A further technical solution is as follows: In S3, the number of times the single-loop wide field-of-view depth-separable convolution operation is performed is five, and the size of the second feature map is 32x32x48.

[0015] A further technical solution is as follows: In S3, the single-loop wide field-of-view depth-separable convolution operation is composed of the results of 7x7 depth convolution and 1x1 point convolution, which are superimposed with the results of max pooling, and then output through the ReLU6 activation function.

[0016] A further technical solution is as follows: In S4, the number of times the double-loop wide field-of-view depth-separable convolution operation is performed is six, and the size of the dense feature map is 8x8x96.

[0017] A further technical solution is as follows: In S4, the dual-loop wide field-of-view depth-separable convolution operation consists of two 5x5 depth convolutions and 1x1 point convolutions, with a ReLU6 activation function added during the process for non-linear processing. The result is superimposed with the result after max pooling and then output through the ReLU6 activation function.

[0018] A further technical solution is as follows: In step S5, the two selected dense feature maps are close to the last obtained dense feature map and are of different sizes.

[0019] A further technical solution is as follows: In S5, the ratio of the preselection box is set to 1:1.5.

[0020] To achieve the above objectives, the present invention provides the following technical solution:

[0021] A lightweight face detection system based on a smart TV camera includes:

[0022] The preprocessing module is used to scale the original image captured by the smart TV camera and use the scaled original image as input to the face detection model.

[0023] The ordinary convolution processing module is used to perform multiple ordinary convolution processes on the original image after it has been scaled by the preprocessing module, so as to process the scaled original image into the first feature map.

[0024] The separable convolution processing module is used to perform several single-loop wide field-of-view depth-separable convolution operations on the first feature map after processing by the ordinary convolution processing module to locate the features of some distant faces and obtain the second feature map; and to perform several double-loop wide field-of-view depth-separable convolution operations on the second feature map to capture the facial features when the distance is close and the target is large, and obtain the dense feature map.

[0025] The face identification module is used to select two dense feature maps from the output of the separable convolutional processing module, select anchor points, and set pre-selection boxes for regression to obtain the position information of the face.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] This invention addresses the issue of long inference times in traditional face detection networks by employing wide-field-of-view, depth-separable convolution operations to reduce the number of preselection boxes. Compared to the traditional small-field-of-view 3x3 convolution, it utilizes two wide-field-of-view convolution operations: 7x7 and 5x5 depthwise convolutions. This accelerates feature extraction, and in the S5 preselection box selection, since the aspect ratio of the face is relatively fixed, the preselection box ratio is set to 1:1.5, significantly reducing the computational load for regressing face bounding boxes and thus improving the efficiency of face location recognition. Attached Figure Description

[0028] Figure 1 This is a flowchart of a lightweight face detection method based on a smart TV camera in an embodiment of the present invention;

[0029] Figure 2 This is a network design diagram of a lightweight face detection method based on a smart TV camera in an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of a single-loop wide-field-of-view depth-separable convolutional structure in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of a dual-loop wide-field-of-view depth-separable convolutional structure in an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of a lightweight face detection system based on a smart TV camera, as described in an embodiment of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] In one embodiment, such as Figure 1-4 As shown, a lightweight face detection method based on a smart TV camera provides a novel image processing strategy to reduce computational resource consumption and improve image processing speed. Specific steps include:

[0035] S1. Scale the original image captured by the smart TV camera and use the scaled original image as the input to the face detection model.

[0036] In this embodiment, the size is specifically scaled to 256x256x3.

[0037] S2. Perform multiple ordinary convolution processes on the original image after scaling in S1 to process the scaled original image into the first feature map.

[0038] Specifically, in this embodiment, two ordinary convolution processes were performed, and the size of the first feature map was 64x64x48.

[0039] S3. Perform several single-loop wide-field depth-separable convolution operations on the first feature map obtained in S2 to locate features of some distant faces and obtain the second feature map to prevent feature nodes from becoming inactive and introducing residual branches as the convolutional layer deepens.

[0040] Specifically, in this embodiment, the number of times the single-loop wide field-of-view depth separable convolution operation is five, and the size of the second feature map is 32x32x48.

[0041] Furthermore, the single-loop wide-view depth-separable convolution is composed of 7x7 depthwise convolution and 1x1 pointwise convolution results superimposed with the results of max pooling, and finally output through the ReLU6 activation function.

[0042] S4. Perform several double-loop wide-field depth-separable convolution operations on the second feature map obtained in S3 to capture facial features when the distance is close and the target is large, and obtain a dense feature map.

[0043] Specifically, in this embodiment, the number of times the double-loop wide field-of-view depth-separable convolution operation is six, and the size of the dense feature map is 8x8x96.

[0044] Furthermore, the dual-loop wide-view depth-separable convolution consists of two 5x5 depthwise convolutions and a 1x1 pointwise convolution, with a ReLU6 activation function added for non-linear processing. The result is superimposed with the result after max pooling and finally output through the ReLU6 activation function.

[0045] S5. Select two dense feature maps from the dense feature maps obtained in S4, select anchor points, and set pre-selection boxes for regression to obtain the location information of the face.

[0046] Preferably, the two selected dense feature maps are close to the last obtained dense feature map and are of different sizes; the ratio of the preselection box is set to 1:1.5.

[0047] This invention addresses the issue of long inference times in traditional face detection networks by employing wide-field-of-view, depth-separable convolution operations to reduce the number of preselection boxes. Compared to the traditional small-field-of-view 3x3 convolution, it utilizes two wide-field-of-view convolution operations: 7x7 and 5x5 depthwise convolutions. This accelerates feature extraction, and in the S5 preselection box selection, since the aspect ratio of the face is relatively fixed, the preselection box ratio is set to 1:1.5, significantly reducing the computational load for regressing face bounding boxes and thus improving the efficiency of face location recognition.

[0048] In another embodiment, such as Figure 5 As shown, a lightweight face detection system based on a smart TV camera includes:

[0049] The preprocessing module is used to scale the original image captured by the smart TV camera and use the scaled original image as input to the face detection model.

[0050] The ordinary convolution processing module is used to perform multiple ordinary convolution processes on the original image after it has been scaled by the preprocessing module, so as to process the scaled original image into the first feature map.

[0051] The separable convolution processing module is used to perform several single-loop wide field-of-view depth-separable convolution operations on the first feature map after processing by the ordinary convolution processing module to locate the features of some distant faces and obtain the second feature map; and to perform several double-loop wide field-of-view depth-separable convolution operations on the second feature map to capture the facial features when the distance is close and the target is large, and obtain the dense feature map.

[0052] The face identification module is used to select two dense feature maps from the output of the separable convolutional processing module, select anchor points, and set pre-selection boxes for regression to obtain the position information of the face.

[0053] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A lightweight face detection method based on a smart TV camera, characterized in that, include: S1. Scale the original image captured by the smart TV camera and use it as input to the face detection model; S2. Perform ordinary convolution processing on the original image after scaling in S1 to process the scaled original image into the first feature map. S3. Perform several single-loop wide-field depth-separable convolution operations on the first feature map obtained in S2 to locate the features of some distant faces and obtain the second feature map. The single-loop wide field-of-view depth-separable convolution operation is specifically performed as follows: the input feature map is convolved with a 7x7 depth and then with a 1x1 point convolution. The result of the 1x1 point convolution is superimposed with the result of the max pooling operation and then output through the ReLU6 activation function. S4. Perform several double-loop wide-field depth-separable convolution operations on the second feature map obtained in S3 to capture facial features that are close to each other and obtain a dense feature map. The dual-loop wide field-of-view depth-separable convolution operation consists of two 5x5 depthwise convolutions and a 1x1 pointwise convolution, with a ReLU6 activation function added for non-linear processing. The result is superimposed with the result of a max pooling operation and then output through the ReLU6 activation function. S5. Select two dense feature maps from the dense feature maps obtained in S4, select anchor points, and set pre-selection boxes for regression to obtain the location information of the face.

2. The lightweight face detection method based on a smart TV camera according to claim 1, characterized in that, In S2, two ordinary convolution processes are performed, and the size of the first feature map is 64x64x48.

3. The lightweight face detection method based on a smart TV camera according to claim 1, characterized in that, In S3, the single-loop wide field-of-view depth-separable convolution operation is performed five times, and the size of the second feature map is 32x32x48.

4. The lightweight face detection method based on a smart TV camera according to claim 1, characterized in that, In S4, the number of times the double-loop wide field-of-view depth-separable convolution operation is performed is six, and the size of the dense feature map is 8x8x96.

5. The lightweight face detection method based on a smart TV camera according to claim 1, characterized in that, In S4, the dual-loop wide-view depth-separable convolution operation sequentially performs a first 5x5 depthwise convolution, a first 1x1 pointwise convolution, a ReLU6 activation function for non-linear processing, a second 5x5 depthwise convolution, and a second 1x1 pointwise convolution. The result of the second 1x1 pointwise convolution is then superimposed with the result of the maximum pooling operation and output through the ReLU6 activation function.

6. The lightweight face detection method based on a smart TV camera according to claim 1, characterized in that, In step S5, the two selected dense feature maps are close to the last obtained dense feature map and are of different sizes.

7. The lightweight face detection method based on a smart TV camera according to claim 6, characterized in that, In S5, the ratio of the preselection box is set to 1:1.

5.

8. A lightweight face detection system based on a smart TV camera, characterized in that, include: The preprocessing module is used to scale the original image captured by the smart TV camera and use the scaled original image as input to the face detection model. The ordinary convolution processing module is used to perform multiple ordinary convolution processes on the original image after it has been scaled by the preprocessing module, so as to process the scaled original image into the first feature map. The separable convolution processing module is used to perform several single-loop wide field-of-view depth-separable convolution operations on the first feature map after processing by the ordinary convolution processing module to locate the features of some distant faces and obtain the second feature map; and to perform several double-loop wide field-of-view depth-separable convolution operations on the second feature map to capture the features of nearby faces and obtain the dense feature map. The single-loop wide-view depth-separable convolution operation is specifically performed as follows: the input feature map is subjected to a 7x7 depthwise convolution followed by a 1x1 pointwise convolution. The result of the 1x1 pointwise convolution is then superimposed with the result of the max pooling operation, and finally output through the ReLU6 activation function. The double-loop wide-view depth-separable convolution operation consists of two 5x5 depthwise convolutions and a 1x1 pointwise convolution, with a ReLU6 activation function added for non-linear processing. The result is then superimposed with the result of the max pooling operation, and finally output through the ReLU6 activation function. The face identification module is used to select two dense feature maps from the output of the separable convolutional processing module, select anchor points, and set pre-selection boxes for regression to obtain the position information of the face.

Images