Method for determining front and side face features based on hotspot area feature fusion
By using convolutional neural networks to locate hotspot regions in images and combining HOG and LBP features, the robustness problem of traditional machine learning classification models under illumination and occlusion was solved, achieving efficient frontal and side profile recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING ICHINAE SCI & TECH CO LTD
- Filing Date
- 2022-12-05
- Publication Date
- 2026-06-09
Smart Images
Figure CN115798012B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data recognition, and in particular to a method for judging frontal and side profile features based on hotspot region feature fusion. Background Technology
[0002] In facial recognition, frontal and side profile recognition is involved. Although a face correction process is added during this process, side profiles cannot be corrected to a frontal view due to left and right rotation. To ensure that frontal images are compared with each other during face comparison, there are several existing methods to select a frontal image from multiple images, including: frontal and side profile recognition methods based on traditional machine learning classification models, frontal and side profile recognition methods based on Euler angles of the face, and frontal and side profile recognition methods based on convolutional neural network classification models.
[0003] Among them, existing methods for determining frontal and side profiles based on traditional machine learning classification models use the features of the entire face region as input to achieve frontal and side profile determination, which has problems such as slow determination speed and poor robustness under uncertain factors such as lighting and occlusion. Summary of the Invention
[0004] The purpose of this invention is to provide a method for judging front and side face features based on hotspot region feature fusion, so as to solve the problems of slow judgment speed and poor robustness under uncertain factors such as lighting and occlusion in the traditional machine learning classification model for judging front and side faces.
[0005] This invention provides a method for determining frontal and side profile features based on hotspot region feature fusion, comprising:
[0006] S1. Based on a pre-defined cropping method, crop the images containing human faces in the original image set to obtain a dataset, which is divided into the original training set and the original test set.
[0007] S2, hotspot grayscale image feature extraction is performed on the images in the dataset to obtain a hotspot grayscale image feature training set and a hotspot grayscale image feature test set respectively;
[0008] S3, using the hotspot grayscale image feature training set as the input to the classification model, and the frontal face label, left face label and right face label in the original training set as classification labels, the classification model constructed using traditional machine learning methods is trained to obtain the final classification model;
[0009] S4, any hotspot grayscale image from the hotspot grayscale image feature image test set is input into the final classification model to complete the front and side face judgment;
[0010] Specifically, in S2, hotspot grayscale feature extraction is performed on any image X, as follows:
[0011] S201, The convolutional neural network classification model is trained using the original training set to obtain the convolutional neural network classification model M;
[0012] S202, use the convolutional neural network classification model M to read the feature map of the image X and obtain a visual heatmap of the feature map;
[0013] S203, Constructing hotspot regions specifically involves: using the convolutional neural network classification model M to obtain the visual heatmaps of each image in the original training set, calculating the mean, and obtaining an average heatmap; selecting at least one sub-region from the average heatmap as a hotspot sub-region; and stitching the hotspot sub-regions together to form a square to obtain the hotspot region.
[0014] S204, using the hotspot region to crop the grayscale image of image X to obtain the hotspot grayscale image of image X; extract the HOG feature and LBP feature of the hotspot grayscale image respectively, standardize the HOG feature and LBP feature respectively, and stitch them together to obtain the hotspot grayscale image feature, thus completing the extraction of the hotspot grayscale image feature of image X.
[0015] In the above embodiments of the present invention, optionally, S1 specifically includes:
[0016] S101, Obtain the original image set containing at least one image;
[0017] S102, according to a preset cropping rule, crop the images containing faces in the original image set to obtain a dataset; the preset cropping rule is that the cropping area is a square, the center of the square is the center of the face target box, and the width of the square is selected from the width and height of the face target box.
[0018] S103, the dataset is divided into the original training set and the original test set; each image in the original training set is labeled; the labels include frontal face, left side face and right side face.
[0019] In the above embodiments of the present invention, optionally, in S202, obtaining the visual heatmap of the feature map specifically involves: performing image scaling and normalization processing on the feature map sequentially to obtain the visual heatmap of the feature map; wherein, the size of the feature map is 1 / 8 of the size of the input image of the convolutional neural network classification model M; the size of the scaled feature map is the same as the size of the visual heatmap, both being 1 / 2 of the size of the input image of the convolutional neural network classification model M.
[0020] In the above embodiments of the present invention, optionally, in S204, the grayscale image of image X is converted into a hotspot grayscale image using pre-acquired hotspot areas, specifically as follows:
[0021] S2041, Obtain any image X from the dataset, and convert image X into a grayscale image X. ′ ;
[0022] S2042, using the pre-acquired hotspot regions, crop the grayscale image X. ′ Obtain the hotspot grayscale image.
[0023] In the above embodiments of the present invention, optionally, in S204, the HOG features and LBP features of the hotspot grayscale image are extracted respectively, the HOG features and the LBP features are standardized respectively, and the hotspot grayscale image features are concatenated to obtain the hotspot grayscale image features, thereby completing the extraction of the hotspot grayscale image features of the image X; specifically:
[0024] S001, extract the HOG features of the hotspot grayscale image, and extract the LBP features of the hotspot grayscale image;
[0025] S002, perform L2 normalization on the HOG features to obtain the normalized HOG features;
[0026] The LBP features are L1 standardized to obtain the standardized LBP features;
[0027] S003, the standardized HOG feature and the standardized LBP feature are concatenated to obtain the hotspot grayscale image feature, thus completing the extraction of the hotspot grayscale image feature of the image X.
[0028] In the above embodiments of the present invention, optionally, the selection requirement for the hot spot sub-region is: the sum of the areas of all hot spot sub-regions / the area of the visualized heat map = 1 / 4; the average heat map is cropped using each hot spot sub-region to obtain a corresponding two-dimensional array, each two-dimensional array is first summed to obtain summed data, and all summed data are accumulated to obtain a final summed result, ensuring that the final summed result obtained by accumulation is maximized when selecting hot spot sub-regions.
[0029] In the above embodiments of the present invention, optionally, the number of hotspot sub-regions is 1, 2, 3 or 4.
[0030] In the above embodiments of the present invention, optionally, when the number of hot spot sub-regions is 1, a sub-region is selected from the average heat map as a hot spot sub-region, and the hot spot sub-region is the hot spot region.
[0031] In the above embodiments of the present invention, optionally, in step S2, hotspot grayscale features are extracted from each image included in the dataset to obtain a hotspot grayscale feature image, wherein the size of the hotspot grayscale feature image is 1 / 4 of the size of its corresponding image.
[0032] Compared with the prior art, the beneficial effects of the frontal and side profile judgment method based on hotspot region feature fusion described in the technical solution of the present invention are:
[0033] 1) The technical solution of the present invention uses the feature map of the convolutional neural network classification model to locate the image region that has a significant impact on the classification effect, thereby reducing the input image of the traditional machine learning classification model to one-quarter of the original image and greatly improving the judgment efficiency.
[0034] 2) Whether or not faces outside the hotspot area are occluded has no impact on the accuracy of the technical solution of this invention, thus improving the robustness of the front and side face judgment method based on the traditional machine learning classification model under occlusion factors.
[0035] 3) The technical solution of this invention uses HOG features and LBP features. HOG features can extract the contour information of the image, and LBP features can extract the texture information of the image. Both are highly robust to illumination factors. There are various normalization methods before stitching. After multiple experiments, the highest accuracy was achieved by normalizing HOG features L2 and LBP features L1 before stitching. Attached Figure Description
[0036] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 A flowchart of a frontal and side profile determination method based on hotspot region feature fusion according to an embodiment of the present invention is shown. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0039] The principle of this application: The technical solution of this application uses the feature maps of a convolutional neural network classification model to locate image regions that have a significant impact on classification performance, thereby reducing the input image of traditional machine learning classification models to one-quarter of the original image. Different standardization methods are used as image features before concatenating HOG and LBP features; HOG features use L2 standardization and LBP features use L1 standardization, resulting in higher classification accuracy.
[0040] Example
[0041] This embodiment provides a method for determining frontal and side profile features based on hotspot region feature fusion, including:
[0042] S1, based on a pre-defined cropping method, crop the images containing faces in the original image set to obtain a dataset. This dataset is divided into an original training set and an original test set, specifically:
[0043] S101, Obtain the original image set containing at least one image;
[0044] S102, according to preset cropping rules, crop the images containing faces in the original image set to obtain a dataset; the preset cropping rules are that the cropping area is a square, the center of the square is the center of the face target bounding box, and the width of the square is selected from the width and height of the face target bounding box; here, the width of the square = max(width of the face target bounding box, height of the face target bounding box) means: when the width of the face target bounding box is greater than the height of the face target bounding box, the width of the square = the width of the face target bounding box; when the width of the face target bounding box is less than the height of the face target bounding box, the width of the square = the height of the face target bounding box; when the width of the face target bounding box is equal to the height of the face target bounding box, the width of the square = the width of the face target bounding box = the height of the face target bounding box.
[0045] S103, the dataset is divided into the original training set and the original test set; each image in the original training set is labeled; the labels include frontal face, left side face and right side face.
[0046] S2, extract the features of hotspot grayscale images from each image included in the dataset to obtain a training set of hotspot grayscale image features and a test set of hotspot grayscale image features.
[0047] S3, using the hotspot grayscale image feature training set as the input to the classification model, and the frontal face label, left face label, and right face label in the original training set as classification labels, the classification model constructed using traditional machine learning methods is trained to obtain the final classification model.
[0048] S4, any hotspot grayscale feature image from the hotspot grayscale feature image test set is input into the final classification model to complete the front and side face judgment.
[0049] Specifically, in S2, hotspot grayscale feature extraction is performed on any image X, as follows:
[0050] S201, The pre-constructed convolutional neural network classification model is trained using the original training set to obtain a high-precision convolutional neural network classification model M;
[0051] S202, the feature map of the image X is read using the convolutional neural network classification model M, and a visual heatmap of the feature map is obtained; wherein, obtaining the visual heatmap of the feature map specifically involves: performing image scaling and normalization processing on the feature map sequentially to obtain the visual heatmap of the feature map; the size of the feature map is 1 / 8 of the size of the input image of the convolutional neural network classification model M; the size of the scaled feature map is the same as the size of the visual heatmap, both being 1 / 2 of the size of the input image of the convolutional neural network classification model M.
[0052] S203, using the hotspot region to crop the grayscale image of image X to obtain the hotspot grayscale image of image X; extract the HOG feature and LBP feature of the hotspot grayscale image respectively, standardize the HOG feature and LBP feature respectively, and stitch them together to obtain the hotspot grayscale image feature, thus completing the extraction of the hotspot grayscale image feature of image X.
[0053] Here, using pre-acquired hotspot regions, the grayscale image of image X is converted into a hotspot grayscale image. Specifically, this involves: obtaining any image X from the dataset and converting image X into a grayscale image X'; then, using the pre-acquired hotspot regions, cropping the grayscale image X' to obtain the hotspot grayscale image. The size of the hotspot grayscale image is 1 / 4 of the size of image X, and the hotspot grayscale image is square.
[0054] Here, the HOG and LBP features of the hotspot grayscale image are extracted separately, and the HOG and LBP features are standardized respectively and concatenated to obtain the hotspot grayscale image features, thus completing the extraction of the hotspot grayscale image features of image X; specifically: S001, extract the HOG features of the hotspot grayscale image, extract the LBP features of the hotspot grayscale image; S002, perform L2 standardization on the HOG features to obtain standardized HOG features; perform L1 standardization on the LBP features to obtain standardized LBP features; S003, concatenate the standardized HOG features and the standardized LBP features to obtain the hotspot grayscale image features, thus completing the extraction of the hotspot grayscale image features of image X.
[0055] In this embodiment, S203, hotspot regions are constructed. Specifically, the convolutional neural network classification model M is used to obtain the visual heatmaps of each image in the original training set. After calculating the mean, an average heatmap is obtained. At least one sub-region is selected from the average heatmap as a hotspot sub-region. Here, the number of hotspot sub-regions is 1, 2, 3, or 4; the purpose is to reduce the efficiency loss caused by image cropping and stitching. The hotspot sub-regions are stitched together to form a square to obtain the hotspot region. Here, when the number of hotspot sub-regions is 1, one sub-region is selected from the average heatmap as a hotspot sub-region, and this hotspot sub-region is the hotspot region; when the number of hotspot sub-regions is two or more, two or more sub-regions are selected from the average heatmap as hotspot sub-regions, and the two or more hotspot sub-regions are stitched together to form a square to obtain the hotspot region. Here, the visualized heatmap is a two-dimensional array (height*width). The visualized heatmap of all images in the training dataset can be represented as a three-dimensional array (image_num*height*width). The mean is the average value of this three-dimensional array along the image_num dimension, resulting in a two-dimensional array (height*width). In the embodiments of this application, the mean is the average heatmap. The hotspot sub-region is determined based on which region of the average heatmap has the largest summation result.
[0056] Here, the hotspot sub-region itself is location information. For example, a hotspot sub-region can be represented as (x1, y1, x2, y2), where x1, y1, x2, and y2 represent the x-axis and y-axis coordinates of the top-left corner (x1, y1) of the hotspot sub-region relative to the top-left corner of the "visualized heatmap," and the x-axis and y-axis coordinates of the bottom-right corner (x2, y2) of the hotspot sub-region relative to the top-left corner of the "visualized heatmap." The selection of hotspot sub-regions in the above embodiments of this application requires two points: First, the sum of the areas of all hotspot sub-regions / the area of the visualized heatmap = 1 / 4. Second, each hotspot sub-region is used to crop the average heatmap to obtain a corresponding two-dimensional array. Each two-dimensional array is first summed to obtain summed data, and all summed data are accumulated to obtain a final summed result. When selecting hotspot sub-regions, it is ensured that the final summed result is maximized.
[0057] Example:
[0058] The frontal / side profile recognition method based on hotspot region feature fusion described in this example includes the following steps:
[0059] Step 1: Given the coordinates of the bounding box of the face in the image (x1, y1, x2, y2), where x1, y1, x2, and y2 represent the x-coordinates and y-coordinates of the top-left and bottom-right corners of the bounding box relative to the top-left corner of the image, respectively. Using ((x1+x2) / 2, (y1+y2) / 2) as the center, crop an image region with a side length of max(x2-x1, y2-y1). If the cropped region exceeds the image boundaries, fill the excess area with the value 127.
[0060] The second step involves creating a frontal / side profile classification dataset using the cropping method from the previous step. The classification labels are: frontal face, left side face, and right side face. Specifically, a frontal face is defined as a face with a yaw angle of no more than 30 degrees; a left side face is defined as a face rotated to the left with a yaw angle of no less than 40 degrees; and a right side face is defined as a face rotated to the right with a yaw angle of no less than 40 degrees. To avoid interference from labeling errors and excessively small differences between data classes, images with yaw angles between 30 and 40 degrees are temporarily disregarded.
[0061] The third step involves constructing a convolutional neural network (CNN) classification model. The model input is a 112*112 pixel grayscale image. If the input image is not 112*112 pixels, it needs to be scaled down to 112*112 pixels. The model outputs three categories: frontal face, left side face, and right side face. This CNN model uses three convolutional layers with a kernel size of 3*3, a stride of 1, and padding of 1, followed by a 2*2 max-pooling layer. The model is trained using a frontal / side-face classification dataset to obtain a high-accuracy CNN classification model.
[0062] Step 4: Use the trained convolutional neural network classification model to perform inference on the training dataset. During the inference process for each image, read the last feature map in the network with a width and height of 14*14 pixels. Accumulate this feature map along the channel dimension, and enlarge it to 56*56 pixels using cubic spline interpolation. Divide the enlarged feature map matrix by the maximum value in the feature map matrix and multiply by 255 to obtain the normalized visualization heatmap.
[0063] Step 5: Calculate the mean of the heatmaps for all images in the training dataset, denoted as the average heatmap. Use a sliding window method to traverse the average heatmap, identifying four non-overlapping hotspot sub-regions with a width and height of 14*14 pixels. The hotspot region with the largest sum of pixel values among these four sub-regions is designated as hotspot regions A, B, C, and D, in order from left to right and top to bottom. Here, the size of the hotspot sub-region is determined based on the magnified feature map, and is 1 / 4 of the size of the magnified feature map.
[0064] Step 6: Read the frontal and profile images from the training dataset, convert them to grayscale images using the formula Gray = 0.2989*R + 0.5870*G + 0.1140*B, and scale them to a size of 56*56 pixels (width*height). Here, Gray, R, G, and B represent the grayscale value, red channel value, green channel value, and blue channel value, respectively. Crop the grayscale image into hotspot grayscale sub-images A, B, C, and D based on the hotspot regions, with each sub-image being 14*14 pixels. Stitch these hotspot grayscale sub-images together to form a 28*28 pixel hotspot grayscale image, with sub-image A placed in the top left corner, B in the top right corner, C in the bottom left corner, and D in the bottom right corner.
[0065] Step 7: Construct HOG feature descriptions with window size winSize=(28,28), block size blockSize=(14,14), block stride=(7,7), cell size cellSize=(7,7), and gradient direction number nbins=9. Perform HOG feature extraction on the hotspot grayscale image to obtain 324-dimensional HOG features.
[0066] Step 8: Process the hotspot grayscale image using the original Local Binary Pattern (LBP) to obtain an LBP feature image with a size of 28*28 pixels and a value ranging from 0 to 255. Perform histogram statistics on the LBP feature image to obtain 256-dimensional LBP features.
[0067] Step 9: Perform L2 standardization on the HOG features and L1 standardization on the LBP features. The 580-dimensional feature formed by splicing the two is used as the feature of the hotspot grayscale image.
[0068] Step 10: Construct a Support Vector Machine (SVM) three-class classification model using a linear kernel function. Use 580-dimensional hotspot grayscale image features as input to the classification model, and the labels for the frontal face, left side face, and right side face from the original dataset as classification labels. Train the model to obtain the final classification model. When using the final classification model for inference, the processing steps for the images to be inferred (i.e., steps four through nine) should remain consistent with the processing steps for the images in the training data.
[0069] The image sizes involved in this embodiment are explained as follows:
[0070] 1) In the fourth step, the input image size for the convolutional neural network is 112*112.
[0071] 2) In the fourth step, the selected feature map size is 14*14. Note: The selected feature map size = the size of the input image of the convolutional neural network * 1 / 8.
[0072] 3) In the fourth step, the size of the selected feature map after magnification is 56*56. Note: The size of the selected feature map after magnification is equal to the size of the input image of the convolutional neural network * 1 / 2.
[0073] 4) In the fourth step, the size of the visualized heatmap is 56*56. Note: The size of the visualized heatmap is the same as the size of the selected feature map after it is enlarged. Both are 1 / 2 of the size of the input image of the convolutional neural network.
[0074] 5) In the fifth step, the total area of the hot spot sub-regions is 14*14*4. The total area of all hot spot sub-regions / the area of the corresponding image in the training set = 1 / 4.
[0075] 6) In the fifth step, the hotspot area size is 28*28. Note: Hotspot area size = 1 / 4 of the input image size of the convolutional neural network.
[0076] 7) In step six, the front and side profile images are "scaled to a size of 56*56 pixels" to match the size of the visualized heatmap. The input image size for traditional machine learning classification models is 56*56 pixels. The front and side profile image size is equal to half the size of the input image for the convolutional neural network.
[0077] 8) In step six, the dimensions of the "hotspot grayscale sub-image" and the "hotspot sub-region" are the same: 14*14*4.
[0078] 9) In step six, the size of the "hotspot grayscale image" and the "hotspot region" are the same: 28*28. Note: Hotspot grayscale image = 1 / 4 of the size of the input image to the convolutional neural network.
[0079] Here, feature extraction using a 28*28 size hotspot grayscale image (9) and feature extraction using a 56*56 size image (7) yields almost no difference in final classification accuracy. Therefore, the invention achieves the effect of reducing the input image of the traditional machine learning classification model to one-quarter of the original image, greatly improving the algorithm efficiency.
[0080] By adopting the above-disclosed technical solution of this invention, the following beneficial effects are obtained:
[0081] 1) The technical solution of the present invention uses the feature map of the convolutional neural network classification model to locate the image region that has a significant impact on the classification effect, thereby reducing the input image of the traditional machine learning classification model to one-quarter of the original image and greatly improving the judgment efficiency.
[0082] 2) Whether or not faces outside the hotspot area are occluded has no impact on the accuracy of the technical solution of this invention, thus improving the robustness of the front and side face judgment method based on the traditional machine learning classification model under occlusion factors.
[0083] 3) The technical solution of this invention uses HOG features and LBP features. HOG features can extract the contour information of the image, and LBP features can extract the texture information of the image. Both are highly robust to illumination factors. There are various normalization methods before stitching. After multiple experiments, the highest accuracy was achieved by normalizing HOG features L2 and LBP features L1 before stitching.
[0084] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.
[0085] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0086] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0087] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."
[0088] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0089] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0090] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0091] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for determining frontal and side profile features based on hotspot region feature fusion, characterized in that, include: S1. Based on a pre-defined cropping method, crop the images containing human faces in the original image set to obtain a dataset, which is divided into the original training set and the original test set. S2, extract the features of hotspot grayscale images from the images in the dataset to obtain a training set and a test set of hotspot grayscale image features respectively; S3, using the hotspot grayscale image feature training set as the input to the classification model, and the frontal face label, left face label and right face label in the original training set as classification labels, the classification model constructed using traditional machine learning methods is trained to obtain the final classification model; S4, any hotspot grayscale image from the hotspot grayscale image feature image test set is input into the final classification model to complete the front and side face judgment; Specifically, in S2, hotspot grayscale feature extraction is performed on any image X, as follows: S201, The convolutional neural network classification model is trained using the original training set to obtain the convolutional neural network classification model M; S202, use the convolutional neural network classification model M to read the feature map of the image X and obtain a visual heatmap of the feature map; S203, Constructing hotspot regions specifically involves: using the convolutional neural network classification model M to obtain the visual heatmaps of each image in the original training set, calculating the mean, and obtaining an average heatmap; selecting at least one sub-region from the average heatmap as a hotspot sub-region; and stitching the hotspot sub-regions together to form a square to obtain the hotspot region. S204, using the hotspot region to crop the grayscale image of image X to obtain the hotspot grayscale image of image X; extract the HOG feature and LBP feature of the hotspot grayscale image respectively, standardize the HOG feature and LBP feature respectively, and stitch them together to obtain the hotspot grayscale image feature, thus completing the extraction of the hotspot grayscale image feature of image X. S1, specifically: S101, Obtain the original image set containing at least one image; S102, according to a preset cropping rule, crop the images containing faces in the original image set to obtain a dataset; the preset cropping rule is that the cropping area is a square, the center of the square is the center of the face target box, and the width of the square is selected from the width and height of the face target box. S103, the dataset is divided into the original training set and the original test set; each image in the original training set is labeled; the labels include frontal face, left side face and right side face; In S202, obtaining the visual heatmap of the feature map specifically involves: sequentially performing image scaling and normalization processing on the feature map to obtain the visual heatmap of the feature map; wherein, the size of the feature map is 1 / 8 of the input image size of the convolutional neural network classification model M; the scaled size of the feature map is the same as the size of the visual heatmap, both being 1 / 2 of the input image size of the convolutional neural network classification model M; In S203, using pre-acquired hotspot areas, the grayscale image of image X is converted into a hotspot grayscale image, specifically as follows: S2031, Obtain any image X from the dataset, and convert the image X into a grayscale image X′; S2032, using the pre-acquired hotspot regions, crop the grayscale image X′ to obtain a hotspot grayscale image; In S203, the HOG and LBP features of the hotspot grayscale image are extracted respectively. The HOG and LBP features are standardized respectively and concatenated to obtain the hotspot grayscale image features, thus completing the extraction of the hotspot grayscale image features of image X; specifically: S001, extract the HOG features of the hotspot grayscale image, and extract the LBP features of the hotspot grayscale image; S002, perform L2 normalization on the HOG features to obtain the normalized HOG features; The LBP features are L1 standardized to obtain the standardized LBP features; S003, the standardized HOG feature and the standardized LBP feature are concatenated to obtain the hotspot grayscale image feature, thus completing the extraction of the hotspot grayscale image feature of the image X; The selection requirement for the hotspot sub-regions is: the sum of the areas of all hotspot sub-regions / the area of the visualized heatmap = 1 / 4; the average heatmap is cropped using each hotspot sub-region to obtain a corresponding two-dimensional array, each two-dimensional array is first summed to obtain summed data, and all summed data are accumulated to obtain a final summed result. When selecting hotspot sub-regions, it is ensured that the final summed result obtained by accumulation is maximized. S2, extract the features of hotspot grayscale images from each image included in the dataset to obtain hotspot grayscale feature images. The size of the hotspot grayscale feature images is 1 / 4 of the size of the corresponding image. The number of hotspot sub-regions is 1, 2, 3, or 4; When the number of hotspot sub-regions is 1, a sub-region is selected from the average heat map as a hotspot sub-region, and this hotspot sub-region is the hotspot region.