Face classification method and equipment based on classification model, and medium
By combining local features and global feature extraction networks in the face classification model, the problem of insufficient exploration of the three-dimensional features of human five-feature features in two-dimensional image analysis is solved, and more efficient and accurate face classification is achieved, and the understanding of the relationship between human facial features and physical constitution is enhanced.
Patent Information
- Application Number
- CN202411791250.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-06-10
AI Technical Summary
The traditional facial features analysis method mainly focuses on two-dimensional image analysis, with a small sample size and failure to fully explore the three-dimensional features of human facial features, resulting in insufficient comprehensive exploration of human facial features.
A face classification method based on classification model is adopted, combining the local feature extraction network and the global feature extraction network, and by obtaining the feature map, local features and global features of the face image to be classified, these features are fused to obtain the face classification results.
By combining three-dimensional facial data and analyzing the face classification results based on global features and local features, the accuracy and robustness of face classification are improved, efficiency and scalability are ensured, and the relationship between human facial features and physical fitness is more comprehensively explored and verified.
Smart Images
Figure CN120126191A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of machine learning, and in particular, to a face classification method, device, and medium based on a classification model. Background Art
[0002] In the field of facial feature analysis, researchers have been committed to automatically identifying and analyzing human faces through computer vision technology. Especially in the research of facial and facial features, facial image processing technology has been widely used in automatic identification and physical analysis. Different facial features reflect the physical types of individuals, and this theory provides a unique perspective for medical and psychological research.
[0003] In order to achieve automatic identification of human facial features, in recent years, the Active Shape Model (ASM) algorithm in computer image recognition technology has been introduced into the extraction and analysis of facial feature points. The ASM model can accurately describe the facial contour and the positions of facial features by automatically marking key feature points in the face image, providing a reliable basis for subsequent feature analysis.
[0004] Based on the ASM algorithm, researchers calculate the distances between key feature points and use the facial index method to calculate ratios, thereby quantifying various feature indicators of the human face and finally obtaining the laws of human facial and facial features. The implementation of this method enables the computer to automatically identify and judge various human facial features, providing a new idea for the analysis of individual physical types.
[0005] Traditional algorithms mainly focus on the two-dimensional image analysis of human facial features, and the sample size used is small. The three-dimensional feature analysis of facial features has not been fully developed, resulting in insufficient exploration of the comprehensiveness of human facial features. Summary of the Invention
[0006] To solve the above technical problems, the present invention provides a face classification method, device, and medium based on a classification model, which extracts the global features and local features of a face image to be classified and identifies the belonging of the population.
[0007] To solve the above technical problems, in a first aspect of the present invention, a face classification method based on a classification model is disclosed. The classification model includes a local feature extraction network and a global feature extraction network. The method includes: Obtain a face image to be classified; Input the face image to be classified into the classification model, and use the local feature extraction network to extract the feature map and local features of the face image to be classified; Use the global feature extraction network to extract the global features of the face image to be classified; Based on the local features and the global features, obtain the face classification result.
[0008] In some embodiments, extracting the global features of the face image to be classified using a global feature extraction network includes: Performing blurring processing and adaptive threshold segmentation on the face image to be classified to generate a face contour map of the face image to be classified; Adjusting the size of the face contour map according to the feature map to obtain a contour mask; Combining the local features and the contour mask to obtain global features.
[0009] In some embodiments, obtaining the face classification result based on the local features and the global features includes: Fusing the local features and the global features to obtain the fused features of the face image to be classified; Determining the probability that the face image to be classified belongs to each face physique category according to the fused features of the face image to be classified; Determining the face classification result of the face image to be classified according to the probability that the face image to be classified belongs to each face physique category.
[0010] In some embodiments, fusing the local features and the global features to obtain the fused features of the face image to be classified includes: Performing global average pooling and summation operations on the local features to obtain a first pooling value; Performing global average pooling and summation operations on the global features to obtain a second pooling value; Calculating the fused features of the face image to be classified according to the local features, the first pooling value, the global features, and the second pooling value.
[0011] In some embodiments, calculating the fused features of the face image to be classified according to the local features, the first pooling value, the global features, and the second pooling value includes:
[0012] Wherein, F2 is the fused feature, F0 is the local feature, V0 is the first pooling value, F1 is the global feature, and V1 is the second pooling value.
[0013] In some embodiments, the classification model is trained using a plurality of training images, the classification label of each training image, and a target loss function.
[0014] In some embodiments, the target loss function includes a first loss function, and the first loss function is
[0015] Among them, F is the feature map output by the local feature extraction network, and M is the contour mask.
[0016] In some embodiments, the target loss function further includes a second loss function, and the second loss function is
[0017] Among them, is the actual value of the i-th classification label, is the predicted probability of the i-th classification label.
[0018] According to a second aspect of the present invention, a computer device is disclosed, including: a processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to perform the steps of a face classification method based on a classification model as described above.
[0019] According to a third aspect of the present invention, a computer storage medium is disclosed, on which a computer program is stored, and when the computer program is executed by a processor, the steps of a face classification method based on a classification model as described above are implemented.
[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides a face classification method, device, and medium based on a classification model. During the classification process, three-dimensional facial data is combined, and the face classification result is analyzed based on global features and local features, more comprehensively exploring and verifying the relationship between human facial features and physical constitution, improving the accuracy and robustness of face classification, and ensuring high efficiency and scalability. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 is a schematic flow chart of the training stage in a face classification method based on a classification model provided by the present invention; Figure 2 is a schematic flow chart of extracting global features in the training stage of a face classification method based on a classification model provided by the present invention; Figure 3 is a schematic structural diagram of a classification model in a face classification method based on a classification model provided by the present invention; Figure 4 is a schematic flow chart of the inference stage in a face classification method based on a classification model provided by the present invention; Figure 5 is a schematic flow chart of extracting global features in the inference stage of a face classification method based on a classification model provided by the present invention; Figure 6A flowchart showing the process of obtaining a face classification result in the inference stage of a face classification method based on a classification model provided by the present invention; Figure 7 A flowchart showing the process of calculating fusion features in the inference stage of a face classification method based on a classification model provided by the present invention. Specific embodiments
[0022] For better understanding and implementation, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0023] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules does not necessarily have to be limited to those steps or modules clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products or devices.
[0024] Embodiments of the present invention disclose a face classification method, device, and medium based on a classification model, which extract global features and local features of a face image to be classified and identify the belonging of a crowd.
[0025] The classification model includes two stages, a training stage and an application stage (also referred to as an inference stage). The classification model is trained using a plurality of training images and the classification labels of each training image with a target loss function.
[0026] As Figure 1 shown below, the training process of the classification model will be described: Step S101, obtain a training data set, where the training data set includes a plurality of training images and the classification label of each training image.
[0027] The training images contain face images I of different ages and genders to ensure the diversity of the training data. The training images are preprocessed, including adjusting the size of the training images, standardization, and data augmentation. Each training image has a corresponding classification label, which identifies the category to which the training image belongs, for example: gold, wood, water, fire, earth.
[0028] Step S102: Train the initial classification model using the target loss function until the training termination condition is reached. The initial classification model includes a local feature extraction network and a global feature extraction network. Use the local feature extraction network to extract the feature map and local features of the training image, and use the global feature extraction network to extract the global features of the training image.
[0029] For the initial classification model, select a suitable convolutional neural network as the backbone network of the model, such as ResNet18, VGG1, etc., which can effectively extract the feature map, local features, and global features of the face image. In this application, ResNet18 is adopted, which has a simple structure, is convenient for training, supports multiple tasks, and has good scalability and flexibility.
[0030] The classification model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input layer receives the preprocessed training image, usually an image matrix of W0×H0×3. The local feature extraction network and the global feature extraction network of the convolutional layer are used to extract the local features and global features in the image respectively. The pooling layer is used to perform the pooling operation. The fully connected layer converts the features based on the pooling operation into the probability distribution of the classification result. The output layer outputs the probability values of five categories through the activation function, corresponding to the classification results respectively.
[0031] In this application, the local feature extraction network adopts ResNet18, and directly extracts the feature map F of the training image through ResNet18. The feature map F is usually a three-dimensional tensor, and its dimensions are H×W×C, where H and W are the height and width of the feature map F, and C is the number of channels, usually 64, 128, 256, or 512, depending on the network level and architecture. In this application, C is 512 to extract richer potential face information.
[0032] As Figure 2 shown, using the global feature extraction network to extract the global features of the training image includes the following steps: Step S201: Blur and adaptively threshold segment the training image to generate the face contour map of the training image; Step S202: Adjust the size of the face contour map according to the feature map output by the local feature extraction network to obtain a contour mask; Step S203: Combine the feature map and the contour mask to obtain the global features.
[0033] The training image is subjected to blurring and adaptive threshold segmentation to generate a face contour map of the training image. The noise in the training image is removed through blurring, making the training image smoother and highlighting the edge information of the face in the training image. A commonly used blurring method is Gaussian blurring, which denoises by performing weighted averaging on each pixel in the image and its neighboring pixels. The adaptive threshold compares the pixel values of each small region with the mean or median of that region and automatically selects an appropriate threshold to determine which pixels belong to the edge. This face contour map is a two-dimensional matrix of 0-1, containing the edge information of the face in the training image, where 1 represents the contour edge and 0 represents the non-contour edge.
[0034] The generated face contour map is scaled to the same size as the feature map F, so that the contour mask M can be processed subsequently together with the feature map F. Assuming the size of the feature map F is H×W, then the contour mask M will also be adjusted to the same size. The contour mask M is a binary matrix of H×W, where each element is 0 or 1, 1 represents the contour part, and 0 represents the non-edge part. This contour mask M can be used to guide the model to focus on the edge features of the face, thereby better extracting global features. During the training process, the contour mask M serves as an important auxiliary information to help the classification model focus on the edge region of the face, thereby learning more refined global features.
[0035] The feature map F is multiplied element-wise with the contour mask M to obtain the global feature F1, selectively strengthening the part of the feature map F related to the contour. Through the contour mask M, the corresponding part of the edge region of the image in the feature map F is enhanced, helping the model better capture the edge features of the image. The non-edge parts of the training image are set to 0, reducing the influence of irrelevant features on classification or other tasks and improving the recognition accuracy.
[0036] After obtaining the global feature, based on the local feature and the global feature, a face classification result is obtained. Specifically, it includes the following steps: Step S301, fusing the local feature and the global feature to obtain the fused feature of the training image, including:
[0037] Where F2 is the fused feature, F0 is the local feature, V0 is the first pooling value, F1 is the global feature, and V1 is the second pooling value.
[0038] Step S302, determining the probability that the training image belongs to each face physique category according to the fused feature of the training image.
[0039] The fully connected layer is used to convert the fused vector into a probability distribution of specific categories. In this application, a fully connected layer with a dimension of 5 is selected, and each dimension corresponds to a prediction score of a category, and each category corresponds to gold, wood, water, fire, or earth. Through the Softmax activation function, each output of the weighted connection layer will be converted into a probability value between 0 and 1, and the sum of the probabilities of all categories is 1.
[0040] Step S303: Determine the face classification result of the training image according to the probability that the training image belongs to each face constitution category.
[0041] Use the argmax function to select the category where the maximum value is located from the probability distribution output by Softmax.
[0042] Furthermore, use the objective loss function to train the classification model until the training termination condition is met. The objective loss function includes a first loss function and a second loss function. In step S203, the first loss function is used to guide the learning of global features. Through the optimization algorithm, the model learns the features of the input training image, obtains the face characteristics corresponding to the five face constitution categories, and can accurately classify.
[0043] The first loss function is
[0044] where F is the feature map and M is the contour mask. The first loss function L1 reflects the role of how the contour mask M participates in the training process. Through the first loss function, the classification model can be forced to pay attention to the entire contour of the face, so as to learn the global features of the face.
[0045] In step S303, the second loss function is
[0046] where is the actual value of the i-th classification label, is the predicted probability of the i-th classification label. By continuously optimizing the second loss function, learn the differences between face constitution categories, and finally accurately classify the training image, optimize the classification ability of the classification model, and ensure that the classification model can accurately predict which category the input image belongs to. At the same time, combined with the first loss function, co-optimize the feature extraction and classification ability of the classification model to improve the overall performance.
[0047] The training termination conditions include but are not limited to the objective loss function reaching a preset threshold, the adjusted objective loss function converging, the training reaching a preset number of times, or the training reaching a preset duration. After the training termination condition is met, determine the final classification model as shown in Figure 3
[0048] As Figure 4 shown, in the application stage, this method includes the following steps: Step S401, obtain the face image to be classified; The face image to be classified generally refers to an image containing facial features captured by a user or a collection device. This image has not been classified or processed and needs to be processed by a classification model to determine its category. The face image to be classified is generally a digital two-dimensional image, usually a color or grayscale image. Common formats include JPEG, PNG, BMP, etc. The image resolution and size can be adjusted according to application requirements, usually 32×32 or 224×224 pixels. Generally, the face image to be classified contains one or more facial features, usually clearly showing key parts such as the face contour, eyes, nose, mouth, etc.
[0049] In some embodiments, after obtaining the face image to be classified and before inputting it into the classification model, the face image to be classified usually needs to be preprocessed to ensure the effective extraction of the facial region. The preprocessing includes identifying and locating the face region in the image through face detection algorithms (such as OpenCV, Dlib), and adjusting the size, contrast, brightness, etc. of the image to meet the requirements of model input.
[0050] Step S402, input the face image to be classified into the classification model, use the local feature extraction network to extract the feature map and local features of the face image to be classified, use the global feature extraction network to extract the global features of the face image to be classified, and obtain the face classification result based on the local features and global features.
[0051] The goal of the face image to be classified is to identify and classify the face physique category through the aforementioned trained face classification model, such as metal, wood, water, fire, earth, etc.
[0052] The local feature extraction network adopts ResNet18, and directly extracts the feature map F of the training image through ResNet18. The feature map F is usually a three-dimensional tensor, and its dimensions are H×W×C, where H and W are the height and width of the feature map, and C is the number of channels, usually 64, 128, 256, or 512, depending on the network layer and architecture. In this application, C is 512 to extract richer potential face information.
[0053] As Figure 5 shown, using the global feature extraction network to extract the global features of the face image to be classified includes: Step S501, perform blurring processing and adaptive threshold segmentation on the face image to be classified to generate the face contour map of the face image to be classified; Step S502: Adjust the size of the face contour map according to the feature map output by the local feature extraction network to obtain a contour mask; Step S503: Combine the feature map and the contour mask to obtain global features.
[0054] The face image to be classified undergoes blurring and adaptive threshold segmentation to generate a face contour map of the face image to be classified. Blurring is used to remove noise in the training image, making the training image smoother and highlighting the edge information of the face in the face image to be classified. A commonly used blurring method is Gaussian blurring, which denoises by performing weighted averaging on each pixel in the face image to be classified and its neighboring pixels. The adaptive threshold will automatically select an appropriate threshold based on the comparison of the pixel values in each small region with the mean or median of that region to determine which pixels belong to the edge. This face contour map is a two-dimensional matrix of 0-1, containing the edge information of the face in the face image to be classified, where 1 represents the contour edge and 0 represents the non-contour edge.
[0055] Scale the generated face contour map to the same size as the feature map F so that the contour mask M can be processed together with the feature map F. Assume the size of the feature map F is H×W, then the contour mask M will also be adjusted to the same size. The contour mask M is a binary matrix of H×W, where each element is 0 or 1, 1 represents the contour part, and 0 represents the non-edge part. This contour mask M can be used to guide the model to focus on the edge features of the face, thereby better extracting global features.
[0056] Multiply the feature map F and the contour mask M element by element to obtain global features F1, selectively strengthening the part of the feature map F related to the contour. Through the contour mask M, enhance the corresponding part of the edge region of the face image to be classified in the feature map F, help the model better capture the edge features of the image, set the non-edge part of the training image to 0, reduce the influence of irrelevant features on classification or other tasks, and improve the recognition accuracy.
[0057] As Figure 6 shown, based on the local features and global features, obtain a face classification result, including: Step S601: Fuse the local features and global features to obtain the fused features of the face image to be classified. Specifically, as Figure 7 shown, including: Step S701: Perform global average pooling and summation operations on the local features to obtain a first pooling value; Step S702: Perform global average pooling and summation operations on the global features to obtain a second pooling value; Step S703: Calculate the fusion feature of the face image to be classified according to the local feature, the first pooling value, the global feature, and the second pooling value, including:
[0058] Among them, F2 is the fusion feature, F0 is the local feature, V0 is the first pooling value, F1 is the global feature, and V1 is the second pooling value.
[0059] Global average pooling and summation operation refer to taking the average of all pixel values in each channel of the local feature and the global feature to form a single value, simplifying the representation of the feature, while reducing the complexity and the number of parameters of the model, so as to facilitate further feature fusion and classification operations. Fusing the global feature and the local feature, combining multi-scale information, enables the fusion feature to contain comprehensive information about the overall target and details of the face image to be classified. Constructing a more comprehensive and accurate feature representation helps reduce the possibility of misclassification, thereby improving the performance and generalization ability of the model.
[0060] Step S602: Determine the probability that the face image to be classified belongs to each face constitution category according to the fusion feature of the face image to be classified.
[0061] The fully connected layer converts the fusion vector F2 into a probability distribution of specific categories. In this application, a fully connected layer with a dimension of 5 is selected, and each dimension corresponds to a prediction score of a category, and each category corresponds to one of metal, wood, water, fire, or earth. Through the Softmax activation function, each output of the weight connection layer will be converted into a probability value between 0 and 1, and the sum of the probabilities of all categories is 1.
[0062] Step S603: Determine the face classification result of the face image to be classified according to the probability that the face image to be classified belongs to each face constitution category.
[0063] Use the argmax function to select the face classification result where the maximum value is located from the probability distribution output by the Softmax activation function. The physical characteristics of an individual can be judged according to the face classification result, and a personalized health management plan can be formulated for traditional Chinese medicine health preservation guidance, etc.
[0064] This application provides a face classification method based on a classification model. In the classification process, three-dimensional facial data is combined, and the face classification result is analyzed based on the global feature and the local feature, more comprehensively exploring and verifying the relationship between human facial features and physical constitution, improving the accuracy and robustness of face classification, and ensuring high efficiency and scalability.
[0065] The present invention also provides a computer device, which may include: a memory storing executable program code; A processor coupled to a memory; A transceiver for communicating with other devices or a communication network to receive or send network messages; A bus for connecting the memory, the processor, and the transceiver for internal communication.
[0066] The transceiver receives the messages transmitted on the network, passes them to the processor through the bus. The processor calls the executable program code stored in the memory through the bus for processing, and passes the processing result to the transceiver through the bus for sending, thereby implementing the method provided by the embodiments of the present application.
[0067] The embodiments of the present application also provide a non-transitory machine-readable storage medium, on which an executable program is stored. When the executable program is run by the processor, the processor is caused to execute the method provided by the above embodiments.
[0068] The embodiments of the present invention disclose a computer-readable storage medium, which stores a computer program for electronic data exchange. Wherein, the computer program causes a computer to execute the described method.
[0069] The embodiments of the present invention disclose a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the described method.
[0070] The above embodiments can be referred to the description of the method and will not be elaborated here.
[0071] The embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.
[0072] Through the above specific descriptions of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, and the storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.
[0073] Finally, it should be noted that: what is disclosed in the embodiments of the present invention is only the preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A face classification method based on a classification model, characterized in that: The classification model includes a local feature extraction network and a global feature extraction network, and the method includes: Obtain the face image to be classified; Inputting the face image to be classified into a classification model, and using a local feature extraction network to extract a feature map and local features of the face image to be classified; Extracting global features of the face image to be classified using a global feature extraction network; Based on the local features and the global features, a face classification result is obtained.
2. The face classification method based on the classification model according to claim 1, characterized in that: Using a global feature extraction network to extract global features of the face image to be classified includes: Performing fuzzy processing and adaptive threshold segmentation on the face image to be classified to generate a face contour map of the face image to be classified; Adjusting the size of the face contour map according to the feature map to obtain a contour mask; The local features are combined with the contour mask to obtain global features.
3. The face classification method based on the classification model according to claim 1, characterized in that: Based on the local features and the global features, a face classification result is obtained, including: Fusing the local features with the global features to obtain fused features of the face image to be classified; Determining the probability that the face image to be classified belongs to each face physical category according to the fusion features of the face image to be classified; The face classification result of the face image to be classified is determined according to the probability that the face image to be classified belongs to each face physical category.
4. The face classification method based on the classification model according to claim 3, characterized in that: The local features and the global features are integrated to obtain the integrated features of the face image to be classified, including: Performing global average pooling and summing operations on the local features to obtain a first pooling value; Performing global average pooling and summing operations on the global features to obtain a second pooling value; The fusion features of the face image to be classified are calculated according to the local features, the first pooling values, the global features and the second pooling values.
5. The face classification method based on the classification model according to claim 4, characterized in that: Calculating the fusion feature of the face image to be classified according to the local feature, the first pooling value, the global feature and the second pooling value includes: Among them, F2 is the fusion feature, F0 is the local feature, V0 is the first pooling value, F1 is the global feature, and V1 is the second pooling value.
6. A face classification method based on a classification model according to claim 5, characterized in that: The classification model is obtained by training with a plurality of training images and a classification label of each training image using a target loss function.
7. The face classification method based on the classification model according to claim 6, characterized in that: The target loss function includes a first loss function, which is L1=-∑abs(F)×M Among them, F is the feature map output by the local feature extraction network, and M is the contour mask.
8. The face classification method based on a classification model according to claim 6, characterized in that: The target loss function also includes a second loss function, which is in, is the actual value of the i-th classification label, is the predicted probability of the i-th classification label.
9. A computer device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of a face classification method based on a classification model as claimed in any one of claims 1 to 8.
10. A computer storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of a face classification method based on a classification model as claimed in any one of claims 1 to 8 are implemented.