Classification methods, devices, equipment and storage media for human facial features
By extracting multi-layer feature maps through deep residual networks and using attention mechanisms for feature fusion and weight setting, the problem of ignoring other feature information in existing face classification tasks is solved, achieving higher classification accuracy and robustness.
Patent Information
- Application Number
- CN202310236273.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-13
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-03-13
AI Technical Summary
Current face classification tasks ignore valuable feature information other than the last layer of feature maps, resulting in low classification accuracy.
Multi-layer feature maps are extracted using a deep residual network, and a significant feature region segmentation is performed using an attention mechanism to generate local feature maps. Feature fusion and weight settings are then applied, and finally, category analysis is performed using the target feature map.
It improves the accuracy and robustness of face classification, enhances feature representation capabilities, and improves the performance of the classification model.
Smart Images

Figure CN117975520B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, device, and storage medium for classifying facial features. Background Technology
[0002] Due to the characteristic of concentrated features in facial images, face classification tasks face greater challenges than conventional image classification tasks in real-world face classification scenarios. Although convolutional neural networks can effectively construct discriminative feature representations, existing face classification tasks only use the feature map of the last layer of the network model for classification, ignoring other valuable feature information layers. Furthermore, directly integrating multiple feature layers will cause redundancy and thus affect classification performance. Summary of the Invention
[0003] In view of this, embodiments of the present invention provide a method, apparatus, device and storage medium for classifying facial features, which solves the technical problem of low accuracy when classifying faces using facial images.
[0004] The first aspect of this invention provides a method for classifying facial features, comprising: acquiring an image to be classified; inputting the image to be classified into a preset deep residual network for multi-layer feature extraction to obtain corresponding multi-layer feature maps; segmenting the image to be classified into salient feature regions using an attention mechanism to determine multiple local feature maps; fusing the multi-layer feature maps to generate multiple fused feature maps; setting weights for the multiple fused feature maps based on the multiple local feature maps to obtain a target feature map; and performing category analysis on the image to be classified using the target feature map to obtain the corresponding face type and complete the classification of the image to be classified.
[0005] In conjunction with the first aspect, in the first embodiment of the first aspect, the step of acquiring the image to be classified and inputting the image to be classified into a preset deep residual network for multi-layer feature extraction to obtain a corresponding multi-layer feature map includes: acquiring the image to be classified, inputting the image to be classified into the deep residual network for feature extraction to obtain a corresponding feature map set; and performing standardization processing on the feature map set to obtain a corresponding multi-layer feature map.
[0006] In conjunction with the first embodiment of the first aspect, in the second embodiment of the first aspect, the step of standardizing the feature map set to obtain the corresponding multi-layer feature map includes: performing upsampling processing on each high-dimensional feature map in the feature map set to obtain a target high-dimensional feature map set; performing convolution operation on each low-dimensional feature map in the feature map set to obtain a target low-dimensional feature map set; and merging the target high-dimensional feature map set and the target low-dimensional feature map set to obtain the corresponding multi-layer feature map.
[0007] In conjunction with the first aspect, in the third embodiment of the first aspect, the step of fusing features from the multi-layer feature maps to generate multiple fused feature maps includes: performing global average pooling on the multi-layer feature maps to obtain a corresponding global receptive field; performing pointwise dimensionality reduction convolution on the multi-layer feature maps through the global receptive field to obtain multi-layer feature maps to be processed; performing activation processing on the multi-layer feature maps to be processed to obtain multi-layer candidate feature maps; and performing dimensionality increase convolution on the multi-layer candidate feature maps to obtain multiple fused feature maps.
[0008] In conjunction with the first aspect, in the fourth embodiment of the first aspect, the step of setting weights on multiple fused feature maps based on multiple local feature maps to obtain a target feature map includes: extracting weight information from each local feature map and each fused feature map to determine a weight information set, wherein the weight information set includes weight information corresponding to each local feature map and weight information corresponding to each fused feature map; and setting weights on the multiple fused feature maps using the weight information set to obtain the target feature map.
[0009] In conjunction with the fourth embodiment of the first aspect, in the fifth embodiment of the first aspect, the step of setting weights for the plurality of fused feature maps using the weight information set to obtain a target feature map includes: performing element-wise multiplication on the plurality of local feature maps and the plurality of fused feature maps using the weight information set to obtain a plurality of weighted feature maps; and performing concatenation on the plurality of weighted feature maps to generate the target feature map.
[0010] In conjunction with the first aspect or the first to fifth embodiments of the first aspect, in the sixth embodiment of the first aspect, the step of performing category analysis on the image to be classified using the target feature map to obtain the corresponding face type includes: numerically mapping the target feature map using a preset normalization function to obtain the corresponding feature value; calculating the loss value of the feature value using a cross-entropy loss function to obtain the target loss value; and performing category analysis on the image to be classified using the target loss value to obtain the corresponding face type.
[0011] According to a second aspect, embodiments of the present invention provide a face classification device based on local features, comprising: The extraction module is used to acquire the image to be classified, input the image to be classified into a preset deep residual network for multi-layer feature extraction, and obtain the corresponding multi-layer feature map; A segmentation module, electrically connected to the extraction module, is used to segment the salient feature regions of the image to be classified using an attention mechanism to determine multiple local feature maps. A fusion module, electrically connected to the segmentation module, is used to perform feature fusion on the multi-layer feature maps to generate multiple fused feature maps. The setting module is electrically connected to the fusion module and is used to set weights on the multiple fused feature maps based on the multiple local feature maps to obtain a target feature map. An analysis module, electrically connected to the setting module, is used to perform category analysis on the image to be classified using the target feature map, obtain the corresponding face type, and complete the classification of the image to be classified.
[0012] According to a third aspect, embodiments of the present invention provide an electronic device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the facial feature classification method described in the first aspect or any embodiment of the first aspect.
[0013] According to a fourth aspect, embodiments of the present invention provide a computer-readable storage medium storing computer instructions for causing the computer to perform the facial feature classification method described in the first aspect or any embodiment of the first aspect.
[0014] In this invention, an image to be classified is acquired and input into a pre-set deep residual network for multi-layer feature extraction, resulting in corresponding multi-layer feature maps. A significant feature region segmentation is performed on the image to be classified using an attention mechanism to determine multiple local feature maps. Feature fusion is then performed on the multi-layer feature maps to generate multiple fused feature maps. Weights are set on the multiple fused feature maps based on the multiple local feature maps to obtain a target feature map. The target feature map is then used to perform category analysis on the image to be classified, obtaining the corresponding face type and completing the classification of the image. This invention extracts and fuses feature information at different scales from the input image to be classified, thereby obtaining richer, more complete, and more accurate input data feature information. Considering face features comprehensively, a local feature attention mechanism is proposed, using the local attention feature map to assign higher weights to local facial regions and providing these weighted features to the face classification framework to better improve the performance and accuracy of the face classification model. Attached Figure Description
[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0016] Figure 1 This is a flowchart of a method for classifying facial features in an embodiment of the present invention.
[0017] Figure 2 This is a flowchart illustrating the process of inputting the image to be classified into a preset deep residual network for multi-layer feature extraction in an embodiment of the present invention.
[0018] Figure 3 This is a flowchart illustrating the weighting of multiple fused feature maps in an embodiment of the present invention.
[0019] Figure 4 This is a flowchart illustrating the weight setting of multiple fused feature maps using a weight information set in an embodiment of the present invention.
[0020] Figure 5 This is a schematic diagram of a face classification device based on local features in an embodiment of the present invention.
[0021] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention.
[0022] Figure label: 501. Extraction module; 502. Fusion module; 503. Segmentation module; 504. Setting module; 505. Analysis module; 601. Processor; 602. Memory. Detailed Implementation
[0023] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0025] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0026] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 , Figure 1 This is a flowchart of a facial feature classification method according to an embodiment of the present invention, such as... Figure 1 As shown, the flowchart includes the following steps: Step S101: Obtain the image to be classified, input the image to be classified into a preset deep residual network for multi-layer feature extraction, and obtain the corresponding multi-layer feature map; It is understood that the executing entity of this invention can be a face classification system based on local features, or it can be a terminal or a server; no specific limitation is made here. This embodiment of the invention will be described using a server as an example.
[0027] It should be noted that this deep residual network has four residual layers, each containing 3, 4, 6, and 3 residual modules respectively. The last feature layer of each residual layer is denoted as... Finally, a fully connected operation is performed to facilitate the classification task. In this embodiment of the invention, in the first step, the server inputs the image to be classified into the first residual layer of the deep residual network for calculation. Specifically, the server passes the image to be classified through a convolutional layer, a BN layer, a ReLU activation function, and a MaxPooling layer for calculation, obtaining output features with a scale of (64, 56, 56). Then The input is fed into the first residual block, which contains some... convolution and Convolution, in order to better utilize input features Combine it with the output features of the residual block Combine them to obtain the output. Furthermore, the server will separately assign features The input is fed into residual blocks 4, 6, and 3 of the remaining 3 residual layers, and the same residual operation is performed to obtain the corresponding multi-layer feature maps. Step S102: Segment the salient feature regions of the image to be classified using an attention mechanism to determine multiple local feature maps;
[0028] Specifically, in this step, the server uses an attention mechanism to segment the salient feature regions of the image to be classified, extracts fine-grained local regions in the image to be classified, finds the salient feature regions in the image to be classified and assigns them higher weights, so as to take into account both the extraction of global semantic features and local detail features of the image, and determines multiple local feature maps, which can highlight the importance of local detail information in face analysis and further improve the accuracy of face classification through face images. Step S103: Perform feature fusion on the multi-layer feature maps to generate multiple fused feature maps;
[0029] It should be noted that, in order to strengthen inter-layer information interaction and enhance the feature representation capability of the network structure, in this embodiment of the invention, the server fuses features from different depth levels. Specifically, the server extracts feature maps from layers 1, 3, and 4 of the four residual layers in the deep residual network, and fuses the extracted feature maps from layer 3 and layer 4. The residual structure can be used to supplement the feature map of layer 4, resulting in an enhanced high-level feature map. At the same time, the features in layer 1 are combined with the deep features. By fusing the features to obtain multiple fused feature maps, the robustness of this method for feature fusion of multi-layer feature maps can be improved.
[0030] Furthermore, the server aggregates the multi-layer feature maps extracted from the source into multi-layer contextual features through channel-wise convolution and pointwise convolution. Specifically, the server first performs same-dimensional processing on the shallow and deep feature maps during feature extraction, and then sums them element-wise to obtain the same-dimensional feature map. Next, global average pooling is performed on the same-dimensional feature map along the channel direction to obtain the global receptive field. Then, a pointwise convolution is used to reduce the dimensionality and compress it, followed by activation operations. Then, pointwise convolution is used to increase the dimensionality to add more non-linear processing and fit the complex correlation between channels. Finally, a batch normalization (BN) layer is added to obtain multiple fused feature maps. Step S104: Set weights for multiple fused feature maps based on multiple local feature maps to obtain the target feature map;
[0031] Specifically, after the server generates local feature maps corresponding to different regions using an attention mechanism, each local feature map represents a part of a specific object. The server then uses multiple fused feature maps extracted from a deep residual network. Convolutional operations are used to generate local feature maps representing different parts. The server then assigns weights from the generated local feature maps to multiple fused feature maps and concatenates these weighted fused feature maps to obtain the target feature map.
[0032] Step S105: Perform category analysis on the image to be classified using the target feature map to obtain the corresponding face type and complete the classification of the image to be classified.
[0033] It should be noted that in the steps of this invention, the server uses the cross-entropy loss function to verify the correctness of the category analysis. The cross-entropy in the cross-entropy loss function can be regarded as the difference in information content between two probability systems. It is mainly used to describe the similarity between the distribution of real samples and the distribution of predicted labels of the trained model. Finally, the server uses the cross-entropy loss function to perform category analysis on the image to be classified and obtain the corresponding face type.
[0034] By performing the above steps, an image to be classified is obtained. This image is then input into a pre-set deep residual network for multi-layer feature extraction, resulting in corresponding multi-layer feature maps. A significant feature region segmentation is performed on the image to be classified using an attention mechanism, determining multiple local feature maps. Feature fusion is then performed on the multi-layer feature maps to generate multiple fused feature maps. Weights are set on these fused feature maps based on the multiple local feature maps to obtain a target feature map. Finally, the target feature map is used to perform category analysis on the image to be classified, obtaining the corresponding face type and completing the classification of the image. In this invention, feature information at different scales is extracted from the input image to be classified and fused to obtain richer, more complete, and more accurate input data feature information. Considering face features comprehensively, a local feature attention mechanism is proposed. This mechanism uses local attention feature maps to assign higher weights to local facial regions and provides these weighted features to the face classification framework to better improve the performance and accuracy of the face classification model.
[0035] In one specific embodiment, the process of executing step S103 may specifically include the following steps: (1) Perform global average pooling on the multi-layer feature map to obtain the corresponding global receptive field; (2) Perform point-by-point dimensionality reduction convolution on the multi-layer feature map through the global receptive field to obtain the multi-layer feature map to be processed; (3) Activation processing is performed on the multi-layer feature maps to be processed to obtain multi-layer candidate feature maps; (4) Perform dimensionality-increasing convolution operation on the multi-layer candidate feature maps to obtain multiple fused feature maps.
[0036] Furthermore, the server aggregates the multi-layer feature maps extracted from the source into multi-layer contextual features through channel-wise convolution and pointwise convolution. Specifically, the server first performs same-dimensional processing on the shallow and deep feature maps during feature extraction, and then sums them element-wise to obtain the same-dimensional feature map. Next, global average pooling is performed on the same-dimensional feature map along the channel direction to obtain the global receptive field. Then, a pointwise convolution is used to reduce the dimensionality and compress it, followed by activation operations. Then, pointwise convolution is used to increase the dimensionality to add more non-linear processing and fit the complex correlation between channels. Finally, a batch normalization (BN) layer is added to obtain multiple fused feature maps.
[0037] Specifically, the server performs global average pooling on multiple fused feature maps to obtain the global receptive field, then performs dimensionality reduction and compression through a pointwise convolution, followed by activation, and then performs pointwise convolution to increase dimensionality to add more non-linear processing and fit the complex correlation between channels.
[0038] In one specific embodiment, the process of executing step S105 may specifically include the following steps: (1) The target feature map is numerically mapped by a preset normalization function to obtain the corresponding feature values; (2) The target loss value is obtained by calculating the loss value of the feature values through the cross-entropy loss function; (3) The target loss value is used to perform category analysis on the image to be classified, obtain the corresponding face type, and complete the classification of the image to be classified.
[0039] Specifically, the method of this invention first maps the target feature map numerically using a normalization function to obtain the corresponding feature values. The server maps the feature values to the (0,1) interval. Then, it uses the cross-entropy function to calculate the loss on the list of scores output by the model, as shown in the following formula: in Represents the cross-entropy loss function. Indicates sample label, This indicates the number of samples in a batch. This represents the corresponding feature value, thereby enabling face classification.
[0040] It should be noted that, in order to quantitatively evaluate the performance of the algorithm, this invention uses Accuracy (Acc) as the evaluation metric. Acc refers to the ratio of the number of samples correctly classified by the classifier to the total number of samples in a given test dataset, and is generally expressed as a percentage. The specific formula is as follows: Wherein, TP represents a positive prediction and a positive actual value, FP represents a positive prediction and a negative actual value, TN represents a negative prediction and a positive actual value, and FN represents a negative prediction and a positive actual value. The accuracies in this embodiment are 96.84%, 86.62%, and 97.12%, respectively. The table below lists the experimental results of this embodiment. It should be noted that Table 1 represents the face classification accuracy values obtained after multi-layer feature extraction and fusion of face images, and assigning weights to discriminative local features. Table 1
[0041] As can be seen from Table 1, the three datasets have their own characteristics, but they are all based on natural scenes. As can be seen from the table above, the present invention has achieved good classification results on all three datasets.
[0042] In one specific embodiment, such as Figure 2 As shown, the process of executing step S101 can specifically include the following steps: S201: Obtain the image to be classified, input the image to be classified into a deep residual network for feature extraction, and obtain the corresponding feature map set; S202: Standardize the feature map set to obtain the corresponding multi-layer feature map.
[0043] In one specific embodiment, the process of performing step S202 may specifically include the following steps: (1) Upsample each high-dimensional feature map in the feature map set to obtain the target high-dimensional feature map set; (2) Perform convolution operation on each low-dimensional feature map in the feature map set to obtain the target low-dimensional feature map set; (3) Merge the target high-dimensional feature map set and the target low-dimensional feature map set to obtain the corresponding multi-layer feature map.
[0044] Specifically, to enhance inter-layer information interaction and improve the feature representation capability of the network structure, in this embodiment of the invention, the server fuses features from different depth levels. Specifically, the server extracts feature maps from layers 1, 3, and 4 of the four residual layers in the deep residual network, and fuses the extracted feature maps from layers 3 and 4. The residual structure can be used to supplement the feature map of layer 3 as a supplement to the feature map of layer 4, resulting in an enhanced high-level feature map. At the same time, the features in layer 1 are combined with the deep features. By fusing the features to obtain multiple fused feature maps, the robustness of this method for feature fusion of multi-layer feature maps can be improved.
[0045] Specifically, when the server performs standardization processing on the feature map set, it upsamples each high-dimensional feature map in the feature map set to obtain a target high-dimensional feature map set, performs convolution operation on each low-dimensional feature map in the feature map set to obtain a target low-dimensional feature map set, and merges the target high-dimensional feature map set and the target low-dimensional feature map set to obtain the corresponding multi-layer feature map.
[0046] In one specific embodiment, such as Figure 3 As shown, the process of executing step S104 can specifically include the following steps: S301: Extract weight information for each local feature map and each fused feature map, and determine the weight information set, wherein the weight information set includes the weight information corresponding to each local feature map and the weight information corresponding to each fused feature map. S302: Weights are set for multiple fused feature maps using a weight information set to obtain the target feature map.
[0047] In this solution, specifically, after the server generates local feature maps corresponding to different regions using an attention mechanism, each local feature map represents a part of a specific object. The server then uses multiple fused feature maps extracted from a deep residual network. Convolutional operations are used to generate local feature maps representing different parts. The server then assigns weights from the generated local feature maps to multiple fused feature maps and concatenates these weighted fused feature maps to obtain the target feature map.
[0048] The server performs element-wise multiplication on each fused feature map and each local feature map to determine the weight information corresponding to each local feature map and the weight information corresponding to each fused feature map. The server sets the weights of multiple fused feature maps using the weight information set to obtain the target feature map. When the server sets the weights of multiple fused feature maps, it sets the weights of multiple fused feature maps using the weight information set to obtain a weighted feature map. Then, the server concatenates the feature maps of each feature fusion to obtain a set of feature maps with local feature weights, and finally obtains the target feature map.
[0049] It should be noted that, in order to verify the effectiveness of the multi-layer feature extraction and fusion and the attention mechanism for adding local features with distinguishable rows in the method of this invention, the ResNet50 network with two modules added respectively was trained on three datasets, including the LFW dataset, and the face classification accuracy values for each dataset were obtained. The results are shown in Table 2. Table 2 mainly shows the impact of the effectiveness of adding multi-layer feature extraction and fusion and the attention mechanism for adding local features with distinguishable rows in the deep residual network on the face expression recognition rate. Table 2
[0050] Table 2 shows that the ResNet50 networks with two modules added respectively were trained on three datasets, including the LFW dataset. The results in Table 2 effectively demonstrate that enhancing the interaction between shallow and deep networks and mining facial features from different levels can provide the model with more facial feature descriptions and improve network performance. Simultaneously, the ResNet50 network with added local feature attention also showed improved classification performance on all three datasets, demonstrating the importance of local regions such as the eyes, nose, and forehead for face classification tasks.
[0051] In one specific embodiment, such as Figure 4 As shown, the process of executing step S302 specifically includes the following steps: S401: Multiple weighted feature maps are obtained by element-wise multiplication of multiple local feature maps and multiple fused feature maps using a set of weighted information. S402: Consolidate multiple weighted feature maps to generate a target feature map.
[0052] Specifically, the server first performs element-wise multiplication on multiple local feature maps and multiple fused feature maps using a weight information set to obtain multiple weighted feature maps. In this embodiment of the invention, the server performs same-dimensional processing on shallow feature maps and deep feature maps during feature extraction, and then performs element-wise multiplication and summation to obtain multiple weighted feature maps. The multiple weighted feature maps are then concatenated to generate the target feature map.
[0053] This invention also provides a face classification device based on local features, such as... Figure 5 As shown, the face classification device based on local features specifically includes: The extraction module 501 is used to acquire the image to be classified, input the image to be classified into a preset deep residual network for multi-layer feature extraction, and obtain the corresponding multi-layer feature map. The segmentation module 502 is electrically connected to the extraction module and is used to segment the salient feature regions of the image to be classified through an attention mechanism to determine multiple local feature maps. A fusion module 503 is electrically connected to the segmentation module and is used to perform feature fusion on the multi-layer feature maps to generate multiple fused feature maps. Setting module 504, which is electrically connected to the fusion module, is used to set weights on multiple fused feature maps based on multiple local feature maps to obtain a target feature map; Analysis module 505, which is electrically connected to the setting module, is used to perform category analysis on the image to be classified through the target feature map, obtain the corresponding face type, and complete the classification of the image to be classified.
[0054] Optionally, the extraction module 501 specifically includes: An extraction unit is used to acquire the image to be classified, input the image to be classified into the deep residual network for feature extraction, and obtain the corresponding feature map set. The processing unit is used to standardize the feature map set to obtain the corresponding multi-layer feature map.
[0055] Optionally, the processing unit is specifically used to: perform upsampling processing on each high-dimensional feature map in the feature map set to obtain a target high-dimensional feature map set; perform convolution operation on each low-dimensional feature map in the feature map set to obtain a target low-dimensional feature map set; and merge the target high-dimensional feature map set and the target low-dimensional feature map set to obtain a corresponding multi-layer feature map.
[0056] Optionally, the fusion module 503 is specifically used for: performing global average pooling on the multi-layer feature maps to obtain the corresponding global receptive field; performing pointwise dimensionality reduction convolution on the multi-layer feature maps through the global receptive field to obtain multi-layer feature maps to be processed; performing activation processing on the multi-layer feature maps to be processed to obtain multi-layer candidate feature maps; and performing dimensionality increase convolution on the multi-layer candidate feature maps to obtain multiple fused feature maps.
[0057] Optionally, the setting module 504 specifically includes: The determining unit is used to extract weight information for each of the local feature maps and each of the fused feature maps, and to determine a set of weight information, wherein the set of weight information includes weight information corresponding to each of the local feature maps and weight information corresponding to each of the fused feature maps; The setting unit is used to set weights for the multiple fused feature maps using the weight information set to obtain the target feature map.
[0058] Optionally, the setting unit is specifically used to: perform element-wise multiplication on multiple local feature maps and multiple fused feature maps using the weight information set to obtain multiple weighted feature maps; and perform concatenation on the multiple weighted feature maps to generate the target feature map.
[0059] Optionally, the analysis module 505 is specifically used to: perform numerical mapping on the target feature map using a preset normalization function to obtain the corresponding feature values; calculate the loss value of the feature values using a cross-entropy loss function to obtain the target loss value; and perform category analysis on the image to be classified using the target loss value to obtain the corresponding face type.
[0060] Through the collaborative efforts of its various components, the image to be classified is acquired. This image is then input into a pre-defined deep residual network for multi-layer feature extraction, yielding corresponding multi-layer feature maps. An attention mechanism is used to segment salient feature regions in the image, identifying multiple local feature maps. Feature fusion is performed on the multi-layer feature maps to generate multiple fused feature maps. Weights are then assigned to these fused feature maps based on the multiple local feature maps to obtain a target feature map. Finally, the target feature map is used to perform category analysis on the image to be classified, determining the corresponding face type and completing the classification of the image. In this invention, feature information at different scales is extracted from the input image to be classified and fused to obtain richer, more complete, and more accurate input data feature information. Considering facial features comprehensively, a local feature attention mechanism is proposed. This mechanism assigns higher weights to local facial regions using local attention feature maps and provides these weighted features to the face classification framework to better improve the performance and accuracy of the face classification model.
[0061] This invention also provides an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 601 and a memory 602, wherein the processor 601 and the memory 602 may be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.
[0062] Processor 601 can be a Central Processing Unit (CPU). Processor 601 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of various types of chips. Memory 602, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the method in this embodiment of the invention. Processor 601 executes various functional applications and data processing, i.e., implements the method, by running the non-transitory software programs, instructions, and modules stored in memory 602.
[0063] The memory 602 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 601, etc. Furthermore, the memory 602 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 602 may optionally include memory remotely located relative to the processor 601, and these remote memories may be connected to the processor 601 via a network. Examples of networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0064] One or more modules are stored in memory 602 and are executed by processor 601.
[0065] This invention also provides a non-transitory computer storage medium storing computer-executable instructions that can execute the personnel statistics method in any of the method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of different types of memory.
[0066] Those skilled in the art will understand that all or part of the processes in the embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, which can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of various types of memory.
[0067] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.
Claims
1. A method for classifying human facial features, characterized in that, Includes the following steps: The image to be classified is obtained and input into a pre-set deep residual network for multi-layer feature extraction to obtain the corresponding multi-layer feature map. The attention mechanism is used to segment salient feature regions of the image to be classified, and multiple local feature maps are determined. Feature fusion is performed on multi-layer feature maps to generate multiple fused feature maps; The target feature map is obtained by weighting multiple fused feature maps based on multiple local feature maps. The target feature map is used to perform category analysis on the image to be classified, obtain the corresponding face type, and complete the classification of the image to be classified. The steps of fusing features from multiple feature maps to generate multiple fused feature maps include: Global average pooling is performed on the multi-layer feature maps to obtain the corresponding global receptive field; By performing point-by-point dimensionality reduction convolution on the multi-layer feature map through the global receptive field, a multi-layer feature map to be processed is obtained. Activation processing is performed on the multi-layer feature maps to be processed to obtain multi-layer candidate feature maps; Perform dimensionality-increasing convolution operations on the multi-layer candidate feature maps to obtain multiple fused feature maps; The steps for obtaining the target feature map by weighting multiple fused feature maps based on multiple local feature maps include: Weight information is extracted for each local feature map and each fused feature map to determine a weight information set, wherein the weight information set includes the weight information corresponding to each local feature map and the weight information corresponding to each fused feature map. The target feature map is obtained by setting weights for multiple fused feature maps using a set of weight information.
2. The method for classifying facial features according to claim 1, characterized in that, The steps include: acquiring the image to be classified, inputting the image into a pre-set deep residual network for multi-layer feature extraction, and obtaining the corresponding multi-layer feature map. The image to be classified is obtained and input into a deep residual network for feature extraction to obtain the corresponding feature map set. The feature map set is standardized to obtain the corresponding multi-layer feature map.
3. The method for classifying facial features according to claim 2, characterized in that, The steps for standardizing the feature map set to obtain the corresponding multi-layer feature maps include: Upsampling is performed on each high-dimensional feature map in the feature map set to obtain the target high-dimensional feature map set. Perform a convolution operation on each low-dimensional feature map in the feature map set to obtain the target low-dimensional feature map set. The target high-dimensional feature map set and the target low-dimensional feature map set are merged to obtain the corresponding multi-layer feature map.
4. The method for classifying facial features according to claim 1, characterized in that, The steps to obtain the target feature map by setting weights for multiple fused feature maps using a set of weight information include: Multiple weighted feature maps are obtained by element-wise multiplication of multiple local feature maps and multiple fused feature maps using a set of weighted information. Multiple weighted feature maps are concatenated to generate the target feature map.
5. The method for classifying facial features according to any one of claims 1-4, characterized in that, The steps for classifying an image by analyzing the target feature map to obtain the corresponding face type include: The target feature map is numerically mapped using a preset normalization function to obtain the corresponding feature values; The target loss value is obtained by calculating the loss value of the feature values using the cross-entropy loss function; The target loss value is used to perform category analysis on the image to be classified, and the corresponding face type is obtained.
6. A face classification device based on local features, used to perform the face feature classification method as described in any one of claims 1-5, characterized in that, include: The extraction module is used to acquire the image to be classified, input the image to be classified into a preset deep residual network for multi-layer feature extraction, and obtain the corresponding multi-layer feature map; A segmentation module, electrically connected to the extraction module, is used to segment salient feature regions of the image to be classified using an attention mechanism to determine multiple local feature maps. A fusion module, electrically connected to the segmentation module, is used to fuse features from multiple feature maps to generate multiple fused feature maps. The setting module is electrically connected to the fusion module and is used to set weights on multiple fused feature maps based on multiple local feature maps to obtain a target feature map. The analysis module, which is electrically connected to the setting module, is used to perform category analysis on the image to be classified through the target feature map, obtain the corresponding face type, and complete the classification of the image to be classified.
7. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the facial feature classification method according to any one of claims 1-5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the facial feature classification method according to any one of claims 1-5.
Citation Information
Patent Citations
Target segmentation system and training method thereof, and target segmentation method and device
CN113486956A
Facial color classification method and device, computer readable program medium and electronic equipment
CN115100723A
