A face age estimation method based on hybrid expert network

Abstract

The application provides a face age estimation method based on a hybrid expert network, which can be applied to the technical field of computer vision. The method comprises the following steps: performing face cutting processing on face data to be identified to generate a pretreatment image, and then inputting the pretreatment image into a feature extraction network to obtain original features; inputting the original features into a feature extraction network based on a full connection layer to obtain global features and a feature extraction network based on a hybrid expert network to obtain local features; then adaptively fusing the extracted global features and local features; finally, inputting the fused features into an output layer to obtain a prediction result of face age. The face age estimation method based on the hybrid expert network can adaptively obtain global and local features of face age, and improve the accuracy of face age estimation.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more particularly to a face age estimation method based on a hybrid expert network. Background Art

[0002] Facial age estimation has been a hot topic in the field of computer vision in recent years. Age is one of the most important biological characteristics of the human body. Facial age estimation has wide applications and strong practical value. It estimates an individual's approximate age or age range based on facial images. It has applications in a variety of fields, including security monitoring, beauty care, and missing persons. Although the development of deep learning technology has enabled the application of automated facial age estimation, the extremely complex facial feature information and the large individual differences make it difficult to accurately capture and identify age features. Therefore, image-based facial age estimation remains a challenging problem. Summary of the Invention

[0003] In view of the above problems, the present invention provides a face age estimation method based on a hybrid expert network.

[0004] According to a first aspect of the present invention, a method for estimating face age based on a hybrid expert network is provided, comprising:

[0005] Based on the authorization of the target object, the face image of the target object is preprocessed to obtain the region of interest of the face image;

[0006] Utilize the trained original feature extraction network to extract original features of the face image from the region of interest;

[0007] The trained global feature extraction network is used to process the original features to obtain the global features of the face image;

[0008] The trained local feature extraction network based on hybrid expert network is used to process the original features to obtain the local features of the face image;

[0009] The global features and local features are adaptively fused, and the feature fusion results are processed using the trained age prediction network to obtain the predicted age of the target object.

[0010] According to an embodiment of the present invention, the facial image of the target object is preprocessed based on the authorization of the target object to obtain the region of interest of the facial image, which includes:

[0011] Based on the authorization of the target object, face detection is performed on the target object to obtain the face image of the target object;

[0012] Based on the authorization of the target object, the face image is aligned, and the non-face area of ​​the aligned face image is cropped to obtain the region of interest.

[0013] According to an embodiment of the present invention, the facial image is aligned based on the authorization of the target object, and the non-face area is cropped and deleted from the aligned facial image to obtain the region of interest including:

[0014] Based on a predefined format, facial images of different angles, postures, and sizes are processed in a unified format to obtain standardized facial images;

[0015] Positioning key facial feature points of the standardized facial image to obtain key point information of the standardized facial image;

[0016] Based on the key point information, the standardized face image is calibrated through affine transformation to obtain the aligned face image;

[0017] The aligned face image is cropped, the non-face area and the background area in the aligned face image are deleted, and the deleted face image is compressed to obtain the region of interest.

[0018] According to an embodiment of the present invention, extracting original features of a face image from a region of interest using the trained original feature extraction network includes:

[0019] An original feature extraction network is constructed based on the Swin Transformer and trained to obtain a trained original feature extraction network, wherein the trained original feature extraction network includes a multi-scale feature extractor and a multi-level channel attention module;

[0020] A multi-scale feature extractor is used to extract features from the region of interest to obtain feature maps of multiple scales, and convolution dimensionality reduction operations are performed on the feature maps of multiple scales to obtain intermediate feature maps of multiple scales.

[0021] A multi-level channel attention module is used to combine multiple intermediate feature maps of different scales in the scale dimension and channel dimension to obtain the original features of the face image.

[0022] According to an embodiment of the present invention, the trained global feature extraction network includes a maximum pooling layer, a ReLU activation layer and multiple fully connected layers.

[0023] According to an embodiment of the present invention, the local features of the face image obtained by processing the original features using the trained local feature extraction network based on the hybrid expert network include:

[0024] Constructing and training a local feature extraction network based on a hybrid expert network to obtain a trained local feature extraction network based on a hybrid expert network, wherein the trained local feature extraction network based on a hybrid expert network includes a face parsing module and a hybrid expert network;

[0025] The face parsing module is used to upsample the original features multiple times, and the results of the multiple upsampling are convolved to obtain the face parsing feature map;

[0026] The hybrid expert network is used to perform feature mapping, feature flattening, routing selection and expert network feature extraction on the face parsing feature map to obtain local features.

[0027] According to an embodiment of the present invention, the hybrid expert network is used to perform feature mapping, feature flattening, routing selection, and expert network feature extraction on the face parsing feature map, and the local features obtained include:

[0028] The embedded network of the hybrid expert network is used to perform multi-layer convolution feature mapping on the face parsing feature map to obtain the initial local feature representation of the face with key facial information;

[0029] The initial local feature representation of the face is flattened using the flattening layer of the embedding network to obtain the initial local feature representation of the face with a preset dimension;

[0030] Constructing a routing network and an expert network group in a hybrid expert network based on a predefined activation function and training the routing network and the expert network group to obtain a trained routing network and a trained expert network group;

[0031] Converting the trained routing network of the initial local features of the face with preset dimensions into a probability distribution of the hybrid expert network;

[0032] Using the trained routing network to dynamically select the expert network in the trained expert network group, the expert network participating in routing is obtained, and the weight corresponding to the probability distribution is calculated;

[0033] Based on the weights corresponding to the probability distribution, the expert network involved in routing is used to extract the original features of the face of the preset dimension;

[0034] A vector operation is performed on the local features of the face obtained through the expert network feature extraction and the weights corresponding to the probability distribution to obtain the local features.

[0035] According to an embodiment of the present invention, the training of the routing network and the expert network group to obtain the trained routing network and the trained expert network group includes:

[0036] In each training round, dynamically selecting an expert network from the expert network group using the routing network, and determining the importance of the selected expert network in the expert network group by calculating the probability of the selected expert network;

[0037] In each training round, based on the calculated importance, an auxiliary loss function based on discrete coefficients is used to load balance the joint training process of the routing network and the expert network group.

[0038] According to an embodiment of the present invention, the above-mentioned method of dynamically selecting an expert network from a trained expert network group using a trained routing network to obtain an expert network participating in routing and calculating a weight corresponding to a probability distribution includes:

[0039] Comparing each probability in the probability distribution with a predefined threshold, and when the probability is greater than or equal to the predefined threshold, setting the weight of the expert network participating in the routing to 1 using a predefined activation function;

[0040] Comparing each probability in the probability distribution with a predefined threshold, and when all probabilities are less than the predefined threshold, calculating the weight of the expert network participating in the routing based on the probability distribution;

[0041] The trained routing network is used to dynamically select the expert network participating in routing, thereby making the hybrid expert network sparse.

[0042] According to an embodiment of the present invention, the adaptive fusion of global features and local features is performed, and the feature fusion result is processed using a trained age prediction network to obtain the predicted age of the target object, including:

[0043] Perform weighted operations on the global feature vector and the local feature vector respectively, and concatenate the results to obtain the feature fusion result;

[0044] The age prediction network is trained using a hybrid loss function based on Gaussian loss and Euclidean loss to obtain a trained age prediction network.

[0045] The trained age prediction network is used to process the feature fusion results to obtain the predicted age of the target object.

[0046] The above-mentioned face age estimation method based on the hybrid expert network provided by the present invention can automatically extract feature maps of various key parts of the target person's face, and by extracting the local features of the target person, it can focus on the key areas of the face; local features help to identify facial age information in the image, and increasing local feature processing can improve the accuracy and interpretability of face age estimation; at the same time, the use of the expert hybrid network can adaptively process the face analysis feature map, extract effective local features, and improve performance; in addition, the present invention can automatically adjust the weights of global and local features through adaptive feature fusion operations, thereby increasing the accuracy of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0048] Figure 1 is a flowchart of a method for estimating face age based on a hybrid expert network according to an embodiment of the present invention;

[0049] Figure 2 2 is a schematic structural diagram of a face age estimation solution based on a hybrid expert network according to an embodiment of the present invention;

[0050] Figure 3 Schematically shows a structural block diagram of a face age estimation device based on a hybrid expert network according to an embodiment of the present invention;

[0051] Figure 4 The block diagram schematically shows an electronic device suitable for implementing a face age estimation method based on a hybrid expert network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0053] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0054] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0055] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0056] To improve the accuracy of facial age estimation and address technical issues with existing solutions, this paper proposes a method for facial age estimation based on a hybrid expert network. This method uses the hybrid expert network to adaptively extract representative local features to enhance the accuracy and interpretability of age estimation. Because unconstrained images may contain multiple faces and background information, an image preprocessing module is used to crop faces, retaining facial information and removing irrelevant information.

[0057] The purpose of this invention is to solve three key problems in face age estimation: (1) How to improve the accuracy and interpretability of face age estimation based on a hybrid expert network; (2) How to adaptively obtain targeted local face age features; and (3) How to effectively fuse global and local features to improve the accuracy of face age estimation.

[0058] First, the facial image is preprocessed. During the preprocessing phase, face detection, alignment, and cropping are performed to capture the region of interest and remove redundant background information. The preprocessed image is then fed into a feature extraction network to obtain raw features. Secondly, the raw features are fed into a fully connected feature extraction network to obtain global features and a hybrid expert network-based feature extraction network to obtain local features. Finally, the extracted global and local features are adaptively fused, and the final age prediction result is obtained through the output layer.

[0059] The above technical solution provided by the present invention is further described below through specific embodiments in conjunction with the accompanying drawings.

[0060] Figure 1 4 is a flowchart of a method for estimating face age based on a hybrid expert network according to an embodiment of the present invention.

[0061] like Figure 1As shown, the above-mentioned face age estimation method based on hybrid expert network includes operations S110 to S150.

[0062] In operation S110 , based on the authorization of the target object, the face image of the target object is preprocessed to obtain a region of interest of the face image.

[0063] In an embodiment of the present invention, before acquiring the target subject's facial image, the target subject's consent or authorization may be obtained. For example, before operation S110, a request to acquire user information may be issued to the target subject. If the target subject agrees or authorizes acquisition of the target subject's facial image, operation S110 is performed.

[0064] In an embodiment of the present invention, a corresponding operation entry can be provided for the target subject, allowing the target subject to choose whether to agree or reject the automated decision result. That is, before processing / decision-making on the target subject's facial image, an instruction to agree or reject the processing / decision-making can be obtained from the target subject through the corresponding operation entry. If the target subject agrees to the processing / decision-making, the processing / decision-making on the target subject's facial image is performed, that is, step S110 is executed. If the target subject rejects the processing / decision-making, the expert decision-making process is entered.

[0065] In operation S110, the face image is input into a preprocessing module to perform face detection, alignment and cropping, remove irrelevant areas, retain the face area of ​​interest, and then input the cropped image into a feature extraction network to obtain original features.

[0066] In operation S110, when the image quality is not high, such as when the face is tilted, the accuracy of face age estimation will be affected. The present invention uses the InsightFace tool to perform face detection, alignment, and cropping to automatically obtain a more accurate facial image, eliminate irrelevant information, reduce background information interference, and improve age estimation accuracy.

[0067] In operation S120, the trained original feature extraction network is used to extract raw features from the facial image in the region of interest. The cropped facial image is input into the Swin Transformer-based feature extraction network. The specific steps are as follows: First, the shared backbone network Swin Transformer extracts feature maps at four different scales. Then, a multi-level channel attention module integrates the facial features at different scales and channels to obtain raw features.

[0068] In operation S130 , the trained global feature extraction network is used to process the original features to obtain global features of the face image.

[0069] For example: the original features are fed into the global feature extraction channel, where the global feature extraction channel consists of a maximum pooling layer, a ReLU activation layer, and two fully connected layers.

[0070] In operation S140 , the trained local feature extraction network based on the hybrid expert network is used to process the original features to obtain local features of the face image.

[0071] For example, the original features are fed into the local feature extraction channel, where the processing process of the local feature extraction channel is as follows: a series of upsampling operations are performed to generate a face parsing feature map, and then the hybrid expert network is used to adaptively extract effective local face features.

[0072] In operation S150 , the global features and the local features are adaptively fused, and the feature fusion result is processed using the trained age prediction network to obtain the predicted age of the target object.

[0073] The global features and local features are adaptively fused; the fused features are used for age prediction through the output layer.

[0074] For example, the extracted global features and local features are adaptively fused to obtain new features. The sum of the global age weighting value and the local age weighting value is 100%, the global age weighting value is 10-90%, and the rest is the local age weighting value.

[0075] The above-mentioned face age estimation method based on the hybrid expert network provided by the present invention can automatically extract feature maps of various key parts of the target person's face, and by extracting the local features of the target person, it can focus on the key areas of the face; local features help to identify facial age information in the image, and increasing local feature processing can improve the accuracy and interpretability of face age estimation; at the same time, the use of the expert hybrid network can adaptively process the face analysis feature map, extract effective local features, and improve performance; in addition, the present invention can automatically adjust the weights of global and local features through adaptive feature fusion operations, thereby increasing the accuracy of the network.

[0076] According to an embodiment of the present invention, the above-mentioned preprocessing of the facial image of the target object based on the authorization of the target object to obtain the region of interest of the facial image includes: performing face detection on the target object based on the authorization of the target object to obtain the facial image of the target object; aligning the facial images based on the authorization of the target object, and obtaining the region of interest by cropping and deleting non-face areas of the aligned facial images.

[0077] According to an embodiment of the present invention, the above-mentioned facial images are aligned based on the authorization of the target object, and the non-face area is deleted by cropping the aligned facial images to obtain the area of ​​interest, including: based on a predefined format, facial images of different angles, different postures, and different sizes are processed to a unified specification to obtain a standardized facial image; key facial feature points of the standardized facial image are located to obtain key point information of the standardized facial image; based on the key point information, the standardized facial image is calibrated through affine transformation to obtain an aligned facial image; the aligned facial image is cropped to delete the non-face area and background area in the aligned facial image, and the deleted facial image is compressed to obtain the area of ​​interest.

[0078] In order to improve the accuracy of face age estimation, it is necessary to preprocess the image and extract original features. Image preprocessing includes three parts: face detection, face alignment and face cropping.

[0079] First, for a size of Grayscale image of First, based on face detection, the face to be analyzed is determined. If there are multiple faces in an image, the numbers are superimposed in sequence.

[0080] Secondly, facial images of different angles, poses, and sizes are uniformly normalized, and key facial feature points, including eyes, nose tip, mouth corners, and eyebrows, are located using MTCNN. Using this key point information, the face is calibrated through an affine transformation.

[0081] Finally, the calibrated face image is cut to remove the background and non-face areas, so that the network pays more attention to the foreground, and the image is scaled to , which can not only preserve facial details but also reduce the computational burden and memory requirements during training.

[0082] According to an embodiment of the present invention, the above-mentioned extraction of original features of a facial image from a region of interest using a trained original feature extraction network includes: constructing an original feature extraction network based on a Swin Transformer and training the original feature extraction network to obtain a trained original feature extraction network, wherein the trained original feature extraction network includes a multi-scale feature extractor and a multi-level channel attention module; using the multi-scale feature extractor to extract features of the region of interest to obtain feature maps of multiple scales, and performing convolution dimensionality reduction operations on the feature maps of multiple scales to obtain intermediate feature maps of multiple scales; using the multi-level channel attention module to combine the intermediate feature maps of multiple scales in the scale dimension and the channel dimension to obtain the original features of the facial image.

[0083] The original feature extraction network is built based on the Swin Transformer network.

[0084] The cropped image I is input into the feature extractor based on Swin Transformer to extract four feature maps of different scales, namely : , , , And through a multi-level channel attention module, facial features of different scales and channels are combined to obtain the original facial features. The specific operation is as follows: first, the feature map Unify the scale and use convolution operation to reduce the dimension, as shown in formulas (1) to (4):

[0085] (1),

[0086] (2),

[0087] (3),

[0088] (4).

[0089] Fuse each feature map and use the channel attention module to obtain the original features, as shown in formula (5):

[0090] (5).

[0091] According to an embodiment of the present invention, the above-mentioned global feature extraction network based on the trained fully connected network is used to process the original features, and the global feature processing includes: a maximum pooling layer, a ReLU activation function layer and multiple fully connected layers.

[0092] According to an embodiment of the present invention, the above-mentioned use of the trained local feature extraction network based on the hybrid expert network to process the original features to obtain local features of the face image includes: constructing a local feature extraction network based on the hybrid expert network and training it to obtain a trained local feature extraction network based on the hybrid expert network, wherein the trained local feature extraction network based on the hybrid expert network includes a face parsing module and a hybrid expert network; using the face parsing module to upsample the original features multiple times, and performing convolution processing on the results of the multiple upsamplings to obtain a face parsing feature map; using the hybrid expert network to perform feature mapping, feature flattening, routing selection and expert network feature extraction processing on the face parsing feature map to obtain local features.

[0093] According to an embodiment of the present invention, the above-mentioned use of a hybrid expert network to perform feature mapping, feature flattening, routing selection and expert network feature extraction processing on the face parsing feature map to obtain local features includes: using the embedded network of the hybrid expert network to perform multi-layer convolution feature mapping on the face parsing feature map to obtain an initial local feature representation of the face with key face information; using the flattening layer of the embedded network to flatten the initial local feature representation of the face to obtain an initial local feature representation of the face of a preset dimension; constructing a routing network and an expert network group in the hybrid expert network based on a predefined activation function and training the routing network and the expert network group to obtain A trained routing network and a trained expert network group; converting the initial local features of a face of a preset dimension into a probability distribution of a hybrid expert network through the trained routing network; using the trained routing network to dynamically select an expert network from the trained expert network group to obtain an expert network participating in routing, and calculating weights corresponding to the probability distribution; based on the weights corresponding to the probability distribution, using the expert network participating in routing to extract features of the original features of the face of the preset dimension; performing vector operations on the local features of the face obtained through feature extraction of the expert network and the weights corresponding to the probability distribution to obtain the local features.

[0094] According to an embodiment of the present invention, the above-mentioned training of the routing network and the expert network group to obtain the trained routing network and the trained expert network group includes: in each training round, using the routing network to dynamically select an expert network from the expert network group, and determining the importance of the selected expert network in the expert network group by calculating the probability of the selected expert network; in each training round, based on the calculated importance, using an auxiliary loss function based on a discrete coefficient to load balance the joint training process of the routing network and the expert network group.

[0095] According to an embodiment of the present invention, the above-mentioned method of dynamically selecting an expert network from a trained expert network group using a trained routing network to obtain an expert network participating in routing, and calculating a weight corresponding to a probability distribution includes: comparing each probability in the probability distribution with a predefined threshold, and when the probability is greater than or equal to the predefined threshold, setting the weight of the expert network participating in routing to 1 using a predefined activation function; comparing each probability in the probability distribution with a predefined threshold, and when the probabilities are all less than the predefined threshold, calculating the weight of the expert network participating in routing based on the probability distribution; and dynamically selecting an expert network participating in routing using the trained routing network to make the hybrid expert network sparse.

[0096] The following is a detailed description of the embodiments and the attached Figure 2 The local feature extraction process involved in the present invention is further described in detail.

[0097] Figure 2 3 is a structural diagram of a face age estimation solution based on a hybrid expert network according to an embodiment of the present invention.

[0098] It should be noted that Figure 2 The facial images involved are derived from open source datasets, which are authorized by the relevant users and used for non-commercial purposes such as scientific research and technical verification with the permission of the relevant users. The facial images of the target person used in the present invention have been authorized by the above-mentioned open source datasets and are only used to illustrate the method provided by the present invention and to verify the advantages and effectiveness of the method provided by the present invention, and will not be used for commercial or other profit-making purposes.

[0099] like Figure 2 As shown, the original facial features obtained by the backbone network Input into the feature extractor based on hybrid expert network to obtain local features. The feature extractor based on hybrid expert network consists of two parts: face parsing module and hybrid expert network. The face parsing module can refine the semantic information of the face and decompose the feature maps of each key part of the face, which helps to extract targeted local features in the subsequent process, thereby improving the accuracy of face estimation. After two upsampling layers and one convolution layer, the face parsing feature map is obtained. .

[0100] Due to the large differences in facial features, in order to adaptively process the targeted facial parsing feature map and improve the accuracy and interpretability of the network, the obtained facial parsing feature map Send it to the hybrid expert module to obtain targeted local features The hybrid expert module includes: an embedding network, a learnable routing network and a set of expert networks built on Resnet18 .

[0101] like Figure 2 As shown in Figure 1: First, the local face feature map is input into the embedding network, and the feature is mapped into a potential local face feature representation to simplify processing and retain key information. The embedding network consists of The convolutional layer And Flatten layer, first size is The local feature map of the face is processed by the convolution layer. , When , as shown in formula (6):

[0102] (6).

[0103] Then, the multidimensional data output by the convolution layer is flattened. Flatten into one-dimensional linear data X for subsequent processing and better expression of local features, as shown in formula (7):

[0104] (7).

[0105] After obtaining the output of the embedding layer, the output is sent to the routing network to dynamically select the number of expert networks to participate in the training. First, the potential local facial feature representation generated by the embedding network is converted into the probability distribution of the expert network, which is used to indicate which local facial feature representation needs to be sent to which expert network. The calculation method is shown in formula (8):

[0106] (8),

[0107] in are learnable image weights, is the learnable noise weight.

[0108] Set a threshold , when the probability Existence In the case of , the task expert network selection is considered to be more professional. In this case, the Keepmax function is used, and the first route is regarded as the expert who solves the problem, and its weight is set to 1, so that the network is sparse. On the contrary, it is considered that all experts with non-zero probabilities are desirable. It is necessary to set the expert network weight according to the corresponding probability distribution to comprehensively calculate the expert output, so as to obtain targeted local features. Through the above processing mechanism, the number of experts participating in the training can be adaptively selected, and local features can be extracted more effectively. The routing network is shown in formulas (9) and (10):

[0109] (9),

[0110] (10).

[0111] Then the face analysis feature map , and the routing selection Router is input into the expert network to obtain targeted local features of the face. The expert network is built based on the pre-trained ResNet18 model. Assuming that the expert group has a total of Thus, local features are obtained , as shown in formula (11):

[0112] (11),

[0113] To improve the load balancing and give all experts more training, the importance of each expert in the expert group is represented by calculating the proportion of the probability assigned to the expert in each round of training The importance of each expert in the expert group is represented by the proportion of the probability assigned to the expert in each round of training, as shown in equation (12):

[0114] (12).

[0115] To prevent over-reliance on some expert networks during training and leave other networks idle, a coefficient of variation (CV) is used to increase an auxiliary loss , and better load balancing is achieved by minimizing , as shown in equation (13):

[0116] (13),

[0117] where CV is defined as the ratio of the standard deviation to the mean, which is a normalized measure of the degree of dispersion of the probability distribution, is a manually adjusted auxiliary loss weight.

[0118] According to the embodiments of the present application, the adaptive feature fusion of the global features and the local features is performed, and the trained age prediction network is used to process the feature fusion result to obtain the predicted age of the target object, which includes: performing weighted operations on the global feature vector and the local feature vector respectively, and performing vector splicing on the operation results to obtain the feature fusion result; training the age prediction network using a hybrid loss function based on Gaussian loss and Euclidean loss to obtain the trained age prediction network; and processing the feature fusion result using the trained age prediction network to obtain the predicted age of the target object.

[0119] In specific embodiments, the adaptive feature fusion of the global features and the local features is performed, assuming that the feature channels of the global and local two-way inputs are , The output channel of the fused features is shown in equation (14):

[0120] (14),

[0121] where is an adaptive learning parameter representing the weight of the global features and the local features.

[0122] In specific embodiments, face age estimation is classified as a regression problem, and ​​​​The input is fed into the output layer of a fully connected neural network to obtain the predicted age value. The training goal of the face age estimation method based on the hybrid expert network is to make the predicted age value as consistent as possible with the image age label value. Euclidean loss can be used to effectively evaluate this indicator. In addition, when the predicted value is far from the label value, using Gaussian loss can slow down the training process. Therefore, a weighted combination of Gaussian loss and Euclidean loss is used as the loss function for face age estimation, as shown in formula (15):

[0123] (15),

[0124] in, is the predicted value, is the true value, is the standard deviation of the annotation age.

[0125] Combining the face estimation loss and auxiliary loss, we get the overall loss function of the model, as shown in formula (16):

[0126] (16).

[0127] Based on the above-mentioned face age estimation method based on hybrid expert network, the present invention also provides a face age estimation device based on hybrid expert network. Figure 3 The device is described in detail.

[0128] Figure 3 The structure block diagram of the face age estimation device based on hybrid expert network according to an embodiment of the present invention is schematically shown.

[0129] like Figure 3 As shown, the face age estimation device 300 based on the hybrid expert network of this embodiment includes a preprocessing module 310 , an original feature extraction module 320 , a global feature acquisition module 330 , a local feature acquisition module 340 and a fusion prediction module 350 .

[0130] The preprocessing module 310 is used to preprocess the face image of the target object based on the authorization of the target object to obtain the region of interest of the face image. In one embodiment, the preprocessing module 310 can be used to perform the operation S110 described above, which will not be repeated here.

[0131] The original feature extraction module 320 is used to extract original features of the face image from the region of interest using the trained original feature extraction network. In one embodiment, the original feature extraction module 320 can be used to perform the operation S120 described above, which will not be repeated here.

[0132] The global feature acquisition module 330 is used to process the original features using the trained global feature extraction network to obtain the global features of the face image. In one embodiment, the global feature acquisition module 330 can be used to perform the operation S130 described above, which will not be repeated here.

[0133] The local feature acquisition module 340 is used to process the original features using the trained local feature extraction network based on the hybrid expert network to obtain local features of the face image. In one embodiment, the local feature acquisition module 340 can be used to perform the operation S140 described above, which will not be repeated here.

[0134] Fusion prediction module 350 is configured to adaptively fuse global and local features and process the fusion results using the trained age prediction network to obtain the predicted age of the target object. In one embodiment, fusion prediction module 350 can be used to perform operation S150 described above and will not be further described here.

[0135] Figure 4 The block diagram schematically shows an electronic device suitable for implementing a face age estimation method based on a hybrid expert network according to an embodiment of the present invention.

[0136] like Figure 4 As shown, an electronic device 400 according to an embodiment of the present invention includes a processor 401, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 402 or a program loaded from a storage unit 408 into a random access memory (RAM) 403. Processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 401 may also include onboard memory for caching purposes. Processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0137] Various programs and data required for the operation of the electronic device 400 are stored in the RAM 403. The processor 401, ROM 402, and RAM 403 are connected to each other via a bus 404. The processor 401 executes the programs in the ROM 402 and / or RAM 403 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 402 and RAM 403. The processor 401 may also execute the programs stored in the one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0138] According to an embodiment of the present invention, electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to bus 404. Electronic device 400 may also include one or more of the following components connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 408 including a hard disk; and a communication section 409 including a network interface card such as a LAN card or modem. Communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 410 as needed, so that computer programs read from the removable media can be installed into storage section 408 as needed.

[0139] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0140] According to an embodiment of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, a computer-readable storage medium may include ROM 402 and / or RAM 403 described above, and / or one or more memories other than ROM 402 and RAM 403.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or couplings fall within the scope of the present invention.

[0143] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A face age estimation method based on a hybrid expert network, characterized in that: The method comprises: Based on the authorization of the target object, preprocessing the facial image of the target object to obtain a region of interest of the facial image; Extracting original features of the face image from the region of interest using the trained original feature extraction network; Processing the original features using the trained global feature extraction network to obtain global features of the face image; Constructing a local feature extraction network based on a hybrid expert network and training it to obtain a trained local feature extraction network based on a hybrid expert network, wherein the trained local feature extraction network based on a hybrid expert network includes a face parsing module and a hybrid expert network, and the hybrid expert network includes an embedding network, a routing network, and an expert network group; Using the face parsing module to perform multiple upsampling on the original features, and performing convolution processing on the results of the multiple upsampling to obtain a face parsing feature map, wherein the face parsing feature map includes semantic information of key parts of the face; Performing multi-layer convolution feature mapping on the face parsing feature map using the embedded network of the hybrid expert network to obtain an initial local feature representation of the face with key facial information; Flattening the initial local feature representation of the face using the flattening layer of the embedding network to obtain an initial local feature representation of the face with a preset dimension; Constructing a routing network and an expert network group in the hybrid expert network based on a predefined activation function; In each training round, dynamically selecting an expert network from the expert network group using the routing network, and determining the importance of the selected expert network in the expert network group by calculating the probability of the selected expert network; In each training round, load balancing is performed on the joint training process of the routing network and the expert network group using an auxiliary loss function based on a discrete coefficient based on the calculated importance; Converting the trained routing network representing the initial local features of the face of the preset dimension into a probability distribution of the hybrid expert network; Based on a preset threshold, dynamically selecting an expert network from the trained expert network group using the trained routing network to obtain an expert network participating in routing, and calculating a weight corresponding to the probability distribution; Based on the weights corresponding to the probability distribution, using the expert network involved in routing to extract the original features of the face of the preset dimension; Performing vector operations on the local features of the face obtained through feature extraction using the expert network and the weights corresponding to the probability distribution to obtain the local features; The global features and the local features are adaptively fused, and the feature fusion results are processed using a trained age prediction network to obtain the predicted age of the target object.

2. The method according to claim 1, characterized in that Based on the authorization of the target object, the facial image of the target object is preprocessed to obtain a region of interest of the facial image including: Based on the authorization of the target object, performing face detection on the target object to obtain a face image of the target object; Based on the authorization of the target object, the facial images are aligned, and the aligned facial images are cropped to remove non-face areas to obtain the region of interest.

3. The method according to claim 2, characterized in that Based on the authorization of the target object, the facial image is aligned, and the aligned facial image is cropped to remove non-face areas, so that the region of interest includes: Based on a predefined format, facial images of different angles, postures, and sizes are processed in a unified format to obtain standardized facial images; Positioning key facial feature points on the standardized facial image to obtain key point information of the standardized facial image; Based on the key point information, calibrating the standardized facial image through affine transformation to obtain the aligned facial image; The aligned face image is cropped to delete the non-face area and the background area in the aligned face image, and the deleted face image is compressed to obtain a region of interest.

4. The method according to claim 1, wherein Extracting original features of the face image from the region of interest using the trained original feature extraction network includes: Constructing an original feature extraction network based on Swin Transformer and training the original feature extraction network to obtain a trained original feature extraction network, wherein the trained original feature extraction network includes a multi-scale feature extractor and a multi-level channel attention module; Using the multi-scale feature extractor to extract features from the region of interest to obtain a plurality of feature maps of different scales, and performing a convolution dimensionality reduction operation on the plurality of feature maps of different scales to obtain a plurality of intermediate feature maps of different scales; The multi-level channel attention module is used to combine the multiple intermediate feature maps of different scales in the scale dimension and the channel dimension to obtain the original features of the face image.

5. The method according to claim 1, wherein The trained global feature extraction network includes a maximum pooling layer, a ReLU activation layer, and multiple fully connected layers.

6. The method according to claim 1, characterized in that Dynamically selecting an expert network from the trained expert network group using the trained routing network to obtain an expert network participating in routing, and calculating a weight corresponding to the probability distribution includes: Comparing each probability in the probability distribution with a predefined threshold, and when the probability is greater than or equal to the predefined threshold, setting the weight of the expert network participating in routing to 1 using the predefined activation function; Comparing each probability in the probability distribution with a predefined threshold, and when all the probabilities are less than the predefined threshold, calculating the weight of the expert network participating in routing based on the probability distribution; The trained routing network is used to dynamically select the expert network participating in routing, thereby making the hybrid expert network sparse.

7. The method according to claim 1, characterized in that Adaptively fusing the global features and the local features, and processing the feature fusion results using the trained age prediction network to obtain the predicted age of the target object includes: Performing weighted operations on the global feature vector and the local feature vector respectively, and performing vector splicing on the operation results to obtain the feature fusion result; Training the age prediction network using a hybrid loss function based on Gaussian loss and Euclidean loss to obtain a trained age prediction network; The trained age prediction network is used to process the feature fusion result to obtain the predicted age of the target object.

Images