Face age estimation method based on hybrid expert network

Abstract

The invention provides a face age estimation method based on a hybrid expert network, which can be applied to the technical field of computer vision. The method comprises the steps of performing face cutting processing on to-be-recognized face data to generate a pre-processed image, and inputting the pre-processed image into a feature extraction network to obtain original features; respectively inputting the original features into a feature extraction network based on a full connection layer to obtain global features and a feature extraction network based on a hybrid expert network to obtain local features; self-adaptive feature fusion is carried out on the extracted global features and local features; and finally, inputting the fusion features into an output layer to obtain a prediction result of the face age. According to the face age estimation method based on the hybrid expert network provided by the invention, global and local features of the face age can be adaptively acquired, and the accuracy of face age estimation is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically to a face age estimation method based on a hybrid expert network. Background Art

[0002] Face age estimation has been a hot topic in the field of computer vision in recent years. Age is one of the most important biological characteristics of the human body. Face age estimation has wide applications and strong practical value. It estimates the approximate age or age range of an individual based on a face image. It can be applied to security monitoring, beauty care, missing persons and other fields. Although with the development of deep learning technology, the application of automated face age estimation has been realized, but due to the extremely complex facial feature information and the large individual differences, age features are difficult to accurately capture and identify, so image-based face age estimation is still a challenging problem. Summary of the invention

[0003] In view of the above problems, the present invention provides a method for estimating face age based on a hybrid expert network.

[0004] According to a first aspect of the present invention, a method for estimating face age based on a hybrid expert network is provided, comprising:

[0005] Based on the authorization of the target object, the face image of the target object is preprocessed to obtain the region of interest of the face image;

[0006] The original features of the face image are extracted from the region of interest using the trained original feature extraction network;

[0007] The original features are processed using the trained global feature extraction network to obtain the global features of the face image;

[0008] The original features are processed using the trained local feature extraction network based on the hybrid expert network to obtain the local features of the face image;

[0009] The global features and local features are adaptively fused, and the feature fusion results are processed using the trained age prediction network to obtain the predicted age of the target object.

[0010] According to an embodiment of the present invention, the facial image of the target object is preprocessed based on the authorization of the target object to obtain the region of interest of the facial image, which includes:

[0011] Based on the authorization of the target object, face detection is performed on the target object to obtain a face image of the target object;

[0012] Based on the authorization of the target object, the face image is aligned, and the non-face area is cropped and deleted from the aligned face image to obtain the region of interest.

[0013] According to an embodiment of the present invention, the facial image is aligned based on the authorization of the target object, and the non-face area is cropped and deleted from the aligned facial image to obtain the region of interest including:

[0014] Based on the predefined format, facial images of different angles, postures and sizes are processed in a unified format to obtain standardized facial images;

[0015] Positioning key facial feature points of the standardized facial image to obtain key point information of the standardized facial image;

[0016] Based on the key point information, the standardized face image is calibrated through affine transformation to obtain an aligned face image;

[0017] The aligned face image is cropped, the non-face area and the background area in the aligned face image are deleted, and the deleted face image is compressed to obtain the region of interest.

[0018] According to an embodiment of the present invention, the above-mentioned method of extracting original features of a face image from a region of interest using the trained original feature extraction network includes:

[0019] An original feature extraction network is constructed based on Swin Transformer and trained to obtain a trained original feature extraction network, wherein the trained original feature extraction network includes a multi-scale feature extractor and a multi-level channel attention module;

[0020] A multi-scale feature extractor is used to extract features from the region of interest to obtain feature maps of multiple scales, and convolution dimension reduction operations are performed on the feature maps of multiple scales to obtain intermediate feature maps of multiple scales.

[0021] A multi-level channel attention module is used to combine multiple intermediate feature maps of different scales in the scale dimension and channel dimension to obtain the original features of the face image.

[0022] According to an embodiment of the present invention, the trained global feature extraction network includes a maximum pooling layer, a ReLU activation layer and multiple fully connected layers.

[0023] According to an embodiment of the present invention, the local features of the face image obtained by processing the original features using the trained local feature extraction network based on the hybrid expert network include:

[0024] Constructing a local feature extraction network based on a hybrid expert network and training it to obtain a trained local feature extraction network based on a hybrid expert network, wherein the trained local feature extraction network based on a hybrid expert network includes a face parsing module and a hybrid expert network;

[0025] The face parsing module is used to upsample the original features multiple times, and the results of the multiple upsampling are convolved to obtain a face parsing feature map;

[0026] The hybrid expert network is used to perform feature mapping, feature flattening, routing selection and expert network feature extraction on the face parsing feature map to obtain local features.

[0027] According to an embodiment of the present invention, the hybrid expert network is used to perform feature mapping, feature flattening, routing selection and expert network feature extraction on the face analysis feature map, and the local features obtained include:

[0028] The embedded network of the hybrid expert network is used to perform multi-layer convolution feature mapping on the face parsing feature map to obtain the initial local feature representation of the face with key face information;

[0029] The initial local feature representation of the face is flattened by using the flattening layer of the embedding network to obtain the initial local feature representation of the face with a preset dimension;

[0030] Based on a predefined activation function, a routing network and an expert network group in the hybrid expert network are constructed, and the routing network and the expert network group are trained to obtain a trained routing network and a trained expert network group;

[0031] The initial local feature representation of the face of the preset dimension is converted into a probability distribution of the hybrid expert network through the trained routing network;

[0032] Using the trained routing network to dynamically select an expert network from the trained expert network group, obtain the expert network participating in routing, and calculate the weight corresponding to the probability distribution;

[0033] Based on the weights corresponding to the probability distribution, using the expert network involved in routing to extract the original features of the face of the preset dimension;

[0034] A vector operation is performed on the local features of the face obtained through the expert network feature extraction and the weights corresponding to the probability distribution to obtain the local features.

[0035] According to an embodiment of the present invention, the training of the routing network and the expert network group to obtain the trained routing network and the trained expert network group includes:

[0036] In each training round, dynamically selecting an expert network from the expert network group using the routing network, and determining the importance of the selected expert network in the expert network group by calculating the probability of the selected expert network;

[0037] In each training round, based on the calculated importance, an auxiliary loss function based on discrete coefficients is used to load balance the joint training process of the routing network and the expert network group.

[0038] According to an embodiment of the present invention, the above-mentioned method of dynamically selecting an expert network in a trained expert network group using a trained routing network to obtain an expert network participating in routing, and calculating a weight corresponding to a probability distribution includes:

[0039] Compare each probability in the probability distribution with a predefined threshold, and when the probability is greater than or equal to the predefined threshold, set the weight of the expert network participating in routing to 1 using a predefined activation function;

[0040] Compare each probability in the probability distribution with a predefined threshold, and when all probabilities are less than the predefined threshold, calculate the weight of the expert network participating in the routing based on the probability distribution;

[0041] The trained routing network is used to dynamically select the expert network involved in routing, thereby making the hybrid expert network sparse.

[0042] According to an embodiment of the present invention, the above-mentioned adaptive feature fusion of global features and local features and processing of the feature fusion result using the trained age prediction network to obtain the predicted age of the target object include:

[0043] Perform weighted operations on the global feature vector and the local feature vector respectively, and concatenate the operation results to obtain the feature fusion result;

[0044] The age prediction network is trained using a mixed loss function based on Gaussian loss and Euclidean loss to obtain a trained age prediction network;

[0045] The trained age prediction network is used to process the feature fusion results to obtain the predicted age of the target object.

[0046] The above-mentioned face age estimation method based on hybrid expert network provided by the present invention can automatically extract feature maps of various key parts of the face of the target person, and by extracting local features of the target person, it can focus on the key areas of the face; local features are helpful to identify the face age information in the image, and increasing local feature processing can improve the accuracy and interpretability of face age estimation; at the same time, the use of expert hybrid network can adaptively process face analysis feature maps, extract effective local features, and improve performance; in addition, the present invention can automatically adjust the weights of global and local features through adaptive feature fusion operations, thereby increasing the accuracy of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0048] Figure 1 is a flow chart of a method for estimating face age based on a hybrid expert network according to an embodiment of the present invention;

[0049] Figure 2 is a structural schematic diagram of a face age estimation solution based on a hybrid expert network according to an embodiment of the present invention;

[0050] Figure 3 A structural block diagram of a face age estimation device based on a hybrid expert network according to an embodiment of the present invention is schematically shown;

[0051] Figure 4 A block diagram of an electronic device suitable for implementing a method for estimating face age based on a hybrid expert network according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0052] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0053] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0054] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0055] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0056] In order to improve the accuracy of face age estimation and solve the technical problems existing in the existing technical solutions, the present invention provides a method for realizing face age estimation based on a hybrid expert network, and uses the hybrid expert network to adaptively extract representative local features to enhance the accuracy and interpretability of age estimation. Since there may be interference problems such as multiple faces and background information in an image under unconstrained conditions, an image preprocessing module is used to crop faces, retain face information, and remove irrelevant information.

[0057] The purpose of this invention is to solve three key problems in face age estimation: (1) How to improve the accuracy and interpretability of face age estimation based on a hybrid expert network. (2) How to adaptively obtain targeted local face age features. (3) How to effectively integrate global features and local features to improve the accuracy of face age estimation.

[0058] First, the face image is preprocessed. Face detection, face alignment and face cropping are performed in the preprocessing stage to obtain the region of interest, remove redundant background information, and input the preprocessed image into the feature extraction network to obtain the original features. Secondly, the original features are input into the feature extraction network based on the fully connected layer to obtain the global features and the feature extraction network based on the hybrid expert network to obtain the local features. Finally, the extracted global features and local features are adaptively fused, and the final age prediction result is obtained through the output layer.

[0059] The above technical solution provided by the present invention is further described below through specific embodiments in conjunction with the accompanying drawings.

[0060] Figure 1 4 is a flowchart of a method for estimating face age based on a hybrid expert network according to an embodiment of the present invention.

[0061] like Figure 1As shown, the above-mentioned face age estimation method based on hybrid expert network includes operations S110 to S150.

[0062] In operation S110, based on the authorization of the target object, the face image of the target object is preprocessed to obtain a region of interest of the face image.

[0063] In an embodiment of the present invention, before acquiring the facial image of the target object, the consent or authorization of the target object may be obtained. For example, before operation S110, a request for acquiring user information may be issued to the target object. When the target object agrees or authorizes that the facial image of the target object may be acquired, operation S110 is performed.

[0064] In an embodiment of the present invention, a corresponding operation entry can be provided for the target object, so that the target object can choose to agree or reject the automated decision result. That is, before the facial image of the target object is processed / decided, an instruction for agreeing or rejecting the processing / decision input by the target object through the corresponding operation entry can be obtained. If the target object agrees to the processing / decision, the facial image of the target object is processed / decided, that is, step S110 is executed. If the target object refuses to perform the processing / decision, the expert decision process is entered.

[0065] Operation S110, inputting the face image into a preprocessing module, performing face detection, alignment and cropping, removing irrelevant areas, retaining the face area of ​​interest, and then inputting the cropped image into a feature extraction network to obtain original features.

[0066] Operation S110, when the image quality is not high, such as the face is tilted, this will affect the accuracy of face age estimation. The present invention uses the InsightFace tool to perform face detection, alignment and cropping, automatically obtain more accurate face images, eliminate irrelevant information, reduce background information interference, and improve the accuracy of age estimation.

[0067] In operation S120, the original features of the face image are extracted from the region of interest using the trained original feature extraction network. The cropped face image is input into the feature extraction network based on Swin Transformer, specifically as follows: first, feature maps of four different scales are extracted through the shared backbone network Swin Transformer, and the original features are obtained by integrating the face features of different scales and channels through a multi-level channel attention module.

[0068] In operation S130, the original features are processed using the trained global feature extraction network to obtain global features of the face image.

[0069] For example: the original features are sent to the global feature extraction channel, where the global feature extraction channel consists of a maximum pooling layer, a ReLU activation layer and two fully connected layers.

[0070] In operation S140, the original features are processed using the trained local feature extraction network based on the hybrid expert network to obtain local features of the face image.

[0071] For example, the original features are sent to the local feature extraction channel, where the local feature extraction channel processing process is: a face parsing feature map is generated through a series of upsampling operations, and then the effective local face features are adaptively extracted through the hybrid expert network.

[0072] In operation S150, the global features and the local features are adaptively fused, and the feature fusion result is processed using the trained age prediction network to obtain the predicted age of the target object.

[0073] The global features and local features are adaptively fused; the fused features are used for age prediction through the output layer.

[0074] For example, the extracted global features and local features are adaptively fused to obtain new features. The sum of the global age weighted value and the local age weighted value is 100%, the global age weighted value is 10-90%, and the rest is the local age weighted value.

[0075] The above-mentioned face age estimation method based on hybrid expert network provided by the present invention can automatically extract feature maps of various key parts of the face of the target person, and by extracting local features of the target person, it can focus on the key areas of the face; local features are helpful to identify the face age information in the image, and increasing local feature processing can improve the accuracy and interpretability of face age estimation; at the same time, the use of expert hybrid network can adaptively process face analysis feature maps, extract effective local features, and improve performance; in addition, the present invention can automatically adjust the weights of global and local features through adaptive feature fusion operations, thereby increasing the accuracy of the network.

[0076] According to an embodiment of the present invention, the above-mentioned preprocessing of the face image of the target object based on the authorization of the target object to obtain the region of interest of the face image includes: performing face detection on the target object based on the authorization of the target object to obtain the face image of the target object; aligning the face images based on the authorization of the target object, and obtaining the region of interest by cropping and deleting non-face areas of the aligned face images.

[0077] According to an embodiment of the present invention, the facial images are aligned based on the authorization of the target object, and the non-face areas are cropped and deleted from the aligned facial images to obtain the region of interest, including: based on a predefined format, facial images of different angles, different postures, and different sizes are processed to a unified specification to obtain a standardized facial image; key facial feature points of the standardized facial image are located to obtain key point information of the standardized facial image; based on the key point information, the standardized facial image is calibrated through an affine transformation to obtain an aligned facial image; the aligned facial image is cropped to delete the non-face area and background area in the aligned facial image, and the deleted facial image is compressed to obtain the region of interest.

[0078] In order to improve the accuracy of face age estimation, it is necessary to preprocess the image and extract original features. Image preprocessing includes three parts: face detection, face alignment and face cutting.

[0079] First, for a size of Grayscale image of First, based on face detection, the face to be analyzed is determined. If there are multiple faces in an image, the numbers are superimposed in sequence.

[0080] Secondly, the facial images of different angles, postures and sizes are uniformly normalized, and the key facial feature points are located based on MTCNN, including eyes, nose tip, mouth corners and eyebrows. The key point information is used to calibrate the face through affine transformation.

[0081] Finally, the calibrated face image is cut to remove the background and non-face areas, so that the network pays more attention to the foreground, and the image is scaled to , which can not only preserve facial details, but also reduce the computational burden and memory requirements during training.

[0082] According to an embodiment of the present invention, the above-mentioned extraction of original features of a face image from a region of interest using a trained original feature extraction network includes: constructing an original feature extraction network based on a Swin Transformer and training the original feature extraction network to obtain a trained original feature extraction network, wherein the trained original feature extraction network includes a multi-scale feature extractor and a multi-level channel attention module; using the multi-scale feature extractor to extract features of the region of interest to obtain feature maps of multiple scales, and performing convolution dimensionality reduction operations on the feature maps of multiple scales to obtain intermediate feature maps of multiple scales; using the multi-level channel attention module to combine the intermediate feature maps of multiple scales in the scale dimension and the channel dimension to obtain the original features of the face image.

[0083] The original feature extraction network is built based on the Swin Transformer network.

[0084] The cropped image I is input into the feature extractor based on Swin Transformer to extract feature maps of four different scales, namely : , , , And through a multi-level channel attention module, the facial features of different scales and channels are combined to obtain the original facial features. The specific operation is as follows: First, the feature map Unify the scale and use convolution operation to reduce the dimension, as shown in formulas (1) to (4):

[0085] (1),

[0086] (2),

[0087] (3),

[0088] (4).

[0089] The feature maps are fused and the channel attention module is used to obtain the original features, as shown in formula (5):

[0090] (5).

[0091] According to an embodiment of the present invention, the above-mentioned global feature extraction network based on the fully connected network that has been trained is used to process the original features, and the global feature processing includes: a maximum pooling layer, a ReLU activation function layer and multiple fully connected layers.

[0092] According to an embodiment of the present invention, the above-mentioned use of the trained local feature extraction network based on the hybrid expert network to process the original features to obtain the local features of the face image includes: constructing a local feature extraction network based on the hybrid expert network and training it to obtain a trained local feature extraction network based on the hybrid expert network, wherein the trained local feature extraction network based on the hybrid expert network includes a face parsing module and a hybrid expert network; using the face parsing module to upsample the original features multiple times, and performing convolution processing on the results of the multiple upsampling to obtain a face parsing feature map; using the hybrid expert network to perform feature mapping, feature flattening, routing selection and expert network feature extraction processing on the face parsing feature map to obtain local features.

[0093] According to an embodiment of the present invention, the above-mentioned use of a hybrid expert network to perform feature mapping, feature flattening, routing selection and expert network feature extraction processing on the face parsing feature map to obtain local features includes: using an embedded network of the hybrid expert network to perform multi-layer convolution feature mapping on the face parsing feature map to obtain an initial local feature representation of the face with key face information; using a flattening layer of the embedded network to flatten the initial local feature representation of the face to obtain an initial local feature representation of the face of a preset dimension; constructing a routing network and an expert network group in the hybrid expert network based on a predefined activation function and training the routing network and the expert network group to obtain A trained routing network and a trained expert network group; converting the initial local features of a human face of a preset dimension into a probability distribution of a hybrid expert network through the trained routing network; dynamically selecting an expert network in the trained expert network group using the trained routing network to obtain an expert network participating in routing, and calculating a weight corresponding to the probability distribution; based on the weight corresponding to the probability distribution, using the expert network participating in routing to extract features from the original features of the human face of the preset dimension; performing vector operations on the local features of the human face obtained through feature extraction of the expert network and the weights corresponding to the probability distribution to obtain the local features.

[0094] According to an embodiment of the present invention, the above-mentioned training of the routing network and the expert network group to obtain the trained routing network and the trained expert network group includes: in each training round, using the routing network to dynamically select an expert network from the expert network group, and determining the importance of the selected expert network in the expert network group by calculating the probability of the selected expert network; in each training round, based on the calculated importance, using an auxiliary loss function based on discrete coefficients to load balance the joint training process of the routing network and the expert network group.

[0095] According to an embodiment of the present invention, the above-mentioned method of dynamically selecting an expert network in a trained expert network group using a trained routing network to obtain an expert network participating in routing, and calculating a weight corresponding to a probability distribution includes: comparing each probability in the probability distribution with a predefined threshold, and when the probability is greater than or equal to the predefined threshold, setting the weight of the expert network participating in routing to 1 using a predefined activation function; comparing each probability in the probability distribution with a predefined threshold, and when the probabilities are all less than the predefined threshold, calculating the weight of the expert network participating in routing based on the probability distribution; and dynamically selecting an expert network participating in routing using a trained routing network to make the hybrid expert network sparse.

[0096] The following is a specific embodiment and combined with the attached Figure 2 The local feature extraction process involved in the present invention is further described in detail.

[0097] Figure 2 4 is a schematic diagram of the structure of a face age estimation solution based on a hybrid expert network according to an embodiment of the present invention.

[0098] It should be noted that Figure 2 The facial images involved are derived from open source datasets, which are authorized by relevant users and used for non-commercial purposes such as scientific research and technical verification with the permission of relevant users. The facial images of the target person used in the present invention have been authorized by the above-mentioned open source datasets, and are only used to illustrate the method provided by the present invention and to verify the advantages and effectiveness of the method provided by the present invention, and will not be used for commercial or other profit-making purposes.

[0099] like Figure 2 As shown, the original facial features obtained by the backbone network Input into the feature extractor based on hybrid expert network to obtain local features. The feature extractor based on hybrid expert network consists of two parts: face parsing module and hybrid expert network. The face parsing module can refine the semantic information of the face and decompose the feature maps of each key part of the face, which can help to extract targeted local features in the subsequent process, thereby improving the accuracy of face estimation. After two upsampling layers and one convolution layer, the face parsing feature map is obtained. .

[0100] Due to the large differences in facial features, in order to adaptively process targeted facial analysis feature maps and improve the accuracy and interpretability of the network, the obtained facial analysis feature maps Send it to the hybrid expert module to obtain targeted local features The hybrid expert module consists of an embedding network and a learnable routing network. And a group of expert networks built on Resnet18 .

[0101] like Figure 2 As shown in Figure 1, the local face feature map is first input into the embedding network to map the features into potential local face feature representations to simplify processing and retain key information. The embedding network consists of The convolutional layer And Flatten layer, first size is The local feature map of the face is processed by the convolution layer. , When , as shown in formula (6):

[0102] (6).

[0103] Then, the multidimensional data output by the convolutional layer is flattened. Flatten into one-dimensional linear data X for subsequent processing and better expression of local features, as shown in formula (7):

[0104] (7).

[0105] After obtaining the output of the embedding layer, the output is sent to the routing network to dynamically select the number of expert networks to participate in the training. First, the potential local face feature representation generated by the embedding network is converted into the probability distribution of the expert network, which is used to indicate which local face feature representation needs to be sent to which expert network. The calculation method is shown in formula (8):

[0106] (8),

[0107] in are learnable image weights, is the learnable noise weight.

[0108] Set a threshold , when the probability Existence In the case of , the task expert network selection is considered to be more professional. At this time, the Keepmax function is used to take the first route as the expert who solves the problem and set its weight to 1, so that the network is sparse. On the contrary, it is considered that all experts with non-zero probabilities are desirable. It is necessary to set the expert network weight according to the corresponding probability distribution to comprehensively calculate the expert output, so as to obtain targeted local features. Through the above processing mechanism, the number of experts participating in the training can be adaptively selected, and local features can be extracted more effectively. The routing network is shown in formulas (9) and (10):

[0109] (9),

[0110] (10).

[0111] Then the face analysis feature map , and the routing selection Router is input into the expert network to obtain targeted local features of the face. The expert network part is built based on the pre-trained ResNet18 model. Assuming that the expert group has a total of Thus, local features are obtained , as shown in formula (11):

[0112] (11),

[0113] In order to improve load balancing and allow all experts to receive more adequate training, in each round of training, each expert is assigned to the expert by calculating The probability ratio To indicate expert The importance of the expert group is shown in formula (12):

[0114] (12).

[0115] To prevent over-reliance on some expert networks during training while other networks are idle, an auxiliary loss is added using the discrete coefficient (CV) , by minimizing To achieve better load balancing, as shown in formula (13):

[0116] (13),

[0117] Among them, CV is defined as the ratio of standard deviation to mean, which is a normalized measure of the discreteness of probability distribution. are manually tuned auxiliary loss weights.

[0118] According to an embodiment of the present invention, the above-mentioned adaptive feature fusion of global features and local features, and processing the feature fusion results using the trained age prediction network to obtain the predicted age of the target object includes: performing weighted operations on the global feature vector and the local feature vector respectively, and vector splicing the operation results to obtain the feature fusion result; training the age prediction network using a mixed loss function based on Gaussian loss and Euclidean loss to obtain the trained age prediction network; using the trained age prediction network to process the feature fusion results to obtain the predicted age of the target object.

[0119] In a specific embodiment, , The feature fusion is performed adaptively. Assuming that the feature channels of the global and local inputs are Hewei , then the output channel of the fusion feature is shown in formula (14):

[0120] (14),

[0121] in, is an adaptive learning parameter, representing the weight of global features and local features.

[0122] In a specific embodiment, face age estimation is classified as a regression problem. The input is sent to the output layer of a fully connected neural network to obtain the predicted age value. The training goal of the face age estimation method based on the hybrid expert network is to make the predicted age value as consistent as possible with the image age label value. The Euclidean loss can be used to effectively evaluate this indicator. In addition, when the predicted value is far from the label value, the use of Gaussian loss can slow down the training process. Therefore, a weighted combination of Gaussian loss and Euclidean loss is used as the loss function for face age estimation, as shown in formula (15):

[0123] (15),

[0124] in, is the predicted value, is the true value, is the standard deviation of the annotation age.

[0125] Combining the face estimation loss and auxiliary loss, we get the overall loss function of the model, as shown in formula (16):

[0126] (16).

[0127] Based on the above-mentioned face age estimation method based on hybrid expert network, the present invention also provides a face age estimation device based on hybrid expert network. Figure 3 The device is described in detail.

[0128] Figure 3 The structure block diagram of the face age estimation device based on the hybrid expert network according to an embodiment of the present invention is schematically shown.

[0129] like Figure 3 As shown, the face age estimation device 300 based on the hybrid expert network of this embodiment includes a preprocessing module 310, an original feature extraction module 320, a global feature acquisition module 330, a local feature acquisition module 340 and a fusion prediction module 350.

[0130] The preprocessing module 310 is used to preprocess the face image of the target object based on the authorization of the target object to obtain the region of interest of the face image. In one embodiment, the preprocessing module 310 can be used to perform the operation S110 described above, which will not be repeated here.

[0131] The original feature extraction module 320 is used to extract the original features of the face image from the region of interest using the trained original feature extraction network. In one embodiment, the original feature extraction module 320 can be used to perform the operation S120 described above, which will not be described in detail here.

[0132] The global feature acquisition module 330 is used to process the original features using the trained global feature extraction network to obtain the global features of the face image. In one embodiment, the global feature acquisition module 330 can be used to perform the operation S130 described above, which will not be described in detail here.

[0133] The local feature acquisition module 340 is used to process the original features using the trained local feature extraction network based on the hybrid expert network to obtain the local features of the face image. In one embodiment, the local feature acquisition module 340 can be used to perform the operation S140 described above, which will not be repeated here.

[0134] The fusion prediction module 350 is used to adaptively fuse the global features and the local features, and use the trained age prediction network to process the feature fusion result to obtain the predicted age of the target object. In one embodiment, the fusion prediction module 350 can be used to perform the operation S150 described above, which will not be repeated here.

[0135] Figure 4 A block diagram of an electronic device suitable for implementing a method for estimating face age based on a hybrid expert network according to an embodiment of the present invention is schematically shown.

[0136] like Figure 4 As shown, the electronic device 400 according to an embodiment of the present invention includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 to a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include an onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0137] In RAM 403, various programs and data required for the operation of electronic device 400 are stored. Processor 401, ROM 402 and RAM 403 are connected to each other via bus 404. Processor 401 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 402 and / or RAM 403. It should be noted that the program can also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0138] According to an embodiment of the present invention, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to the bus 404. The electronic device 400 may further include one or more of the following components connected to the input / output (I / O) interface 405: an input portion 406 including a keyboard, a mouse, etc.; an output portion 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 408 including a hard disk, etc.; and a communication portion 409 including a network interface card such as a LAN card, a modem, etc. The communication portion 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage portion 408 as needed.

[0139] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0140] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 402 and / or RAM 403 described above and / or one or more memories other than ROM 402 and RAM 403.

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0143] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A face age estimation method based on a hybrid expert network, characterized in that: The method comprises: Based on the authorization of the target object, preprocessing the face image of the target object to obtain a region of interest of the face image; Extracting original features of the face image from the region of interest using the trained original feature extraction network; Processing the original features using the trained global feature extraction network to obtain the global features of the face image; Processing the original features using a trained local feature extraction network based on a hybrid expert network to obtain local features of the face image; The global features and the local features are adaptively fused, and the feature fusion results are processed using a trained age prediction network to obtain the predicted age of the target object.

2. The method according to claim 1, characterized in that Based on the authorization of the target object, the face image of the target object is preprocessed to obtain the region of interest of the face image including: Based on the authorization of the target object, performing face detection on the target object to obtain a face image of the target object; Based on the authorization of the target object, the face image is aligned, and the aligned face image is cropped and non-face areas are deleted to obtain the region of interest.

3. The method according to claim 2, characterized in that Based on the authorization of the target object, the face image is aligned, and the aligned face image is cropped to remove the non-face area, so that the region of interest includes: Based on the predefined format, facial images of different angles, postures and sizes are processed in a unified format to obtain standardized facial images; Positioning key facial feature points of the standardized facial image to obtain key point information of the standardized facial image; Based on the key point information, calibrating the standardized face image through affine transformation to obtain the aligned face image; The aligned face image is cropped, the non-face area and the background area in the aligned face image are deleted, and the deleted face image is compressed to obtain the region of interest.

4. The method according to claim 1, characterized in that: Extracting the original features of the face image from the region of interest using the trained original feature extraction network includes: Building an original feature extraction network based on Swin Transformer and training the original feature extraction network to obtain a trained original feature extraction network, wherein the trained original feature extraction network includes a multi-scale feature extractor and a multi-level channel attention module; Using the multi-scale feature extractor to extract features from the region of interest to obtain a plurality of feature maps of different scales, and performing convolution dimensionality reduction operations on the plurality of feature maps of different scales to obtain a plurality of intermediate feature maps of different scales; The multi-level channel attention module is used to combine the multiple intermediate feature maps of different scales in the scale dimension and the channel dimension to obtain the original features of the face image.

5. The method according to claim 1, characterized in that The trained global feature extraction network includes a maximum pooling layer, a ReLU activation layer and multiple fully connected layers.

6. The method according to claim 1, characterized in that The local features of the face image are obtained by processing the original features using the trained local feature extraction network based on the hybrid expert network, including: Constructing a local feature extraction network based on a hybrid expert network and training it to obtain a trained local feature extraction network based on a hybrid expert network, wherein the trained local feature extraction network based on a hybrid expert network includes a face parsing module and a hybrid expert network; The face analysis module is used to perform multiple upsampling on the original features, and the results of the multiple upsampling are subjected to convolution processing to obtain a face analysis feature map; The hybrid expert network is used to perform feature mapping, feature flattening, routing selection and expert network feature extraction processing on the face analysis feature map to obtain the local features.

7. The method according to claim 6, characterized in that The hybrid expert network is used to perform feature mapping, feature flattening, routing selection and expert network feature extraction processing on the face analysis feature map, and the local features obtained include: Performing multi-layer convolution feature mapping on the face parsing feature map using the embedding network of the hybrid expert network to obtain an initial local feature representation of the face with key face information; Flattening the initial local feature representation of the face using the flattening layer of the embedding network to obtain an initial local feature representation of the face with a preset dimension; Constructing a routing network and an expert network group in the hybrid expert network based on a predefined activation function and training the routing network and the expert network group to obtain a trained routing network and a trained expert network group; The initial local feature representation of the face of the preset dimension is converted into the probability distribution of the hybrid expert network through the trained routing network; Dynamically selecting an expert network from the trained expert network group using the trained routing network to obtain an expert network participating in routing, and calculating a weight corresponding to the probability distribution; Based on the weights corresponding to the probability distribution, using the expert network involved in routing to extract the original features of the face of the preset dimension; A vector operation is performed on the local features of the face obtained through the expert network feature extraction and the weights corresponding to the probability distribution to obtain the local features.

8. The method according to claim 7, characterized in that The routing network and the expert network group are trained to obtain a trained routing network and a trained expert network group, including: In each training round, dynamically selecting an expert network from the expert network group using the routing network, and determining the importance of the selected expert network in the expert network group by calculating the probability of the selected expert network; In each of the training rounds, based on the calculated importance, an auxiliary loss function based on discrete coefficients is used to load balance the joint training process of the routing network and the expert network group.

9. The method according to claim 7, characterized in that: Dynamically selecting an expert network in the trained expert network group using the trained routing network to obtain an expert network participating in routing, and calculating a weight corresponding to the probability distribution includes: Comparing each probability in the probability distribution with a predefined threshold, and when the probability is greater than or equal to the predefined threshold, setting the weight of the expert network participating in routing to 1 using the predefined activation function; Compare each probability in the probability distribution with a predefined threshold, and when the probabilities are all less than the predefined threshold, calculate the weight of the expert network participating in routing based on the probability distribution; The trained routing network is used to dynamically select the expert network participating in routing, thereby making the hybrid expert network sparse.

10. The method according to claim 1, characterized in that Adaptively fusing the global features and the local features, and processing the feature fusion results using the trained age prediction network to obtain the predicted age of the target object includes: Performing weighted operations on the global feature vector and the local feature vector respectively, and performing vector splicing on the operation results to obtain the feature fusion result; Training the age prediction network using a mixed loss function based on Gaussian loss and Euclidean loss to obtain a trained age prediction network; The feature fusion result is processed using the trained age prediction network to obtain the predicted age of the target object.

Images