Age prediction method, device, computer equipment and readable storage medium

Abstract

The present application proposes an age prediction method, and the method includes: obtaining multiple face images of a target user to form an input image group; inputting the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map; inputting one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map; performing feature fusion on the first feature map and the second feature map to obtain a target feature map; and performing age prediction based on the target feature map to obtain the face age of the target user. This technical solution can improve the recognition accuracy of the face age recognition model.

Description

Technical Field

[0001] This application relates to the field of face image recognition, and particularly to an age prediction method, device, computer device, and readable storage medium. Background Art

[0002] Face images often contain a lot of face feature information. Among them, age, as an important feature information, has been widely used in the field of face recognition. Especially on the mobile APP side, face age prediction is actually a challenging task. As each face image ages, the aging of the face is actually a slow process. Different people at the same age stage have some similar features. However, in existing technologies, age is usually regarded as a single label information. In the model training process, based on statistical methods, whether it is based on regression or classification methods, a one-to-one correspondence between face images and ages is established, and the similar features between images of the same age group or the same age value are learned. However, the existing age prediction models learning the similar aging features of the same age will result in large fluctuations in the predicted age values when the same person shows different facial expressions, resulting in low accuracy and lack of certain stability in the age prediction results. Summary of the Invention

[0003] This application provides an age prediction method, device, computer device, and computer-readable storage medium to solve the technical problems of low accuracy and lack of certain stability in the age prediction results of existing age prediction technologies.

[0004] In a first aspect, an age prediction method is provided. The method includes:

[0005] Obtain multiple face images of a target user to form an input image group;

[0006] Input the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map;

[0007] Input one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map;

[0008] Perform feature fusion on the first feature map and the second feature map to obtain a target feature map;

[0009] Perform age prediction based on the target feature map to obtain the face age of the target user.

[0010] In a second aspect, an age prediction device is provided. The device includes:

[0011] An acquisition module, configured to acquire multiple face images of a target user to form an input image group;

[0012] A first feature extraction module, configured to input the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map;

[0013] A second feature extraction module, configured to input one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map;

[0014] A feature fusion module, configured to perform feature fusion on the first feature map and the second feature map to obtain a target feature map;

[0015] An age prediction module, configured to perform age prediction based on the target feature map to obtain the face age of the target user.

[0016] In a third aspect, there is provided a computer device, including a memory and a processor. When the computer program stored in the memory is executed by the processor, the processor is caused to perform the following steps:

[0017] Acquire multiple face images of a target user to form an input image group;

[0018] Input the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map;

[0019] Input one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map;

[0020] Perform feature fusion on the first feature map and the second feature map to obtain a target feature map;

[0021] Perform age prediction based on the target feature map to obtain the face age of the target user.

[0022] In a fourth aspect, there is provided a computer-readable storage medium, storing a computer program. When the computer program is executed by a processor, the processor is caused to perform the following steps:

[0023] Acquire multiple face images of a target user to form an input image group;

[0024] Input the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map;

[0025] Input one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map;

[0026] Perform feature fusion on the first feature map and the second feature map to obtain a target feature map;

[0027] Based on the target feature map, perform age prediction to obtain the face age of the target user.

[0028] The present application can achieve the following beneficial effects: By extracting spatial features through the first feature extraction model, the spatial features of face images under different expressions can be effectively extracted; by extracting age features through the second feature extraction model, the age features of face images can be effectively extracted; by performing feature fusion on the first feature map and the second feature map, the extracted age features can be fused with the spatial features with spatial variation characteristics, and the spatial information and age information in the face image can be effectively retained; by performing age prediction on the target feature map obtained by fusion, the problem that the age values measured for the same person vary greatly due to facial expression changes can be solved, and the problem of lack of certain stability in model prediction can be solved. Description of the Drawings

[0029] Figure 1 It is a schematic structural diagram of a face age recognition model provided by an embodiment of the present application;

[0030] Figure 2 It is a schematic flowchart of an age prediction method provided by an embodiment of the present application;

[0031] Figure 3 It is a schematic flowchart of an age prediction method provided by an embodiment of the present application;

[0032] Figure 4 It is a schematic structural diagram of a three-dimensional feature extraction model provided by an embodiment of the present application;

[0033] Figure 5 It is a schematic structural diagram of a two-dimensional feature extraction model provided by an embodiment of the present application;

[0034] Figure 6 It is a schematic flowchart of an age prediction method provided by an embodiment of the present application;

[0035] Figure 7 It is a schematic structural diagram of an age prediction device provided by an embodiment of the present application;

[0036] Figure 8 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Detailed Embodiments

[0037] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0038] The technical solution of the present application is applicable to various scenarios of face recognition. Specifically, the technical solution of the present application is applicable to predicting the corresponding face age by recognizing a face image in a face recognition scenario. In practical applications, it can be to recognize the face image of this scenario through a face age recognition model with an age prediction function, so as to determine the age of the face corresponding to the face image.

[0039] To facilitate the understanding of the technical solution of the present application, the face age recognition model in the present application will be introduced first. As Figure 1 shown, the age recognition model in the present application includes two channels, namely a three-dimensional feature extraction model 10 and a two-dimensional feature extraction model 20; it also includes a feature fusion model 30 and a prediction model 40. Among them, the three-dimensional feature extraction model 10 is used to perform spatial feature extraction on the input image group input into the three-dimensional feature extraction model 10 to obtain a first feature map; the two-dimensional feature extraction model 20 is used to perform age feature extraction on any one face image of a person in the input image group input into the two-dimensional feature extraction model 20 to obtain a second feature map; the feature fusion model 30 is used to perform feature fusion on the first feature map and the second feature map to obtain a target feature map with both spatial and age features; the prediction model 40 performs age prediction on the target feature map through a softmax classification layer to obtain the face age of the user.

[0040] In one embodiment, as Figure 2 shown, the present application proposes an age prediction method, and the method includes:

[0041] Step 201, obtain multiple face images of the target user to form an input image group.

[0042] Among them, the target user refers to the object of age prediction. The face image refers to the user's facial image input as the input of the age prediction model. Specifically, it can be multiple photos taken by the user, or multiple face images intercepted from the video taken by the user. It can be understood that the multiple face images can also be multiple face images of the target user obtained from a pre-established image database.

[0043] Among them, the input image group is a set of face images of a target user with different expressions.

[0044] In a specific embodiment, as Figure 2 shown, after obtaining multiple face images of the target user to form an input image group, it further includes:

[0045] Step 2011, perform coordinate affine transformation on the multiple face images respectively to adjust the faces in the multiple face images to the frontal face state.

[0046] Specifically, taking one face image out of multiple face images as an example for illustration. It can be understood that each face image in the input image group is adjusted according to this illustration. Specifically, first determine the facial feature points of the face image; based on the facial feature points in the face image, determine the left-eye center coordinates, right-eye center coordinates, and nose center coordinates respectively representing the center positions of the left eye, right eye, and nose; calculate the angle θ of the left-right rotation of the face in the face image based on the left-eye center coordinates and the right-eye center coordinates; then, according to the rotation transformation matrix, adjust the face image centered on the nose center coordinates to adjust the face image to a frontal face state.

[0047] Among them, the calculation formula for the angle θ of the left-right rotation of the face is as follows:

[0048]

[0049] Among them, (x1, y1) represents the left-eye center coordinates, and (x2, y2) represents the right-eye center coordinate values.

[0050] Among them, the rotation transformation matrix is as follows:

[0051]

[0052] Among them, (x, y) represents the coordinate values of the pixel points in the face image, and (x ′ , y ′ ) represents the coordinate values of the pixel points after the coordinate affine transformation.

[0053] Step 2012: Perform face valid region cropping on the multiple face images after the coordinate affine transformation, and perform normalization processing on the cropped face valid regions.

[0054] Among them, in the face image after the coordinate affine transformation, the face valid region can be cropped from the face image after the coordinate affine transformation with the center at the nose center position and the length being the maximum distance between the key points in the face image after the affine transformation, and then the cropped face valid region is adjusted to the size required by the face age recognition model to obtain the normalized face image. Exemplarily, the cropped face image valid region can be adjusted to 256*256.

[0055] In a specific embodiment, the performing face valid region cropping on the multiple face images after the coordinate affine transformation includes: determining multiple facial feature points of the target face image; the target face image is any one of the multiple face images after the coordinate affine transformation; determining the feature point distances between the multiple facial feature points pairwise; and performing face valid region cropping on the target face image according to the feature point distances.

[0056] Among them, multiple facial feature points of the target face image can be determined through a facial key point algorithm. Specifically, the facial key point algorithm can be a deep convolutional neural network algorithm (DCNN, Deep Convolutional Network).

[0057] Among them, the coordinates of multiple facial feature points are obtained, the feature point distances z between every two of the multiple facial feature points are calculated, and then the effective face area of the target face image can be intercepted according to the feature point distance z. It can be understood that in order to more accurately and comprehensively determine the effective face area, the maximum feature point distance between every two of the multiple facial feature points can be determined first, and then the effective face area can be determined according to the maximum feature point distance. It can be understood that in order to speed up the calculation speed, the maximum feature point distance between every two of the multiple facial feature points can be determined by determining the feature point distance between the facial feature points in the diagonal direction.

[0058] Among them, the formula for determining the feature point distance z between every two of the multiple facial feature points is:

[0059]

[0060] Among them, (x3, y3) and (x4, y4) represent the coordinates of any two different facial feature points.

[0061] In this embodiment, by transforming the face image into a face image in a frontal face state and intercepting the face area, the face age recognition model can extract more accurate and effective facial features, which helps to improve the accuracy of face age prediction.

[0062] Step 202, input the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map.

[0063] Among them, the first feature extraction model can be the three-dimensional feature extraction model 10 in the age recognition model as Figure 1 shown, specifically, it can be any three-dimensional convolutional model capable of realizing spatial feature extraction. When a person makes different expressions, there will be different spatial feature changes in the face. The three-dimensional convolutional model can extract the face space of the face image with different expressions, etc. By inputting the input image group into the three-dimensional feature extraction model, the spatial features with spatial change characteristics in the face image can be extracted to obtain a first feature map.

[0064] Among them, the three-dimensional feature extraction model can be composed of multiple layers of three-dimensional feature extraction networks. Among them, each layer of the three-dimensional feature extraction network can include multiple convolutional kernels. The number and size of the convolutional kernels included in different three-dimensional feature extraction networks are different, and different numbers of convolutional kernels can be used to extract different numbers of facial feature maps.

[0065] Among them, each layer of the three-dimensional feature extraction network also includes a three-dimensional max pooling layer, which is used to reduce the dimension of the face feature map processed by the convolution kernel, so as to improve the calculation speed and at the same time improve the robustness of the extracted features.

[0066] In a specific embodiment, the first feature extraction model includes N layers of three-dimensional feature extraction networks, and each layer of the three-dimensional feature extraction network includes a three-dimensional convolutional layer and a three-dimensional max pooling layer, where N is a positive integer greater than or equal to 1; the step of inputting the input image group into the pre-established first feature extraction model for spatial feature extraction to obtain a first feature map includes: inputting the (i - 1)-th spatial feature map into the three-dimensional convolutional layer in the i-th layer of the three-dimensional feature extraction network for spatial feature extraction to obtain the i-th initial spatial feature map; inputting the i-th initial spatial feature map into the three-dimensional max pooling layer in the i-th layer of the three-dimensional feature extraction network for dimension reduction processing to obtain the i-th spatial feature map; determining the N-th spatial feature map as the first feature map; where 1 ≤ i ≤ N; when i = 1, the (i - 1)-th spatial feature map is the input image group.

[0067] In this embodiment, as Figure 3 shown, the three-dimensional feature extraction model can be composed of 4 layers of three-dimensional feature extraction networks. Among them, the three-dimensional feature extraction network C1 includes a three-dimensional convolutional layer with 32 convolutional kernels and a three-dimensional max pooling layer of size 2*2; the three-dimensional convolutional layer with 32 convolutional kernels is used to perform spatial feature extraction on the face image input to the three-dimensional feature extraction model to obtain 32 face feature maps with a size of 256*256, and then the 32 face feature maps with a size of 256*256 are subjected to dimension reduction processing by the three-dimensional max pooling layer of size 2*2 to obtain 32 face feature maps with a size of 128*128.

[0068] In this embodiment, as Figure 3 shown, the three-dimensional feature extraction network C2 is connected to the three-dimensional feature extraction network C1. The three-dimensional feature extraction network C2 includes a three-dimensional convolutional layer with 64 convolutional kernels and a three-dimensional max pooling layer of size 2*2; the three-dimensional convolutional layer with 64 convolutional kernels is used to perform spatial feature extraction on the face feature map output by the three-dimensional feature extraction network C1 to obtain 64 face feature maps with a size of 128*128, and then the 64 face feature maps with a size of 128*128 are subjected to dimension reduction processing by the three-dimensional max pooling layer of size 2*2 to obtain 64 face feature maps with a size of 64*64.

[0069] In this embodiment, as Figure 3As shown, the three-dimensional feature extraction network C3 is connected to the three-dimensional feature extraction network C2. The three-dimensional feature extraction network C3 includes a three-dimensional convolutional layer with 128 convolutional kernels and a three-dimensional max pooling layer of size 2×2. The three-dimensional convolutional layer with 128 convolutional kernels is used to perform spatial feature extraction on the face feature map output by the three-dimensional feature extraction network C2 to obtain 128 face feature maps with a size of 64×64, and then the 128 face feature maps with a size of 64×64 are subjected to dimensionality reduction processing through the three-dimensional max pooling layer of size 2×2 to obtain 128 face feature maps with a size of 32×32.

[0070] In this embodiment, as Figure 3 shown, the three-dimensional feature extraction network C4 is connected to the three-dimensional feature extraction network C3. The three-dimensional feature extraction network C4 includes a three-dimensional convolutional layer with 256 convolutional kernels and a three-dimensional max pooling layer of size 2×2. The three-dimensional convolutional layer with 256 convolutional kernels is used to perform spatial feature extraction on the face feature map output by the three-dimensional feature extraction network C3 to obtain 256 face feature maps with a size of 32×32, and then the 256 face feature maps with a size of 32×32 are subjected to dimensionality reduction processing through the three-dimensional max pooling layer of size 2×2 to obtain 256 face feature maps with a size of 16×16.

[0071] In this embodiment, the three-dimensional feature extraction networks C1, C2, C3, and C4 all perform spatial feature extraction through 3D convolution operations. Among them, the formula for calculating the size of the feature map in 3D convolution operation is:

[0072]

[0073] where padding is the padding value, Stride is the padding step, w in is the size of the input image, W out is the size of the output feature map, and F is the number of convolutional kernels. In practical applications, for example, both the padding step and the padding value can be set to 1.

[0074] In this embodiment, from Figure 3 it can be seen that as the three-dimensional feature extraction network progresses, the size of the spatial feature map gradually becomes smaller and the number gradually becomes larger. As the size of the spatial feature map becomes smaller and the number increases, the spatial feature extraction becomes more refined, and the spatial features of each face image can be extracted in all directions, so as to extract various spatial features of a person's face when showing different expressions.

[0075] Step 203: Input one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map.

[0076] Among them, the second feature extraction model can be, for example, Figure 1 the two-dimensional feature extraction model 20 in the age recognition model shown, specifically, it can be any two-dimensional convolutional model capable of extracting age features. Input any face image in the input image group into the two-dimensional feature extraction model, and the age features of the face image can be extracted to obtain the second feature map.

[0077] Among them, the two-dimensional feature extraction model can be composed of multiple layers of two-dimensional feature extraction networks. Among them, each layer of the two-dimensional feature extraction network can include multiple convolutional kernels. The number and size of the convolutional kernels included in different two-dimensional feature extraction networks are different, and different numbers of convolutional kernels can be used to extract different numbers of face feature maps.

[0078] Among them, each layer of the two-dimensional feature extraction network also includes a two-dimensional max pooling layer, which is used to perform dimensionality reduction processing on the face feature map processed by the convolutional kernel, so as to improve the calculation speed and at the same time improve the robustness of the extracted features.

[0079] In a specific embodiment, the second feature extraction model includes N layers of two-dimensional feature extraction networks, and each layer of the two-dimensional feature extraction network includes a two-dimensional convolutional layer and a two-dimensional max pooling layer, where N is a positive integer greater than or equal to 1; the step of inputting one of the face images in the input image group into the pre-established second feature extraction model for age feature extraction to obtain the second feature map includes: inputting the (i - 1)-th age feature map into the two-dimensional convolutional layer in the i-th layer of the two-dimensional feature extraction network for age feature extraction to obtain the i-th initial age feature map; inputting the i-th initial age feature map into the two-dimensional max pooling layer in the i-th layer of the two-dimensional feature extraction network for dimensionality reduction processing to obtain the i-th age feature map; determining the N-th age feature map as the second feature map; where 1 ≤ i ≤ N; when i = 1, the (i - 1)-th age feature map is one of the face images in the input image group.

[0080] In this embodiment, as Figure 4 shown, the three-dimensional feature extraction model can be composed of 4 layers of two-dimensional feature extraction networks. Among them, the two-dimensional feature extraction network D1 includes a two-dimensional convolutional layer with 32 convolutional kernels and a two-dimensional max pooling layer with a size of 2 * 2; the two-dimensional convolutional layer with 32 convolutional kernels is used to perform age feature extraction on the face image input into the two-dimensional feature extraction model to obtain 32 face feature maps with a size of 256 * 256, and then the 32 face feature maps with a size of 256 * 256 are subjected to dimensionality reduction processing through the two-dimensional max pooling layer with a size of 2 * 2 to obtain 32 face feature maps with a size of 128 * 128.

[0081] In this embodiment, as Figure 3As shown in the figure, the two-dimensional feature extraction network D2 is connected to the two-dimensional feature extraction network D1. The two-dimensional feature extraction network D2 includes a two-dimensional convolutional layer with 64 convolutional kernels and a two-dimensional max pooling layer of size 2*2. The two-dimensional convolutional layer with 64 convolutional kernels is used to extract age features from the face feature map output by the two-dimensional feature extraction network D1 to obtain 64 face feature maps with a size of 128*128. Then, the 64 face feature maps with a size of 128*128 are downsampled by the two-dimensional max pooling layer of size 2*2 to obtain 64 face feature maps with a size of 64*64.

[0082] In this embodiment, as Figure 3 shown, the two-dimensional feature extraction network D3 is connected to the two-dimensional feature extraction network D2. The two-dimensional feature extraction network D3 includes a two-dimensional convolutional layer with 128 convolutional kernels and a two-dimensional max pooling layer of size 2*2. The two-dimensional convolutional layer with 128 convolutional kernels is used to extract age features from the face feature map output by the two-dimensional feature extraction network D2 to obtain 128 face feature maps with a size of 64*64. Then, the 128 face feature maps with a size of 64*64 are downsampled by the two-dimensional max pooling layer of size 2*2 to obtain 128 face feature maps with a size of 32*32.

[0083] In this embodiment, as Figure 3 shown, the two-dimensional feature extraction network D4 is connected to the two-dimensional feature extraction network D3. The two-dimensional feature extraction network D4 includes a two-dimensional convolutional layer with 256 convolutional kernels and a two-dimensional max pooling layer of size 2*2. The two-dimensional convolutional layer with 256 convolutional kernels is used to extract age features from the face feature map output by the two-dimensional feature extraction network D3 to obtain 256 face feature maps with a size of 32*32. Then, the 256 face feature maps with a size of 32*32 are downsampled by the two-dimensional max pooling layer of size 2*2 to obtain 256 face feature maps with a size of 16*16.

[0084] In this embodiment, the two-dimensional feature extraction networks D1, D2, D3, and D4 all perform age feature extraction through 2D convolution operations. Among them, the formula for calculating the size of the feature map in 2D convolution operation is:

[0085]

[0086] where padding is the padding value, Stride is the padding step, w in is the size of the input image, W out is the size of the output feature map, and F is the number of convolutional kernels. In practical applications, by way of example, both the padding step and the padding value can be set to 1.

[0087] In this embodiment, from Figure 3 It can be seen that as the two-dimensional feature extraction network progresses, the size of the age feature map gradually decreases and the number gradually increases. As the size of the age feature map decreases and the number increases, the age feature extraction becomes more refined, and the age features of each face image can be extracted in all directions, so as to extract various age features of the face when a person shows different expressions.

[0088] Step 204: Perform feature fusion on the first feature map and the second feature map to obtain a target feature map.

[0089] Among them, through three-dimensional feature extraction, a spatial feature map of 16*16*256 can be obtained, and through two-dimensional feature extraction, an age feature map of 16*16*256 can be obtained, and the two are of equal size. The spatial feature map and the age feature map are subjected to feature fusion through convolution operation to re-obtain a target feature map of 1*131072 size with both spatial and age features. Specifically, it can be through, for example, Figure 1 The feature fusion model 30 in the age recognition model shown in the figure performs feature fusion to obtain a target feature map of 1*131072 size.

[0090] In a specific embodiment, the performing feature fusion on the first feature map and the second feature map to obtain a target feature map includes: performing splicing processing on the first feature map and the second feature map to obtain a third feature map; performing convolution on the third feature map to obtain the target feature map.

[0091] Among them, the spatial feature map of 16*16*256 and the age feature map of 16*16*256 are spliced to obtain a feature map of 16*16*512. At this time, only the spatial feature map and the age feature map are spliced together, and no feature fusion is performed. After performing convolution calculation on the spliced feature map to obtain a target feature map of 1*131072 size, the feature fusion is realized.

[0092] In this embodiment, by performing feature fusion on the first feature map and the second feature map, the extracted age features can be fused with the spatial features with spatial variation characteristics, and the spatial information and age information in the face image can be effectively retained.

[0093] Step 205: Perform age prediction based on the target feature map to obtain the face age of the target user.

[0094] Among them, after obtaining the target feature map of 1*131072 size, the target feature map can be input into, for example, Figure 1Perform age prediction on the prediction model 40 in the age recognition model shown. Specifically, perform age prediction on the target feature map with a size of 1*131072 through a multi-class softmax classification layer to obtain the face age of the target user. Perform age prediction through the fused target feature map to obtain the predicted ages of the target user under different expressions. It can solve the problem that the age values measured for the same person vary greatly due to facial expression changes, and solve the problem of the lack of a certain stability in model prediction.

[0095] This application provides an age prediction method, including: obtaining multiple face images of a target user to form an input image group; inputting the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map; inputting one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map; performing feature fusion on the first feature map and the second feature map to obtain a target feature map; performing age prediction based on the target feature map to obtain the face age of the target user. By extracting spatial features through the first feature extraction model, it can effectively extract the spatial features of face images under different expressions; by extracting age features through the second feature extraction model, it can effectively extract the age features of face images; by performing feature fusion on the first feature map and the second feature map, it can fuse the extracted age features with the spatial features with spatial variation characteristics, and can effectively retain the spatial information and age information in the face image; by performing age prediction through the fused target feature map, it can solve the problem that the age values measured for the same person vary greatly due to facial expression changes, and solve the problem of the lack of a certain stability in model prediction.

[0096] In a specific embodiment, the face age recognition model is obtained through pre-training. In the process of training the face age recognition model, a large number of face images are obtained, and corresponding age labels are attached to each face image, and the face images are divided into a training set and a test set according to a certain proportion. The training set is used to train the model, and the test set is used to test and adjust the parameters of the trained model. Then, each face image containing an age label is input into the face age recognition model that has not been trained yet for training, so that the face age recognition model can learn the common spatial characteristics and spatial feature changes among a large number of face images with different expressions corresponding to the same age, as well as the different spatial characteristics and spatial feature changes among a large number of face images with different expressions corresponding to different ages, so that the face age recognition model has the ability to distinguish the face ages under different ages and different expressions, and further enables the predicted results output by the trained face age recognition model to match the actual ages in the age labels corresponding to each age image.

[0097] In this embodiment, as Figure 6As shown, the method further includes:

[0098] Step 601: Obtain a face image sample set, where the face image sample set includes multiple groups of sample input image groups and age labels corresponding to each of the multiple groups of sample input image groups.

[0099] Among them, a sample input image group refers to an image combination composed of multiple face images of the same person with different expressions, that is, a sample input image group includes multiple face images of the same sample, and the face expressions in these multiple face images are different.

[0100] Among them, the age label corresponding to a sample input image group refers to the label of the real age of the face (i.e., the user) in the sample input image group.

[0101] Among them, the face image sample set refers to a set of sample input image groups of different ages and different samples.

[0102] Specifically, the sample images in the sample input image group can be multiple photos taken of the sample, or multiple face images intercepted from the video of the sample. It can be understood that the multiple face images can also be multiple face images of the sample obtained from a pre-established image database. It should be understood that during the actual training process, multiple face image sample sets can be obtained simultaneously for one training to train the face age recognition model. Since the processing process of each sample input image group is the same during one training process, the training process of the face age recognition model is introduced in this embodiment of the present application with the processing process of one sample input image group.

[0103] Specifically, after obtaining the sample input image group, label its corresponding age label, and specifically, the one-hot method can be used for labeling.

[0104] Step 602: Input the target sample input image group into the initial first feature extraction model for spatial feature extraction to obtain a sample first feature map, where the target sample input image group is any one of the multiple groups of sample input image groups.

[0105] Among them, the initial first feature extraction model is a three-dimensional feature extraction model, specifically, it can be any three-dimensional convolution model capable of realizing spatial feature extraction. When a person's face makes different expressions, there will be different spatial feature changes. The three-dimensional convolution model can extract the face space when the face image has different expressions, etc. Inputting the target sample input image group into the three-dimensional feature extraction model can extract the spatial features with spatial change characteristics in the face image to obtain a sample first feature map.

[0106] Step 603: Input one of the face images in the target sample input image group into the initial second feature extraction model for age feature extraction to obtain a sample second feature map.

[0107] Among them, the initial second feature extraction model is a two-dimensional feature extraction model, specifically, it can be any two-dimensional convolutional model that can implement age feature extraction. Input one of the face images in the target sample input image group into the two-dimensional feature extraction model, and the age features of the face image can be extracted to obtain a sample second feature map.

[0108] Step 604: Perform feature fusion on the sample first feature map and the sample second feature map to obtain a sample target feature map.

[0109] Among them, the sample first feature map and the sample second feature map are first concatenated and then convolved to obtain the sample target feature map. Among them, concatenating the sample first feature map and the sample second feature map only concatenates the spatial feature map and the age feature map together, without performing feature fusion. After performing convolution calculation on the concatenated feature map, feature fusion is achieved. By performing feature fusion on the sample first feature map and the sample second feature map, the extracted age features can be fused with the spatial features with spatial variation characteristics, and the spatial information and age information in the face image can be effectively retained.

[0110] Step 605: Perform age prediction based on the sample target feature map to obtain the predicted age of the sample user corresponding to the target sample input image group.

[0111] Among them, age prediction is performed through the fused target feature map to obtain the predicted age of the sample in different expressions.

[0112] Step 606: Calculate the age loss according to the predicted age and the age label corresponding to the target sample input image group, and perform iterative parameter tuning on the initial first feature extraction model and the initial second feature extraction model based on the age loss to obtain the first feature extraction model and the second feature extraction model.

[0113] Among them, the age loss is used to reflect the deviation between the predicted age and the true age. The larger the age loss, the less accurate the face age recognition model is. The smaller the age loss, the more accurate the face age recognition model is.

[0114] In a specific embodiment, the age loss can be determined by constructing a loss function. Specifically, it can be a softmax loss function, and the function is:

[0115]

[0116] Among them, the output in the softmax classification layer is marked as Yc = {y c1 , y c2 , …, y cn}, cn represents the size of the age value range, N represents the total number of samples, represents the true age value label of the j-th sample, Yc represents the predicted age probability value, and L c is the age loss value of the model.

[0117] In a specific embodiment, the predicted age and the age label can also be directly presented in numerical form, and then the difference between the two can be used to measure the deviation between the predicted age and the age label. The deviation corresponding to a sample face image is equal to the absolute value of the difference between the predicted age and the age label, denoted as Lc = |Y - P|, where Y is the true age value and P is the predicted age value.

[0118] In a specific implementation, the adam algorithm is used to optimize the model parameters. The number of iterations is set to 500 times, the initial learning rate is set to 0.001, the weight decay is set to 0.0005, and the learning rate decays to 1 / 10 of the original every 50 iterations. Through iterative parameter tuning, a trained first feature extraction model and a trained second feature extraction model are obtained.

[0119] In this embodiment, the trained first feature extraction model is a three-dimensional feature extraction model with the ability to extract spatial characteristics; the trained second feature extraction model is a two-dimensional feature extraction model with the ability to extract age characteristics; the spatial characteristics extracted by the three-dimensional feature extraction model and the age characteristics extracted by the two-dimensional feature extraction are fused, which well makes up for the defect that the two-dimensional feature extraction model cannot extract age features with spatial characteristics.

[0120] As Figure 7 shown, in an embodiment, the present invention provides an age prediction device, and the device includes:

[0121] An acquisition module 701, configured to acquire multiple face images of a target user to form an input image group.

[0122] A first feature extraction module 702, configured to input the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map.

[0123] A second feature extraction module 703, configured to input one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map.

[0124] A feature fusion module 704, configured to perform feature fusion on the first feature map and the second feature map to obtain a target feature map.

[0125] An age prediction module 705 for predicting the age based on the target feature map to obtain the facial age of the target user.

[0126] As Figure 8 shown, in one embodiment, it is an internal structure diagram of a computer device. The computer device can be an age prediction device, or a terminal or server connected to an age prediction device. As Figure 8 shown, the computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and can also store a computer program. When the computer program is executed by the processor, the processor can implement an age prediction method. The internal memory can also store a computer program. When the computer program is executed by the processor, the processor can execute an age prediction method. The network interface is used for communication with the outside. Those skilled in the art can understand that Figure 8 the structure shown in

[0127] In one embodiment, an age prediction method provided by the present application can be implemented in the form of a computer program, and the computer program can run on a computer device as Figure 8 shown. Each program template constituting the age prediction device can be stored in the memory of the computer device. For example, an acquisition module 701, a first feature extraction module 702, a second feature extraction module 703, a feature fusion module 704, and an age prediction module 705.

[0128] A computer device, including a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: obtaining multiple facial images of a target user to form an input image group; inputting the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map; inputting one of the facial images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map; performing feature fusion on the first feature map and the second feature map to obtain a target feature map; predicting the age based on the target feature map to obtain the facial age of the target user.

[0129] In one embodiment, after obtaining multiple face images of the target user and forming an input image group, the method further includes: performing coordinate affine transformation on the multiple face images respectively to adjust the faces in the multiple face images to the frontal face state; intercepting the effective face regions of the multiple face images after the coordinate affine transformation, and performing normalization processing on the intercepted effective face regions.

[0130] In one embodiment, the intercepting the effective face regions of the multiple face images after the coordinate affine transformation includes: determining multiple face feature points of a target face image; the target face image being any one of the multiple face images after the coordinate affine transformation; determining the feature point distances between the multiple face feature points pairwise; and intercepting the effective face region of the target face image according to the feature point distances.

[0131] In one embodiment, the obtaining the target feature map by performing feature fusion on the first feature map and the second feature map includes: splicing the first feature map and the second feature map to obtain a third feature map; and performing convolution on the third feature map to obtain the target feature map.

[0132] In one embodiment, the first feature extraction model includes N layers of three-dimensional feature extraction networks, each layer of three-dimensional feature extraction network including a three-dimensional convolutional layer and a three-dimensional max pooling layer, N being a positive integer greater than or equal to 1; the inputting the input image group into a pre-established first feature extraction model to perform spatial feature extraction to obtain a first feature map includes: inputting the (i - 1)-th spatial feature map into the three-dimensional convolutional layer in the i-th layer of the three-dimensional feature extraction network to perform spatial feature extraction to obtain the i-th initial spatial feature map; inputting the i-th initial spatial feature map into the three-dimensional max pooling layer in the i-th layer of the three-dimensional feature extraction network to perform dimensionality reduction processing to obtain the i-th spatial feature map; and determining the N-th spatial feature map as the first feature map; where 1 ≤ i ≤ N; when i = 1, the (i - 1)-th spatial feature map is the input image group.

[0133] In one embodiment, the second feature extraction model includes N layers of two-dimensional feature extraction networks. Each layer of the two-dimensional feature extraction network includes a two-dimensional convolutional layer and a two-dimensional max pooling layer, where N is a positive integer greater than or equal to 1. The step of inputting one of the face images in the input image group into the pre-established second feature extraction model for age feature extraction to obtain a second feature map includes: inputting the (i - 1)-th age feature map into the two-dimensional convolutional layer in the i-th layer of the two-dimensional feature extraction network for age feature extraction to obtain the i-th initial age feature map; inputting the i-th initial age feature map into the two-dimensional max pooling layer in the i-th layer of the two-dimensional feature extraction network for dimensionality reduction processing to obtain the i-th age feature map; and determining the N-th age feature map as the second feature map, where 1 ≤ i ≤ N. When i = 1, the (i - 1)-th age feature map is one of the face images in the input image group.

[0134] In one embodiment, when the computer program is executed by the processor, the processor is further caused to perform the following steps: obtaining a face image sample set, where the face image sample set includes multiple groups of sample input image groups and age labels respectively corresponding to the multiple groups of sample input image groups; inputting a target sample input image group into an initial first feature extraction model for spatial feature extraction to obtain a sample first feature map, where the target sample input image group is any one of the multiple groups of sample input image groups; inputting one of the face images in the target sample input image group into an initial second feature extraction model for age feature extraction to obtain a sample second feature map; performing feature fusion on the sample first feature map and the sample second feature map to obtain a sample target feature map; performing age prediction based on the sample target feature map to obtain the predicted age of the sample user corresponding to the target sample input image group; calculating an age loss based on the predicted age and the age label corresponding to the target sample input image group, and iteratively adjusting the parameters of the initial first feature extraction model and the initial second feature extraction model based on the age loss to obtain the first feature extraction model and the second feature extraction model.

[0135] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor is caused to perform the following steps: obtaining multiple face images of a target user to form an input image group; inputting the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map; inputting one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map; performing feature fusion on the first feature map and the second feature map to obtain a target feature map; and performing age prediction based on the target feature map to obtain the face age of the target user.

[0136] In one embodiment, after obtaining multiple face images of the target user and forming an input image group, the method further includes: respectively performing coordinate affine transformation on the multiple face images to adjust the faces in the multiple face images to a frontal face state; intercepting the effective face regions of the multiple face images after the coordinate affine transformation, and performing normalization processing on the intercepted effective face regions.

[0137] In one embodiment, the intercepting the effective face regions of the multiple face images after the coordinate affine transformation includes: determining multiple face feature points of a target face image; the target face image is any one of the multiple face images after the coordinate affine transformation; determining the feature point distances between the multiple face feature points pairwise; and intercepting the effective face region of the target face image according to the feature point distances.

[0138] In one embodiment, the obtaining the target feature map by performing feature fusion on the first feature map and the second feature map includes: splicing the first feature map and the second feature map to obtain a third feature map; and performing convolution on the third feature map to obtain the target feature map.

[0139] In one embodiment, the first feature extraction model includes N layers of three-dimensional feature extraction networks, and each layer of three-dimensional feature extraction network includes a three-dimensional convolutional layer and a three-dimensional max pooling layer, where N is a positive integer greater than or equal to 1; the inputting the input image group into a pre-established first feature extraction model to perform spatial feature extraction to obtain a first feature map includes: inputting the (i - 1)-th spatial feature map into the three-dimensional convolutional layer in the i-th layer of the three-dimensional feature extraction network to perform spatial feature extraction to obtain the i-th initial spatial feature map; inputting the i-th initial spatial feature map into the three-dimensional max pooling layer in the i-th layer of the three-dimensional feature extraction network to perform dimensionality reduction processing to obtain the i-th spatial feature map; and determining the N-th spatial feature map as the first feature map; where 1 ≤ i ≤ N; when i = 1, the (i - 1)-th spatial feature map is the input image group.

[0140] In one embodiment, the second feature extraction model includes N layers of two-dimensional feature extraction networks, and each layer of two-dimensional feature extraction network includes a two-dimensional convolutional layer and a two-dimensional max pooling layer, where N is a positive integer greater than or equal to 1; the step of inputting one of the face images in the input image group into the pre-established second feature extraction model for age feature extraction to obtain a second feature map includes: inputting the (i - 1)-th age feature map into the two-dimensional convolutional layer in the i-th layer of two-dimensional feature extraction network for age feature extraction to obtain the i-th initial age feature map; inputting the i-th initial age feature map into the two-dimensional max pooling layer in the i-th layer of two-dimensional feature extraction network for dimensionality reduction processing to obtain the i-th age feature map; determining the N-th age feature map as the second feature map; where 1 ≤ i ≤ N; when i = 1, the (i - 1)-th age feature map is one of the face images in the input image group.

[0141] In one embodiment, when the computer program is executed by the processor, the processor is further caused to perform the following steps: obtaining a face image sample set, where the face image sample set includes multiple groups of sample input image groups and age labels corresponding to the multiple groups of sample input image groups respectively; inputting a target sample input image group into an initial first feature extraction model for spatial feature extraction to obtain a sample first feature map, where the target sample input image group is any one of the multiple groups of sample input image groups; inputting one of the face images in the target sample input image group into an initial second feature extraction model for age feature extraction to obtain a sample second feature map; performing feature fusion on the sample first feature map and the sample second feature map to obtain a sample target feature map; performing age prediction based on the sample target feature map to obtain a predicted age of the sample user corresponding to the target sample input image group; calculating an age loss based on the predicted age and the age label corresponding to the target sample input image group, and iteratively adjusting the parameters of the initial first feature extraction model and the initial second feature extraction model based on the age loss to obtain the first feature extraction model and the second feature extraction model.

[0142] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0143] The above disclosure is only for the preferred embodiments of the present application. Of course, the scope of rights of the present application cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. An age prediction method, characterized in that, The method includes: Obtaining multiple face images of a target user to form an input image group; Inputting the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map, where the first feature extraction model includes N layers of three-dimensional feature extraction networks, and each layer of three-dimensional feature extraction network includes a three-dimensional convolutional layer and a three-dimensional max pooling layer, and N is a positive integer greater than or equal to 1; Inputting one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map, where the second feature extraction model includes N layers of two-dimensional feature extraction networks, and each layer of two-dimensional feature extraction network includes a two-dimensional convolutional layer and a two-dimensional max pooling layer, and N is a positive integer greater than or equal to 1; Performing feature fusion on the first feature map and the second feature map to obtain a target feature map; Performing age prediction based on the target feature map to obtain the face age of the target user.

2. The method according to claim 1, wherein After obtaining multiple face images of the target user to form an input image group, it further includes: Performing coordinate affine transformation on the multiple face images respectively to adjust the faces in the multiple face images to the frontal face state; Performing face effective region interception on the multiple face images after coordinate affine transformation, and performing normalization processing on the intercepted face effective regions.

3. The method according to claim 2, characterized in that The performing face effective region interception on the multiple face images after coordinate affine transformation includes: Determining multiple face feature points of a target face image; the target face image is any one of the multiple face images after coordinate affine transformation; Determining the feature point distances between the multiple face feature points pairwise; Performing face effective region interception on the target face image according to the feature point distances.

4. The method according to claim 1, characterized in that The performing feature fusion on the first feature map and the second feature map to obtain a target feature map includes: Performing splicing processing on the first feature map and the second feature map to obtain a third feature map; Performing convolution on the third feature map to obtain the target feature map.

5. The method according to claim 1, wherein The inputting the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map includes: Inputting the (i - 1)-th spatial feature map into the three-dimensional convolutional layer in the i-th layer of three-dimensional feature extraction network for spatial feature extraction to obtain the i-th initial spatial feature map; Inputting the i-th initial spatial feature map into the three-dimensional max pooling layer in the i-th layer of three-dimensional feature extraction network for dimensionality reduction processing to obtain the i-th spatial feature map; Determining the N-th spatial feature map as the first feature map; where 1 ≤ i ≤ N; when i = 1, the (i - 1)-th spatial feature map is the input image group.

6. The method according to claim 1, characterized in that, The inputting one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map includes: Inputting the (i - 1)-th age feature map into the two-dimensional convolutional layer in the i-th layer of two-dimensional feature extraction network for age feature extraction to obtain the i-th initial age feature map; Input the i-th initial age feature map into the two-dimensional max pooling layer in the i-th layer of the two-dimensional feature extraction network for dimensionality reduction processing to obtain the i-th age feature map; Determine the N-th age feature map as the second feature map; where 1 ≤ i ≤ N; when i = 1, the (i - 1)-th age feature map is one of the face images in the input image group.

7. The method according to claim 1, wherein The method further includes: Obtain a face image sample set, where the face image sample set includes multiple groups of sample input image groups and the age labels corresponding to the multiple groups of sample input image groups respectively; Input the target sample input image group into the initial first feature extraction model for spatial feature extraction to obtain a sample first feature map, where the target sample input image group is any one of the multiple groups of sample input image groups; Input one of the face images in the target sample input image group into the initial second feature extraction model for age feature extraction to obtain a sample second feature map; Perform feature fusion on the sample first feature map and the sample second feature map to obtain a sample target feature map; Perform age prediction based on the sample target feature map to obtain the predicted age of the sample user corresponding to the target sample input image group; Calculate an age loss according to the predicted age and the age label corresponding to the target sample input image group, and perform iterative parameter tuning on the initial first feature extraction model and the initial second feature extraction model based on the age loss to obtain the first feature extraction model and the second feature extraction model.

8. An age prediction device, characterized in that, The device includes: An acquisition module, configured to acquire multiple face images of a target user to form an input image group; A first feature extraction module, configured to input the input image group into a pre-established first feature extraction model for spatial feature extraction to obtain a first feature map, where the first feature extraction model includes N layers of three-dimensional feature extraction networks, and each layer of three-dimensional feature extraction network includes a three-dimensional convolutional layer and a three-dimensional max pooling layer, and N is a positive integer greater than or equal to 1; A second feature extraction module, configured to input one of the face images in the input image group into a pre-established second feature extraction model for age feature extraction to obtain a second feature map, where the second feature extraction model includes N layers of two-dimensional feature extraction networks, and each layer of two-dimensional feature extraction network includes a two-dimensional convolutional layer and a two-dimensional max pooling layer, and N is a positive integer greater than or equal to 1; A feature fusion module, configured to perform feature fusion on the first feature map and the second feature map to obtain a target feature map; An age prediction module, configured to perform age prediction based on the target feature map to obtain the face age of the target user.

9. A computer device, including a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 - 7.

10. A computer-readable storage medium, storing a computer program, and when the computer program is executed by a processor, the processor executes the steps of the method according to any one of claims 1 - 7.

Images