Unbalance mark distribution learning method suitable for face attraction evaluation

Abstract

The invention discloses an unbalanced mark distribution learning method suitable for face attraction evaluation, and relates to the technical field of machine learning. According to the invention, an imbalance mark distribution learning network comprising a feature encoder, a full-mark encoder and a mark prediction decoder is constructed; in the training process, during back propagation, a gradient decomposition multi-momentum distribution mechanism is adopted to decompose a distribution alignment loss gradient into a parallel component and an orthogonal component relative to a mark prediction loss gradient; and maintaining independent momentum buffer variables for the marked prediction loss gradient, the parallel component and the orthogonal component respectively, fusing the independent momentum buffer variables by using gating to generate a total update gradient, and then updating the learnable network parameters based on the total update gradient. When the predicted face attraction score mark distribution obtained by the method is used for face attraction evaluation, the obtained face attraction evaluation result is more in line with the evaluation effect of group cognition.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to an imbalanced labeled distribution learning method suitable for facial attractiveness evaluation. Background Technology

[0002] In facial attractiveness assessment applications, the distribution of facial attractiveness labels often exhibits a significant imbalance. This imbalance manifests as some rating levels having extremely high descriptive power, while others have extremely low descriptive power, leading to feature drift problems in imbalanced label distribution learning networks during prediction. Therefore, researchers have begun to focus on learning methods based on imbalanced label distributions. Existing mainstream methods typically introduce distribution alignment to constrain the feature space and alleviate feature drift. However, current imbalanced label distribution-based learning methods largely ignore the gradient conflict between the label prediction and distribution alignment tasks when jointly optimizing them. This makes it difficult to obtain a predicted facial attractiveness rating label distribution that closely approximates the real facial attractiveness rating label distribution. Since the predicted facial attractiveness rating label distribution cannot closely approximate the real distribution, facial attractiveness assessments based on the predicted label distribution are unlikely to achieve evaluation results that better align with public perception. To address this, the applicant has designed an imbalanced label distribution learning method suitable for facial attractiveness assessment. Summary of the Invention

[0003] To address the aforementioned shortcomings in existing technologies, this application proposes an imbalanced labeled distribution learning method suitable for facial attractiveness evaluation.

[0004] An imbalanced labeled distribution learning method suitable for facial attractiveness evaluation includes the following steps: S1. Construct an imbalanced label distribution learning network comprising a feature encoder, a fully labeled encoder, and a label prediction decoder; wherein, The feature encoder is used to extract features from the feature vector of the input face image, mapping the original face image feature vector into a feature representation; A fully labeled encoder is used to encode the global labeled distribution of the input face attractiveness score, mapping it to a global labeled representation; The label prediction decoder obtains the reconstructed face attractiveness score label distribution based on the global label representation, and the reconstructed face attractiveness score label distribution is called the true face attractiveness score label distribution; the label prediction decoder obtains the predicted face attractiveness score label distribution based on the feature representation, and the predicted face attractiveness score label distribution is called the predicted face attractiveness score label distribution; the predicted face attractiveness score label distribution is the final output of the imbalanced label distribution learning network; S2. The imbalanced label distribution learning network is trained based on the training set and the total optimization loss of the imbalanced label distribution learning network to obtain the imbalanced label distribution learning network model. During the training process of the imbalanced label distribution learning network, a gradient decomposition multi-momentum allocation mechanism is used to adaptively update the learnable parameters during backpropagation. The gradient decomposition multi-momentum allocation mechanism for adaptive updating of learnable parameters specifically includes the following steps: decomposing the distribution alignment loss gradient into parallel and orthogonal components relative to the label prediction loss gradient; maintaining independent momentum buffer variables for the label prediction loss gradient, parallel components, and orthogonal components respectively; using gating to fuse the independent momentum buffer variables to generate the total update gradient; and then updating the learnable network parameters based on the total update gradient.

[0005] S3. Obtain the face image feature vector based on the imbalanced dataset to be predicted, and then input it into the imbalanced label distribution learning network model for forward propagation once to obtain the predicted face attractiveness score label distribution.

[0006] Preferably, in step S2, the training set is obtained based on the SCUT-FBP dataset; the SCUT-FBP dataset is an attractiveness rating dataset for face images, which contains face image feature vector samples and their corresponding rating label distributions; in the SCUT-FBP dataset, the rating label distribution for each face image is text data, and the rating label distribution for each face image is obtained by statistically analyzing the voting ratios of all users at each rating level, and the rating label distribution is the global label distribution for face attractiveness rating.

[0007] Preferably, in step S2, the total optimization loss of the imbalanced label distribution learning network includes label prediction loss, distribution alignment loss, and label prediction dynamic weights and distribution alignment dynamic weights used to balance the label prediction loss and distribution alignment loss.

[0008] Preferably, in step S2, the label prediction dynamic weights and distribution alignment dynamic weights are dynamically generated using the gradient norm.

[0009] Preferably, the label prediction dynamic weights in the t-th epoch are obtained. Alignment of dynamic weights with distribution The method includes the following steps: Step A: Calculate the gradient norm of the label prediction loss and the gradient norm of the distribution alignment loss in the t-th epoch; Step B: Calculate the sum of gradient norms in the t-th epoch; Step C: Obtain the label prediction dynamic weights in the t-th epoch based on the sum of gradient norms Sum(t) in the t-th epoch. Dynamic weights aligned with distribution .

[0010] Preferably, step C specifically includes the following steps: Step C-1: Obtain the dynamic weights for label prediction in the t-th epoch based on the ratio of the gradient norm of the label prediction loss to the sum of the gradient norms. ; Step C-2: Obtain the distribution alignment dynamic weights in the t-th epoch based on the ratio of the distribution alignment loss gradient norm to the sum of gradient norms in the t-th epoch. Among them, the label predicts the dynamic weights. Dynamic weights aligned with distribution The sum of them equals 1.

[0011] Preferably, in step S2, before adaptively updating the learnable parameters using the gradient decomposition multi-momentum allocation mechanism, the label prediction loss gradient and distribution alignment loss gradient in the current Epoch are calculated, and the label prediction dynamic weights and distribution alignment dynamic weights are obtained based on the gradient norms of the label prediction loss gradient and the distribution alignment loss gradient.

[0012] Preferably, in step S2, independent momentum buffer variables, parallel components, and orthogonal components are assigned to the labeled predicted loss gradient, parallel components, and orthogonal components respectively; specifically, this includes the following steps: First, the standard deviations of the labeled prediction loss gradient, parallel component, and orthogonal component within time step T are calculated using a sliding window. Then, the momentum coefficients corresponding to the labeled prediction loss gradient, parallel components, and orthogonal components are adaptively updated based on the standard deviation of the labeled prediction loss gradient, parallel components, and orthogonal components within time step T.

[0013] Preferably, in step S2, the gating and fusion of independent momentum buffer variables is used to generate the total update gradient, which specifically includes the following steps: First, a normalization factor is calculated based on the standard deviations of the parallel components and the orthogonal components. Second, the ratios of the standard deviations of the parallel components and the orthogonal components to the normalization factor are calculated respectively to obtain the fusion weights of the parallel components and the fusion weights of the orthogonal components. Then, the momentum buffer variables of the parallel components and the momentum buffer variables of the orthogonal components are weighted and summed based on the fusion weights of the parallel components and the fusion weights of the orthogonal components to obtain the fused aligned gradient momentum. Finally, based on the labeled prediction loss gradient momentum, the fused alignment gradient momentum, and the labeled prediction dynamic weights and distribution alignment dynamic weights, the total update gradient for the current epoch is generated.

[0014] Preferably, step S2 includes the following steps: inputting the face image feature vectors in the training set into the feature encoder, and inputting the global label distribution of the face attractiveness score in the training set into the full label encoder; after the face image feature vectors are forward propagated by the feature encoder to output the feature representation, the feature representation is transmitted to the label prediction decoder, and the label prediction decoder outputs the predicted face attractiveness score label distribution; The global label distribution of facial attractiveness scores in the training set is forward-propagated by the fully labeled encoder to output a global label representation. The global label representation is then transmitted to the label prediction decoder, which outputs the true facial attractiveness score label distribution. Then, the total optimization loss L of the unbalanced label distribution learning network is calculated. total The algorithm performs backpropagation guided by the total optimization loss. During backpropagation, the gradients of the label prediction loss and distribution alignment loss in the current epoch are first calculated. Based on the gradient norms of the label prediction loss and distribution alignment loss gradients, the dynamic weights of label prediction and distribution alignment are obtained. Then, a gradient decomposition multi-momentum allocation mechanism is executed to adaptively update the learnable parameters. After iterating for 300 epochs, a training segment is completed, resulting in an imbalanced label distribution learning network model. In the process of training the imbalanced label distribution learning network, a total of 10 training segments are performed, resulting in 10 imbalanced label distribution learning network models.

[0015] Compared with the prior art, the beneficial technical effects of the present invention are as follows: In this application, the dynamic weight λ for prediction is labeled. p Alignment of dynamic weights λ with distribution dThe gradient norm of the label prediction loss and the gradient norm of the distribution alignment loss are dynamically generated. This application refers to the above setting as a gradient norm-based dynamic weighting mechanism. This gradient norm-based dynamic weighting mechanism allows this application to learn the imbalanced label distribution more accurately and obtain a more accurate predicted facial attractiveness score label distribution without introducing a complex hyperparameter adjustment process. In addition, this application also uses a gradient decomposition multi-momentum allocation mechanism to adaptively update the learnable parameters, including the following steps: performing gradient component decomposition of the distribution alignment loss gradient relative to the label prediction loss gradient to obtain parallel and orthogonal components relative to the label prediction loss gradient; The label prediction loss gradient, parallel components, and orthogonal components each maintain independent momentum buffer variables. Finally, a total update gradient is generated through gated fusion, and parameters are updated based on this total update gradient. The gradient decomposition multi-momentum allocation mechanism in this application effectively reduces the mutual interference between the distribution alignment task and the label prediction task. This mechanism significantly improves the stability and convergence of imbalanced label distribution learning, enabling the imbalanced label distribution learning network to obtain better model parameters. Consequently, this application can obtain a predicted face attractiveness rating label distribution that more closely approximates the real face attractiveness rating label distribution based on the imbalanced label distribution learning network model. Tests show that when the method described in this application evaluates face attractiveness based on the predicted face attractiveness rating label distribution obtained from the test set, it can balance the accuracy of face attractiveness ranking and directory coverage, making the face attractiveness evaluation results more consistent with the evaluation effect of group cognition. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the network structure of the method of the present invention. Detailed Implementation

[0017] Example 1: An imbalanced labeled distribution learning method suitable for facial attractiveness evaluation, wherein the imbalanced labeled distribution learning method for facial attractiveness evaluation predicts on an imbalanced dataset—the SCUT-FBP dataset—including the following steps: S1. Construct an imbalanced label distribution learning network; The network structure of the imbalanced label distribution learning network is as follows: Figure 1 As shown; the imbalanced label distribution learning network includes a feature encoder, a full-label encoder, and a label prediction decoder.

[0018] The feature encoder is used to extract features from the feature vector of the input face image, mapping the original face image feature vector into a feature representation; A fully labeled encoder is used to encode the global labeled distribution of facial attractiveness scores, mapping it to a global labeled representation. In this application, the global labeled distribution of facial attractiveness scores is obtained from the SCUT-FBP dataset, which is the dataset described in the paper "Imbalanced Label Distribution". The SCUT-FBP dataset, published in *Learning*, contains 500 frontal, unobstructed facial images. The faces are aged between 15 and 60, with neutral facial expressions. Each image is 350×350 pixels. All facial images were rated for aesthetic appeal by 75 volunteers. The rating process involved randomly displaying images one by one and asking raters to rate their attractiveness on a scale of 1 to 5, where 1 represents the lowest attractiveness and 5 the highest. Based on the raters' ratings, a label distribution was generated for each facial image. In other words, the rating label distribution for each facial image in the SCUT-FBP dataset is text data, obtained by statistically analyzing the voting proportions of all raters at each rating level (1 to 5), reflecting the group's subjective perception of the facial image. In this application, this rating label distribution is the global label distribution for facial attractiveness ratings. The SCUT-FBP dataset also includes facial feature vectors for each facial image.

[0019] The label prediction decoder obtains the reconstructed face attractiveness score label distribution based on the global label representation. The reconstructed face attractiveness score label distribution is called the real face attractiveness score label distribution. The label prediction decoder obtains the predicted face attractiveness score label distribution based on the feature representation. The predicted face attractiveness score label distribution is called the predicted face attractiveness score label distribution. The predicted face attractiveness score label distribution is the final output of the imbalanced label distribution learning network.

[0020] The feature encoder in this application has the same structure as the feature encoder disclosed in CN 120387494 B; the full-tag encoder in this application has the same structure as the full-tag encoder disclosed in CN 120387494 B; and the tag prediction decoder in this application has the same structure as the decoder disclosed in CN 120387494 B.

[0021] The imbalanced label distribution learning network in this application enables the implementation of two tasks: label prediction and distribution alignment. The label prediction task is accomplished based on a feature encoder and a label prediction decoder. Specifically, the label prediction decoder receives the feature representation output by the feature encoder and outputs a predicted facial attractiveness score label distribution. The distribution alignment task is accomplished based on a feature encoder and a fully labeled encoder. The fully labeled encoder obtains a global label representation based on the feature representation output by the feature encoder. Then, it calculates the distance between the feature representation output by the feature encoder and the global label representation output by the fully labeled encoder. This distance is the distribution alignment loss, which can be used to constrain the feature space so that its distribution structure aligns with the semantic structure of the label space.

[0022] S2. Obtain the training and test sets, and train the imbalanced label distribution learning network for 10 training segments based on the training sets and the total optimization loss to obtain 10 imbalanced label distribution learning network models, including the following steps: S2-1. Obtaining the training and test sets includes the following specific steps: S2-1-1. Obtain the original dataset, which is the SCUT-FBP dataset. The SCUT-FBP dataset used in this application is an existing dataset. Specifically, the SCUT-FBP dataset is from the paper "Imbalanced Label Distribution". The SCUT-FBP dataset, published in the book *Learning*, contains 500 frontal, unobstructed facial images of people aged 15 to 60 with neutral expressions. The images are 350×350 pixels in size. All images were annotated with aesthetic ratings by 75 volunteers. The rating method involved randomly displaying each image and asking the raters to rate its attractiveness within a 5-point range. In other words, in the SCUT-FBP dataset, each facial image was rated for attractiveness by multiple annotators according to discrete rating levels, ranging from 1 to 5, where 1 represents the lowest attractiveness and 5 represents the highest. The rating label distribution for each facial image is text data, obtained by statistically analyzing the voting proportions of all users at each rating level. This reflects the subjective perception of the attractiveness of the facial image's feature vector sample by the group. In this application, the aforementioned rating label distribution is the global label distribution for facial attractiveness ratings.

[0023] S2-1-2. The original dataset is divided into 10 subsets using a 10x cross-validation method. Each subset includes 50 facial image feature vectors and their corresponding global label distributions of facial attractiveness scores. In the process of training the imbalanced label distribution learning network, 10 training segments are trained, each consisting of 300 epochs. In each training segment, 9 subsets are selected as the training set, and the remaining subsets are selected as the test set. The test sets selected in the 10 training segments of this application are all different.

[0024] S2-2: Input the face image feature vectors in the training set into the feature encoder, and input the global label distribution of face attractiveness scores in the training set into the full label encoder; after the face image feature vectors are forward propagated by the feature encoder to output feature representation, the feature representation is transmitted to the label prediction decoder, and the label prediction decoder outputs the predicted face attractiveness score label distribution. The global label distribution of facial attractiveness scores in the training set is forward-propagated by the fully labeled encoder to output a global label representation. The global label representation is then transmitted to the label prediction decoder, which outputs the true facial attractiveness score label distribution. Then, the total optimization loss L of the unbalanced label distribution learning network is calculated. total Then, guided by the total optimization loss, backpropagation is performed; during backpropagation, the gradient g of the label prediction loss in the current Epoch is first calculated. p Alignment of loss gradient g with distribution d The label prediction dynamic weights and distribution alignment dynamic weights are obtained based on the gradient norms of the label prediction loss gradient and the distribution alignment loss gradient. Then, the learnable parameters are adaptively updated using the gradient decomposition multi-momentum allocation mechanism. Here, the label prediction loss gradient refers to the gradient of the label prediction loss, and the distribution alignment loss gradient refers to the gradient of the distribution alignment loss. After 300 epochs of iteration, a training segment is completed, resulting in an imbalanced label distribution learning network model. In this application, a total of 10 training segments are performed during the training of the imbalanced label distribution learning network, resulting in 10 imbalanced label distribution learning network models. This application can obtain imbalanced label distribution learning network models with better parameters. Therefore, when using the imbalanced label distribution learning network models obtained in this application to predict imbalanced datasets in the field of face attractiveness evaluation, it can obtain a predicted face attractiveness score label distribution that is closer to the real face attractiveness score label distribution, thereby improving the accuracy of the label distribution prediction results in the face attractiveness evaluation task.

[0025] In this embodiment, the hyperparameters are set as follows during the training of the imbalanced label distribution learning network: the initial learning rate is set to 0.001, the batch size is set to 64, and the weight decay coefficient is set to... This is used to suppress parameter overfitting problems in label prediction tasks and distribution alignment tasks.

[0026] In this application, the total optimization loss L of the unbalanced label distribution learning network is... total Includes labeled prediction loss L p Distribution alignment loss L d And the label prediction dynamic weight λ used to balance the label prediction loss and the distribution alignment loss. p Alignment of dynamic weights λ with distribution d The total optimization loss L of the unbalanced labeled distribution learning network total The calculation method is shown in equation (1): L total =λ p L p +λ d L d (1) In equation (1), L p Label the predicted loss; L d For distribution alignment loss; λ p To label the predicted dynamic weights; λ d For the distributed alignment dynamic weights, where λ p +λ d =1.

[0027] This application dynamically calculates the label prediction dynamic weight λ of the label prediction loss using the gradient norm method. p Distribution alignment dynamic weights λ of distribution alignment loss d Label the predicted dynamic weights λ p Alignment of dynamic weights λ with distribution d The settings can be used to adaptively balance the label prediction loss and distribution alignment loss. This can effectively prevent the label prediction task from dominating the gradient update and suppressing the distribution alignment task, and also effectively prevent the distribution alignment task from dominating the label prediction task. This allows the imbalanced label distribution learning network to simultaneously take into account the accuracy of label prediction and the constraint effect of distribution alignment, thereby learning the imbalanced label distribution more accurately and obtaining a predicted face attractiveness score label distribution that is closer to the real face attractiveness score label distribution. In this application, the predicted loss L is labeled. pThe label prediction loss is generated during the process of obtaining the predicted face attractiveness score label distribution based on feature representation by the label prediction decoder. The label prediction loss can be used to minimize the difference between the predicted face attractiveness score label distribution and the real face attractiveness score label distribution. In this application, the label prediction loss is obtained by calculating the KL divergence loss between the real face attractiveness score label distribution and the predicted face attractiveness score label distribution. In this application, the distribution alignment loss L d It is obtained by calculating the cosine similarity between the feature representation output by the feature encoder and the global label representation output by the full label encoder; the distribution alignment loss can be used to constrain the feature space so that its distribution structure is aligned with the semantic structure of the label space, thereby alleviating the feature drift caused by the unbalanced face attractiveness score label distribution.

[0028] In this application, the dynamic weight λ for prediction is labeled. p Alignment of dynamic weights λ with distribution d It is dynamically generated using the gradient norm; taking the t-th epoch of the imbalanced label distribution learning network training process as an example, the dynamic weights of the label predictions in the t-th epoch are obtained during the training process of the imbalanced label distribution learning network. Alignment of dynamic weights with distribution The method includes the following steps: Step A: Calculate the gradient norm of the label prediction loss and the gradient norm of the distribution alignment loss in the t-th epoch; specifically: First, calculate the label prediction loss L in the t-th epoch. p (t) Gradient of the predicted loss g with respect to the labeled parameters of the learnable network p (t) and distribution alignment loss L d (t) Alignment of loss gradient g with respect to the distribution of learnable network parameters d (t); then, calculate the gradient g of the labeled prediction loss respectively. p (t) and distribution-aligned loss gradient g d The L2 norm of (t) is used to label the gradient of the prediction loss g. p The L2 norm of (t) is called the norm of the label prediction loss gradient; the distribution alignment loss gradient g d The L2 norm of (t) is called the distribution alignment loss gradient norm; the label prediction loss gradient norm is the label prediction loss gradient g. p The magnitude of (t) and the norm of the distribution alignment loss gradient are the distribution alignment loss gradient g. d The modulus of (t); where, Calculate the label prediction loss L in the t-th epoch. p (t) Gradient of the predicted loss g with respect to the labeled parameters of the learnable network p(t) and distribution alignment loss L d (t) Alignment of loss gradient g with respect to the distribution of learnable network parameters d (t), specifically including the following steps: Step A1: Determine the set of learnable network parameters, specifically: Let θ denote the learnable parameters of the imbalanced label distribution learning network. The learnable parameters θ include the learnable parameters θ of the feature encoder. e The learnable parameters θ of the fully labeled encoder g and the learnable parameters θ of the label prediction decoder d In other words, the learnable parameter θ is determined by θ e θ g and θ d Together constitute; Step A2: In the t-th epoch, extract a batch of face image feature vector samples and their corresponding global label distributions of face attractiveness scores from the training set according to the batch size; input the batch of face image feature vector samples and their corresponding global label distributions of face attractiveness scores into the imbalanced label distribution learning network, perform forward propagation, and obtain the predicted label distributions and reconstructed label distributions of all face image feature vector samples. All reconstructed label distributions are called the true label distributions. Specifically: input the batch of face image feature vectors into the feature encoder to obtain the feature representation; input the batch of global label distributions of face attractiveness scores into the full label encoder to obtain the global label representation; input the feature representations into the label prediction decoder to obtain the predicted label distributions of all face image feature vector samples; input the global label representations into the label prediction decoder to obtain the reconstructed label distributions of all face image feature vector samples. All reconstructed label distributions of face image feature vector samples are called the true label distributions. Step A3: Based on the true label distribution and predicted label distribution of all face image feature vector samples, calculate the label prediction loss L corresponding to the t-th epoch. p (t) Alignment loss with distribution L d (t), specifically including the following steps: Calculate the label prediction loss L in the t-th epoch. p (t), including the following steps: First, calculate the label prediction loss corresponding to each face image feature vector sample; wherein, the label prediction loss corresponding to each face image feature vector sample is obtained by calculating the KL divergence between the true label distribution and the predicted label distribution corresponding to the face image feature vector sample; then, calculate the average value of the label prediction loss corresponding to all face image feature vector samples, as the label prediction loss L of the batch samples extracted in the t-th epoch.p (t); Calculate the distribution alignment loss L in the t-th epoch. d (t), including the following steps: First, the distribution alignment loss corresponding to each face image feature vector sample is calculated. The distribution alignment loss for each face image feature vector sample is obtained by calculating the cosine similarity between the feature representation and the global label representation corresponding to the face image feature vector sample. Then, the average distribution alignment loss of all face image feature vector samples is calculated and used as the distribution alignment loss L for the batch of samples extracted in the t-th epoch. d (t); Step A4: Apply the label prediction loss L calculated in step A3 to each of the following steps. p (t) Alignment loss with distribution L d (t) is backpropagated, and then the loss L is predicted based on the label. p (t) Calculate the label prediction loss L in the t-th epoch. p (t) Labeled prediction loss gradient g with respect to the learnable network parameters θ p (t), and based on the distribution alignment loss L d (t) Calculate the distribution alignment loss L in the t-th epoch. d (t) Alignment of loss gradient g with respect to the distribution of learnable network parameters θ d (t); In this embodiment, the predicted loss gradient g is labeled. p (t) contains the feature encoder parameter θ e With the label prediction decoder parameter θ d The corresponding gradient component, the distribution-aligned loss gradient gd(t), includes the feature encoder parameters θ. e With full-mark encoder parameters θ g The corresponding gradient components; for learnable parameters that are not involved in the calculation of distribution alignment loss and label prediction loss, their gradient components are set to zero to ensure that the gradient g of the label prediction loss is zero. p (t) Aligns with the distribution loss gradient g d Consistency in the parameter dimension of (t) facilitates subsequent gradient norm calculation and dynamic weight determination; Step B: Calculate the sum of gradient norms in the t-th epoch; The sum of gradient norms in the t-th epoch, Sum(t), is obtained by calculating the sum of the gradient norms of the label prediction loss in the t-th epoch and the gradient norms of the distribution alignment loss in the t-th epoch, as shown in equation (2): (2) In equation (2), Let the gradient norm of the label prediction loss in the t-th epoch be denoted as . Let represent the distribution alignment loss gradient norm in the t-th epoch.

[0029] Step C: Obtain the label prediction dynamic weights in the t-th epoch based on the sum of gradient norms Sum(t) in the t-th epoch. Dynamic weights aligned with distribution Specifically, it includes the following steps: Step C-1: Obtain the dynamic weights for label prediction in the t-th epoch based on the ratio of the gradient norm of the label prediction loss to the sum of the gradient norms. The calculation method is shown in equation (3): (3) In equation (3), The gradient norm of the label prediction loss in the t-th epoch. Let be the distribution alignment loss gradient norm in the t-th epoch, and Sum(t) be the sum of gradient norms in the t-th epoch.

[0030] Step C-2: Obtain the distribution alignment dynamic weights in the t-th epoch based on the ratio of the distribution alignment loss gradient norm to the sum of gradient norms in the t-th epoch. The calculation method is shown in equation (4): (4) In equation (4), Let the distribution alignment loss gradient norm be the norm in the t-th epoch. Let Sum(t) be the gradient norm of the label prediction loss in the t-th epoch, and let Sum(t) be the sum of the gradient norms in the t-th epoch. In this application, the label prediction dynamic weights in the t-th epoch are... Dynamic weights aligned with the distribution in the t-th epoch The sum equals 1; the predicted dynamic weights λ will be labeled in the first epoch of each of the 10 training segments. p Alignment of dynamic weights λ with distribution d Initialize to 0.5; In this application, when the label prediction loss gradient norm in the t-th epoch... When the value is large, the label prediction dynamic weights calculated according to equation (3) are used. The value will also be relatively large; this indicates that when the label prediction task generates a strong update signal, the imbalanced label distribution learning network will adaptively increase the proportion of the label prediction loss corresponding to that task in the total loss, so as to make full use of the update signal. When the distribution in the t-th epoch is aligned, the loss gradient norm is... When the value is large, the distribution alignment dynamic weights calculated according to equation (4) are... The weights will also be relatively large; this indicates that when the distribution alignment task requires a significant adjustment of the feature space, the imbalanced label distribution learning network will assign higher weights to the distribution alignment task to ensure the constraint of feature alignment. In this application, the dynamic weight λ for prediction is labeled. p Alignment of dynamic weights λ with distribution d It is dynamically generated based on the gradient norm of the label prediction loss and the gradient norm of the distribution alignment loss. This application refers to the above setting as a gradient norm-based dynamic weighting mechanism. This gradient norm-based dynamic weighting mechanism enables this application to learn the imbalanced label distribution more accurately and obtain a more accurate predicted face attractiveness score label distribution without introducing an additional complex hyperparameter adjustment process. This allows for obtaining a more accurate face attractiveness evaluation result based on a more accurate predicted face attractiveness score label distribution.

[0031] In this application, since the previous example used the t-th epoch in the training process of an imbalanced label distribution learning network, the dynamic weights of the label prediction in the t-th epoch are obtained. Alignment of dynamic weights with distribution Therefore, dynamic weights are predicted based on the labels in the t-th epoch. Alignment of dynamic weights with distribution What we get is the total optimization loss L of the network learning the imbalanced label distribution in the t-th epoch. total And backpropagation is performed under the guidance of the total optimization loss in the t-th epoch; During backpropagation, the gradient of the labeled prediction loss in the current Epoch (which is the (t+1)th Epoch) is first calculated. Align the loss gradient with the distribution Then, a gradient decomposition multi-momentum allocation mechanism is executed to adaptively update the learnable parameters; specifically, during backpropagation, firstly, the labeled prediction loss gradient in the current Epoch (the current Epoch is the (t+1)th Epoch) is calculated. Align the loss gradient with the distribution Based on label prediction loss gradient and distribution-aligned loss gradient The gradient norm is used to obtain the labeled prediction dynamic weights. Dynamic weights aligned with distribution Then, the learnable parameters are adaptively updated using the gradient decomposition multi-momentum allocation mechanism. In this application, a gradient decomposition multi-momentum allocation mechanism is used to adaptively update learnable parameters, including the following steps: aligning the distribution of the loss gradient. Gradient of predicted loss relative to label Perform gradient component decomposition to obtain the gradient of the prediction loss relative to the label. Parallel components and orthogonal components ;Predict the gradient of the label loss , parallel component and orthogonal components Independent momentum buffer variables are maintained separately, and the total update gradient is generated through gating fusion. This process is called the gradient decomposition multi-momentum allocation mechanism. Then, the parameters are updated based on the total update gradient. This enables the present application to effectively alleviate the gradient conflict problem between the label prediction task and the distribution alignment task when jointly optimizing the label prediction task and the distribution alignment task. Furthermore, the gradient decomposition multi-momentum allocation mechanism is used to adaptively update the learnable parameters, specifically including the following steps: a) Align the distribution with the loss gradient Gradient of predicted loss relative to label Perform gradient component decomposition to obtain the gradient of the prediction loss relative to the label. Parallel components and orthogonal components Among them, parallel components The calculation method is shown in equation (9): (9) In equation (9), Represents the distribution-aligned loss gradient and label the predicted loss gradient The inner product, Indicates the gradient of the predicted loss. The square of the modulus; Based on distribution alignment loss gradient and parallel components Calculate orthogonal components Orthogonal components The calculation method is shown in equation (10): (10) b) Label the predicted loss gradients respectively. , parallel component and orthogonal components Assign independent momentum buffer variables , parallel component and orthogonal components Specifically: First, the gradient of the label prediction loss is calculated using a sliding window. , parallel component and orthogonal components Standard deviation within time step T Standard deviation The calculation method is shown in equation (11): (11) In equation (11), k represents the time step, and the value of k is [1, T]. The squared norm of a vector is denoted by 2. for , or ; Among them, when for When, it means that the gradient of the labeled prediction loss is calculated using formula (11). At time step The standard deviation within; at this time, for , representing the gradient of the labeled prediction loss in the (t+1)th epoch; for , represents the average gradient of the prediction loss for all labels in the (t+1)th epoch; when for When, it means that the parallel component is calculated using formula (11). The standard deviation within time step T; at this point... for , representing the parallel component in the (t+1)th epoch; for , represents the average value of all parallel components in the (t+1)th epoch; when for When, it means that the orthogonal component is calculated using formula (11). The standard deviation within time step T; at this point... for , representing the orthogonal component in the (t+1)th epoch; for , represents the average value of all orthogonal components in the (t+1)th epoch; Then, the loss gradient is predicted based on the label. , parallel component and orthogonal components Standard deviation within time step T Adaptive update of label prediction loss gradient , parallel component and orthogonal components Corresponding momentum coefficient Received the updated , , ,in, , and The update methods are all as shown in equation (12): (12) In equation (12), Label the predicted loss gradient , parallel component and orthogonal components The corresponding lower bound of the momentum coefficient, Label the predicted loss gradient , parallel component and orthogonal components The upper bound of the corresponding momentum coefficient; in this embodiment, and The values ​​are 0.1 and 0.99 respectively; In this embodiment, for , or ; when for When, it indicates that the gradient of the predicted loss is labeled. The corresponding momentum coefficient; at this time, for This means that the gradient of the labeled prediction loss is calculated using formula (11). The standard deviation within time step T; when for When, it indicates parallel components. The corresponding momentum coefficient; at this time, for This indicates that the parallel components are calculated using formula (11). The standard deviation within time step T; when for When, it represents orthogonal components. The corresponding momentum coefficient; at this time, for This indicates that the parallel components are calculated using formula (11). At time step Within the standard deviation; c) Update the momentum coefficient , or Update the labeled prediction loss gradient using exponential moving average. , parallel component and orthogonal components Independent momentum buffer variables are used to obtain the labeled predictive loss gradient momentum. Parallel component momentum buffer variable and orthogonal component momentum buffer variable Mark the predicted loss gradient momentum Parallel component momentum buffer variable and orthogonal component momentum buffer variable The update methods are all as shown in equation (13): (13) In equation (13), This represents the labeled prediction loss gradient momentum in the (t+1)th epoch. Parallel component momentum buffer variable Or orthogonal component momentum buffer variable ; when for At this time, represents the gradient momentum of the labeled prediction loss in the (t+1)th epoch; at this time, for , indicating the labeled prediction loss gradient The corresponding momentum coefficient; for , representing the labeled prediction loss gradient momentum in the t-th epoch; for , representing the gradient of the labeled prediction loss in the (t+1)th epoch; when for At this time, represents the momentum buffer variable of the parallel component in the (t+1)th epoch; at this time, for , represents the parallel component The corresponding momentum coefficient; for , represents the momentum buffer variable of the parallel component in the t-th epoch; for , representing the parallel component in the (t+1)th epoch; when for At this time, represents the orthogonal component momentum buffer variable in the (t+1)th epoch; at this time, for , representing orthogonal components The corresponding momentum coefficient; for , represents the orthogonal component momentum buffer variable in the t-th epoch; for , representing the orthogonal component in the (t+1)th epoch; In this application, the gradients of the labeled prediction loss will be stored separately. , parallel component and orthogonal components The regions of independent momentum buffer variables are considered as different independent momentum buffers, and each independent momentum buffer is physically isolated from the others.

[0032] d) Utilize gating to fuse independent momentum buffer variables to generate the total update gradient; specifically: First, based on the standard deviation of the parallel components in the (t+1)th epoch calculated in step b), and the standard deviation of orthogonal components To calculate the normalization factor, the normalization factor is equal to the standard deviation of the parallel components in the (t+1)th epoch. Standard deviation of orthogonal components The sum; secondly, calculate the standard deviation of the parallel components in the (t+1)th epoch respectively. Standard deviation of orthogonal components in the (t+1)th epoch The ratio of the ratio to the normalization factor yields the parallel component fusion weight in the (t+1)th epoch. Weights fused with orthogonal components Parallel component fusion weights Weights fused with orthogonal components The calculation methods are shown in equations (14) and (15) respectively: (14) In equation (14), This represents the fusion weight of parallel components in the (t+1)th epoch. This represents the standard deviation of the orthogonal components in the (t+1)th epoch. This represents the standard deviation of the orthogonal components in the (t+1)th epoch; (15) In equation (15), This represents the fusion weight of the orthogonal components in the (t+1)th epoch. This represents the standard deviation of the orthogonal components in the (t+1)th epoch. This represents the standard deviation of the orthogonal components in the (t+1)th epoch; Then, the weights are fused based on the parallel components in the (t+1)th epoch. Weights fused with orthogonal components The momentum buffer variable of the parallel components in the (t+1)th epoch and orthogonal component momentum buffer variable We perform a weighted summation to obtain the fused aligned gradient momentum in the (t+1)th epoch. Align gradient momentum The calculation method is shown in equation (16): (16) In equation (16), Let represent the aligned gradient momentum at the t-th epoch. This represents the fusion weight of parallel components in the (t+1)th epoch. Let represent the momentum buffer variable of the parallel components in the (t+1)th epoch. This represents the fusion weight of the orthogonal components in the (t+1)th epoch. This represents the orthogonal component momentum buffer variable in the (t+1)th epoch; Finally, the loss gradient momentum is predicted based on the labels in the (t+1)th epoch. Aligning gradient momentum and the label prediction dynamic weights in the t-th epoch Distribution Alignment Dynamic Weights Generate the total update gradient in the (t+1)th epoch. Using the SGD optimizer based on the total update gradient The learnable network parameters are updated as shown in equation (17): (17) In this application, steps a) to d) complete the adaptive update process of the learnable network parameters. This application refers to the content described in steps a) to d) as the gradient decomposition multi-momentum allocation mechanism. This mechanism effectively reduces the mutual interference between the distribution alignment task and the label prediction task. The gradient decomposition multi-momentum allocation mechanism significantly improves the stability and convergence of imbalanced label distribution learning, enabling the imbalanced label distribution learning network to obtain better model parameters. Therefore, this application can obtain a predicted face attractiveness rating label distribution that more closely approximates the real face attractiveness rating label distribution based on the imbalanced label distribution learning network model.

[0033] The gradient decomposition multi-momentum allocation mechanism of this application can be implemented in mainstream deep learning frameworks (such as PyTorch or TensorFlow). Specifically, in the SGD optimizer, independent momentum buffer variables and momentum coefficients are configured for labeling the predicted loss gradient, parallel components, and orthogonal components.

[0034] S3. Based on the imbalanced dataset with an imbalanced distribution of the facial attractiveness rating labels to be predicted, obtain the facial image feature vectors. Then, input the facial image feature vectors into the learning network model of the 10 imbalanced label distributions obtained in step S3 and propagate it forward once to output 10 predicted facial attractiveness rating label distributions. The method of obtaining the facial image feature vectors based on the imbalanced dataset to be predicted is consistent with the feature construction method in the SCUT-FBP dataset, that is, extract the corresponding facial image feature vector for each facial image.

[0035] The predicted face attractiveness rating label distribution output in this application includes information from multiple annotators who rated the attractiveness of each face image according to discrete rating levels. Therefore, existing face attractiveness evaluation systems can evaluate face attractiveness based on the aforementioned 10 predicted face attractiveness rating label distributions. Since the predicted face attractiveness rating label distribution output in step S3 of this invention is closer to the actual face attractiveness rating label distribution, the face attractiveness evaluation results obtained based on this predicted face attractiveness rating label distribution are more consistent with the group's subjective perception, resulting in higher evaluation accuracy.

[0036] Test 1: The method described in this application (referred to as Ours method in Table 1) is compared with existing methods such as SA-BFGS (from Xin Geng's paper "Label Distribution Learning"), EDLRL (from Xiuyi Jia et al.'s paper "Facial Emotion Distribution Learning by Exploiting Low-Rank Label Correlations Locally"), LDLSF (from Tingting Ren et al.'s paper "Label Distribution Learning with Label-Specific Features"), LDL-LCLR (from Tingting Ren et al.'s paper "Label Distribution Learning with Label Correlations via Low-Rank Approximation"), Adam-LDL-SCL (from Xiuyi Jia et al.'s paper "Label Distribution Learning with Label Correlations on Local Samples"), LDL-LDM (from Jing Wang et al.'s paper "Label Distribution Learning by Exploiting Label Distribution Manifold"), and OFR-FL (from Tsung-Yi Lin et al.'s paper "Focal Loss for Dense Object"). The methods mentioned include OFR-CB (from the paper "Focal Loss for Dense Object Detection" by Tsung-Yi Lin et al.), OFR-DB (from the paper "Focal Loss for Dense Object Detection" by Tsung-Yi Lin et al.), RDA (from the paper "Imbalanced Label Distribution Learning" by Xingyu Zhao et al.), DILDL (from the unbalanced label distribution learning method based on decoupling operation disclosed in patent ZL202510450586.X), and MO4ILDL (from patent ZLCN202510883930).The publicly disclosed methods for learning unbalanced distributions (such as the one based on asymmetric momentum optimization for learning unbalanced labeled distributions) were tested using the same testing strategy, and the test results are shown in Table 1.

[0037] The testing strategy adopted in this application is as follows: tests are conducted on the test set and the corresponding imbalanced label distribution learning network model in 10 training segments. The KL distance value between the predicted face attractiveness score label distribution and the real face attractiveness score label distribution output by each imbalanced label distribution learning network model is calculated based on the KL distance method. Then, these KL distances are averaged to obtain the KL distance average value. Then, the absolute value of the difference between each KL distance value and the KL distance average value is calculated, and the maximum absolute value is the fluctuation error. The fluctuation error is added to the KL distance average value to obtain the final error. This final error can show the difference between the predicted face attractiveness score label distribution and the real face attractiveness score label distribution.

[0038] Table 1. Final errors of different methods on the SCUT-FBP dataset

[0039] As shown in Table 1, the method described in this invention performs better on the SCUT-FBP dataset. The final error obtained by the method described in this application is 0.3887±0.0449, which is 5.22% lower than the final error of the MO4ILDL method (0.4101±0.0742). This result indicates that the method described in this application outperforms the MO4ILDL method in obtaining the predicted face attractiveness score label distribution on the SCUT-FBP dataset, and the predicted face attractiveness score label distribution obtained by the method described in this application on the SCUT-FBP dataset is closer to the actual face attractiveness score label distribution.

[0040] To verify the contribution of the gradient decomposition multi-momentum allocation mechanism and the gradient norm-based dynamic weighting mechanism in this application to the method described herein, targeted ablation experiments were specifically conducted on the SCUT-FBP dataset. Six evaluation metrics—Chebyshev distance, Clark distance, Canberra measure, KL divergence, cosine coefficient, and Intersection similarity—were used to evaluate the distance or similarity between the real and predicted face attractiveness rating label distributions. These six metrics were used as performance indicators for the imbalanced label distribution learning network model. The test results are shown in Table 2. Table 2 Ablation test results

[0041] In Table 2, smaller values ​​for Chebyshev distance, Clark distance, Canberra measure, and KL divergence indicate better performance; larger values ​​for cosine coefficient and intersection similarity indicate better performance.

[0042] As shown in Table 2, when both the gradient norm-based dynamic weighting mechanism and the gradient decomposition multi-momentum allocation mechanism are introduced simultaneously, the Chebyshev distance, Clark distance, Canberra measure, and KL divergence obtained by the method described in this application are significantly reduced. This indicates that the predicted face attractiveness rating label distribution obtained by the method described in this application is closer to the real face attractiveness rating label distribution. Simultaneously, the cosine coefficient and intersection similarity obtained by the method described in this application are significantly improved, which also indicates that the predicted face attractiveness rating label distribution obtained by the method described in this application is closer to the real face attractiveness rating label distribution. In summary, the method described in this application can more effectively learn the imbalanced label distribution, making the predicted face attractiveness rating label distribution closer to the real face attractiveness rating label distribution.

[0043] For example, as can be seen from Table 2, when both the gradient decomposition multi-momentum allocation mechanism and the gradient norm-based dynamic weighting mechanism are introduced simultaneously, the KL divergence index test result obtained by the method described in this application is 0.389.

[0044] Table 2 also shows that when the gradient decomposition multi-momentum allocation mechanism is omitted and only the gradient norm-based dynamic weighting mechanism is used, the KL divergence index test result is 0.405. Clearly, the KL divergence index increases by 4.11% when using only the gradient norm-based dynamic weighting mechanism compared to the method in this application (which uses both the gradient decomposition multi-momentum allocation mechanism and the gradient norm-based dynamic weighting mechanism). Table 2 also shows that compared to the method in this application (which uses both the gradient decomposition multi-momentum allocation mechanism and the gradient norm-based dynamic weighting mechanism), when the gradient decomposition multi-momentum allocation mechanism is omitted and only the gradient norm-based dynamic weighting mechanism is used, the Chebyshev distance increases from 0.176 to 0.190, an increase of 7.95%; the Clark distance increases from 0.678 to 0.705, an increase of 3.98%; and the Canberra measure increases from 1.108... The index rose to 1.312, an increase of 18.41%; while the cosine coefficient decreased from 0.875 to 0.861, a decrease of 1.60%; and the intersection similarity decreased from 0.772 to 0.750, a decrease of 2.85%. This indicates that the method described in this application, which simultaneously introduces a gradient decomposition multi-momentum allocation mechanism and a gradient norm-based dynamic weighting mechanism, yields a predicted face attractiveness score label distribution that is closer to the actual face attractiveness score label distribution than the case where the gradient decomposition multi-momentum allocation mechanism is omitted and only adaptive momentum allocation is used.

[0045] As can be seen from Table 2, compared with the method of this application, when omitting the gradient norm-based dynamic weighting mechanism and only using the gradient decomposition multi-momentum allocation mechanism, the obtained KL divergence index increased from 0.388 to 0.401, an increase of 3.35%; Chebyshev distance increased from 0.176 to 0.184, an increase of 4.55%; Clark distance increased from 0.678 to 0.698, an increase of 2.95%; Canberra measure increased from 1.108 to 1.228, an increase of 10.83%; while the cosine coefficient decreased from 0.875 to 0.866, a decrease of 1.03%; and the intersection similarity decreased from 0.772 to 0.754, a decrease of 2.33%. This indicates that the method described in this application, which simultaneously introduces a gradient decomposition multi-momentum allocation mechanism and a gradient norm-based dynamic weighting mechanism, yields a predicted face attractiveness score label distribution that is closer to the actual face attractiveness score label distribution than the case where the gradient norm-based dynamic weighting mechanism is omitted and only the gradient decomposition multi-momentum allocation mechanism is used.

[0046] In other words, this application can only obtain a predicted face attractiveness score label distribution that is closer to the real face attractiveness score label distribution by combining the gradient decomposition multi-momentum allocation mechanism and the gradient norm-based dynamic weighting mechanism.

[0047] Test 2: To objectively verify the effectiveness of the method described in this application compared with the predicted face attractiveness score label distributions obtained by the twelve existing imbalanced label distribution learning methods in Table 1 when evaluating face attractiveness, Test 2 of this application was conducted according to the following steps: 1) The standard image preprocessing module in the open-source face feature analysis toolkit OpenFace (open-source URL: https: / / github.com / TadasBaltrusaitis / OpenFace) is used to automatically process face images in the test set of the SCUT-FBP dataset, completing face detection, key point localization, and pose alignment to obtain preprocessed images; this step aims to eliminate interference from non-target variables introduced by differences in pose, scale, and position in the original images; 2) The global label distribution of face attractiveness scores inherent in the SCUT-FBP dataset and corresponding to the preprocessed images is used as the benchmark ground value for face attractiveness evaluation; 3) Input the preprocessed images into the models trained by the method of this application and the twelve comparison methods listed in Table 1, respectively, to obtain the predicted face attractiveness score label distribution output by each method; 4) A consistent mapping operation is used for the global label distribution of facial attractiveness scores and the predicted facial attractiveness score label distribution output by each method to ensure a fair conversion from the score label distribution to a sortable scalar. The mapping operation specifically includes the following steps: 4-1) The attractiveness rating level of the preprocessed image corresponding to the global label distribution of the face attractiveness rating and its corresponding descriptive degree are weighted and summed to calculate the ground expected face attractiveness rating of the preprocessed image. 4-2) The attractiveness rating level of the preprocessed image corresponding to the predicted face attractiveness rating label distribution output by each method and its corresponding descriptive degree are weighted and summed to calculate the predicted expected face attractiveness rating corresponding to the preprocessed image. 5) Sort the true expected face attractiveness scores in descending order to obtain the baseline ranking list of face attractiveness; sort the predicted expected face attractiveness scores in descending order to obtain the predicted ranking list of face attractiveness. 6) The Surprise framework (open source URL: https: / / github.com / NicolasHug / Surprise) was used to conduct offline evaluations of the benchmark ranking list of facial attractiveness and the predicted ranking list of facial attractiveness, respectively. The test results are shown in Table 3.

[0048] The test results were comprehensively evaluated using metrics such as hit rate (HR@10), normalized cumulative gain (NDCG@10), mean average accuracy (MAP@10), mean reciprocal ranking (MRR), and coverage. Higher values ​​for HR@10, NDCG@10, MAP@10, MRR, and Coverage indicate better system performance.

[0049] In this application, HR@10 represents the proportion of real, highly attractive faces in the test set that were successfully matched among the top 10 face image feature vector samples in the ranking results. NDCG@10 is used to measure the ranking quality of the top 10 face attractiveness ranking results. Specifically, NDCG@10 represents the normalized depreciation cumulative gain of the test set's real high attractive faces being successfully matched in the top 10 recommended face image feature vector samples. MAP@10 represents the average accuracy of the top 10 face attractiveness ranking results; specifically, MAP@10 represents the average accuracy of successfully matching real high-attractive faces in the test set among the top 10 recommended face image feature vector samples. MRR represents the average of the reciprocals of the first occurrence positions of the face image feature vector samples associated with the true expected face attractiveness score in the predicted ranking list; Coverage measures the proportion of face image feature vector samples that are covered at least once in all the top 10 ranking results out of the total number of face image feature vector samples in the test set.

[0050] Table 3 shows the test results of the Surprise framework for offline evaluation of the baseline and predicted ranking lists.

[0051] As can be seen from Table 3, when the predicted face attractiveness score label distribution obtained by the method described in this application is used to evaluate face attractiveness, the offline evaluation results of the predicted face attractiveness ranking list are significantly better than the offline evaluation results of the predicted face attractiveness ranking list obtained by the twelve existing imbalanced label distribution learning methods described in Table 3.

[0052] For example, as shown in Table 3, the predicted face attractiveness score label distribution obtained by the method described in this application achieves an HR@10 index of 0.64 when evaluating face attractiveness. Compared with the best method (i.e., MO4ILDL method) among the twelve existing imbalanced label distribution learning methods in Table 3, the predicted face attractiveness score label distribution obtained when evaluating face attractiveness has an HR@10 index (0.60) that is improved by about 6.7%. This indicates that, under the same test protocol and dataset conditions, when the method described in this application evaluates face attractiveness based on the predicted face attractiveness score label distribution obtained from the test set, the probability of successfully hitting the real high-attractiveness face image feature vector samples in the top 10 face image feature vector samples in the ranking results is higher. The predicted face attractiveness score label distribution obtained by the method described in this application yields an NDCG@10 index of 0.55 when evaluating face attractiveness; compared to the predicted face attractiveness score label distribution obtained by the MO4ILDL method (0.51), the NDCG@10 index is improved by approximately 7.8%. This indicates that, under the same testing protocol and dataset conditions, when evaluating face attractiveness based on the predicted face attractiveness score label distribution obtained by the method described in this application from the test set, the normalized loss cumulative gain is higher when the feature vector samples of real high-attractiveness face images in the test set are successfully matched in the top 10 of the ranking.

[0053] The predicted face attractiveness score label distribution obtained by the method described in this application yields a MAP@10 index of 0.42 when evaluating face attractiveness; compared with the predicted face attractiveness score label distribution obtained by the MO4ILDL method (0.39), the MAP@10 index is improved by approximately 7.7%; this indicates that, under the same testing protocol and dataset conditions, when evaluating face attractiveness based on the predicted face attractiveness score label distribution obtained by the method described in this application on the test set, the average accuracy of successfully hitting the feature vector samples of real high-attractiveness face images in the test set is higher among the top 10 ranked faces.

[0054] The predicted face attractiveness score label distribution obtained by the method described in this application achieves an MRR of 0.53 when evaluating face attractiveness, which is approximately 8.2% higher than the MRR of 0.49 obtained by the predicted face attractiveness score label distribution obtained by the MO4ILDL method. This indicates that, under the same testing protocol and dataset conditions, the method described in this application, when evaluating face attractiveness based on the predicted face attractiveness score label distribution obtained from the test set, can place the feature vector samples of real high-attractiveness face images at a higher position in the ranking list, thereby effectively improving the hit rate and ranking quality of the top 10 face attractiveness ranking results, and enhancing the accuracy and usability of the face attractiveness ranking results generated based on the predicted face attractiveness score label distribution. Furthermore, the predicted face attractiveness score label distribution obtained by the method described in this application yields an attractiveness coverage index of 0.38 when evaluating face attractiveness; compared to the predicted face attractiveness score label distribution obtained by the MO4ILDL method, the coverage index (0.34) is improved by approximately 11.8% when evaluating face attractiveness. This indicates that, under the same testing protocol and dataset conditions, when evaluating face attractiveness based on the predicted face attractiveness score label distribution, the method described in this application achieves a higher proportion of face image feature vector samples that are covered at least once in all the top 10 ranking results, out of the total number of face image feature vector samples in the test set.

[0055] In summary, the test results obtained from Test 2 based on the test set show that, under the same test protocol and dataset conditions, the predicted face attractiveness score label distribution obtained by the method described in this application significantly improves the HR@10, NDCG@10, MAP@10, MRR, and attractiveness coverage indices when evaluating face attractiveness. Clearly, the method described in this application, when evaluating face attractiveness based on the predicted face attractiveness score label distribution obtained from the test set, can balance the accuracy of face attractiveness ranking and directory coverage, making the face attractiveness evaluation results more consistent with the evaluation effect of group cognition.

Claims

1. An imbalanced labeled distribution learning method suitable for facial attractiveness evaluation, characterized in that: Includes the following steps: S1. Construct an imbalanced label distribution learning network that includes a feature encoder, a fully labeled encoder, and a label prediction decoder; A feature encoder is used to obtain feature representations; The full-label encoder is used to obtain the global label representation; the label prediction decoder obtains the distribution of real and predicted face attractiveness score labels based on the global label representation and feature representation, respectively. S2. Train the network based on the training set and the total optimization loss to obtain an imbalanced label distribution learning network model; During training, a gradient decomposition multi-momentum allocation mechanism is used to adaptively update the learnable parameters during backpropagation, including the following steps: decomposing the distribution-aligned loss gradient into parallel and orthogonal components relative to the label prediction loss gradient. Independent momentum buffer variables are maintained for the labeled prediction loss gradient, parallel components, and orthogonal components respectively. The independent momentum buffer variables are fused by gating to generate the total update gradient, and then the learnable network parameters are updated based on the total update gradient. S3. Based on the imbalanced dataset to be predicted, obtain the feature vector of the face image, input it into the forward propagation of the imbalanced label distribution learning network model, and obtain the predicted face attractiveness score label distribution.

2. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 1, characterized in that: In step S2, the training set is obtained based on the SCUT-FBP dataset. In the SCUT-FBP dataset, the rating label distribution of each face image is text data. The rating label distribution of each face image is obtained by statistically analyzing the voting ratio of all users at each rating level. The rating label distribution is the global label distribution of face attractiveness rating.

3. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 1, characterized in that: In step S2, the total optimization loss of the imbalanced label distribution learning network includes label prediction loss, distribution alignment loss, label prediction dynamic weights, and distribution alignment dynamic weights.

4. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 3, characterized in that: In step S2, the labeled prediction dynamic weights and distribution-aligned dynamic weights are dynamically generated using the gradient norm.

5. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 4, characterized in that: The method for obtaining the label prediction dynamic weights and distribution alignment dynamic weights in the t-th epoch includes the following steps: Step A: Calculate the gradient norm of the label prediction loss and the gradient norm of the distribution alignment loss in the t-th epoch; Step B: Calculate the sum of gradient norms in the t-th epoch; Step C: Obtain the label prediction dynamic weights and distribution alignment dynamic weights in the t-th epoch based on the sum of gradient norms Sum(t) in the t-th epoch.

6. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 5, characterized in that: In step C, the sum of the label prediction dynamic weights and the distribution alignment dynamic weights in the t-th epoch is equal to 1.

7. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 1, characterized in that: In step S2, before adaptively updating the learnable parameters using the gradient decomposition multi-momentum allocation mechanism, the label prediction loss gradient and distribution alignment loss gradient in the current Epoch are calculated, and the label prediction dynamic weights and distribution alignment dynamic weights are obtained based on the gradient norms of the label prediction loss gradient and the distribution alignment loss gradient.

8. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 7, characterized in that: In step S2, independent momentum buffer variables, parallel components, and orthogonal components are assigned to the labeled predicted loss gradient, parallel components, and orthogonal components respectively; specifically Includes the following steps: First, the standard deviations of the labeled prediction loss gradient, parallel component, and orthogonal component within time step T are calculated using a sliding window. Then, the momentum coefficients corresponding to the labeled prediction loss gradient, parallel components, and orthogonal components are adaptively updated based on the standard deviation of the labeled prediction loss gradient, parallel components, and orthogonal components within time step T.

9. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 1, characterized in that: In step S2, the independent momentum buffer variables are fused using gating to generate the total update gradient, which specifically includes the following steps: First, a normalization factor is calculated based on the standard deviations of the parallel components and the orthogonal components. Second, the ratios of the standard deviations of the parallel components and the orthogonal components to the normalization factor are calculated respectively to obtain the fusion weights of the parallel components and the fusion weights of the orthogonal components. Then, the momentum buffer variables of the parallel components and the momentum buffer variables of the orthogonal components are weighted and summed based on the fusion weights of the parallel components and the fusion weights of the orthogonal components to obtain the fused aligned gradient momentum. Finally, based on the labeled prediction loss gradient momentum, the fused alignment gradient momentum, and the labeled prediction dynamic weights and distribution alignment dynamic weights, the total update gradient for the current epoch is generated.

10. The imbalanced labeled distribution learning method for facial attractiveness evaluation according to claim 1, characterized in that: Step S2 includes the following steps: inputting the face image feature vectors and the global label distribution of face attractiveness scores from the training set into the feature encoder and the fully labeled encoder, respectively; calculating the total optimization loss, and performing backpropagation under the guidance of the total optimization loss; during the backpropagation process, first calculating the label prediction loss gradient and the distribution alignment loss gradient in the current epoch, and obtaining the label prediction dynamic weights and the distribution alignment dynamic weights based on the gradient norms of the label prediction loss gradient and the distribution alignment loss gradient; then, using the gradient decomposition multi-momentum allocation mechanism to adaptively update the learnable parameters; after iterating for 300 epochs, completing the training of one training segment, and obtaining an imbalanced label distribution learning network model; training the imbalanced label distribution learning network for 10 training segments, obtaining 10 imbalanced label distribution learning network models.

Images