Arrayed face beauty prediction method, device and storage medium

Abstract

The application discloses an array type face beauty prediction method and device and a storage medium. The method comprises the following steps: extracting multiple different scale face beauty features from a face image through multiple feature extractors; array type fusing the multiple different scale face beauty features to obtain multiple fused features; performing multiple binary classification processing on the multiple fused features through a face beauty classification network to obtain multiple classification results, wherein the face beauty classification network is obtained through supervised training of a cost-sensitive loss function, and the cost-sensitive loss function is a loss function set according to cost-sensitive training labels; and combining the multiple classification results to make a decision and obtain a face beauty prediction result. The application can strengthen information supervision through multi-scale face beauty feature extraction and array type fusion, and can effectively reduce the average cost of classification errors through optimization of the cost-sensitive loss function, thereby improving the accuracy of classification prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image data processing, in particular to an array type face beauty prediction method, device and storage medium. BACKGROUND

[0002] Face beauty prediction is to use machine learning method to intelligently predict the degree of beauty according to the aesthetic features of different face images, so as to make the machine have similar face beauty perception ability to human.

[0003] In the related art, face beauty prediction is to extract features from face images, and then predict according to the features to obtain a prediction result. However, the current face beauty prediction has the problems of insufficient feature extraction ability and insufficient prediction accuracy. SUMMARY

[0004] The present application aims to at least solve one of the problems in the prior art. To this end, the present application provides an array type face beauty prediction method, device and storage medium, which has strong feature extraction ability and accurate face beauty prediction result.

[0005] The first aspect of the present application provides an array type face beauty prediction method, comprising the following steps:

[0006] extracting multiple face beauty features of different scales from the face image through multiple feature extractors;

[0007] array type fusion of the multiple face beauty features of different scales to obtain multiple fusion features;

[0008] multiple binary classification processing of the multiple fusion features through a face beauty classification network to obtain multiple classification results, wherein the face beauty classification network is obtained through supervision and correction by a cost-sensitive loss function, and the cost-sensitive loss function is a loss function set according to cost-sensitive training labels;

[0009] decision making in combination with the multiple classification results to obtain a face beauty prediction result.

[0010] According to the above embodiments of the present application, at least the following advantages are achieved: through the multi-scale facial beauty feature extraction of the facial image by the plurality of feature extractors, the feature extraction capability can be effectively improved, comprehensive feature data can be provided for subsequent prediction work, and through the array fusion of facial beauty features of different scales, the information supervision effect can be strengthened, and the fitting performance of the model can be improved, and through the cost-sensitive loss function, the facial beauty classification network can be optimized to effectively reduce the average cost of classification errors, reduce the influence of unbalanced data samples used for training on the facial beauty classification network, thereby improving the classification prediction effect, and through the integrated decision-making of the classification results of each binary classification task, the optimal facial beauty prediction result can be obtained by comprehensively analyzing the classification results of each binary classification task, thereby improving the accuracy of the facial beauty prediction result.

[0011] According to some embodiments of the first aspect of the present application, the plurality of different scale facial beauty features are extracted from the facial image by the plurality of feature extractors, including:

[0012] Three feature extractors are constructed respectively by using a convolutional neural network, a width learning system, and a transformer model.

[0013] The facial image is feature-extracted by the three feature extractors respectively to obtain three different scale facial beauty features.

[0014] According to some embodiments of the first aspect of the present application, the plurality of different scale facial beauty features are array-fused to obtain a plurality of fusion features, including:

[0015] The plurality of scale facial beauty features are array-distributed to obtain a feature array.

[0016] Each two facial beauty features in the feature array are fused to obtain a plurality of fusion features.

[0017] According to some embodiments of the first aspect of the present application, after each two facial beauty features in the feature array are fused to obtain a plurality of fusion features, the method further includes:

[0018] The plurality of fusion features are fused to obtain secondary fusion features, wherein the secondary fusion features are used as input to the facial beauty classification network for binary classification processing to obtain corresponding classification results.

[0019] According to some embodiments of the first aspect of the present application, the training method of the facial beauty classification network includes:

[0020] The facial training set is input to the facial beauty classification network, wherein the facial training set includes a plurality of groups of corresponding facial training images and beauty level training labels, and the beauty level training labels have a plurality of dimensions.

[0021] The various binary classification tasks in the face beauty classification network classify the face training images to obtain the classification training results;

[0022] Supervised training is performed on each binary classification task based on each dimension of the beauty level training label, and the trained face beauty classification network is obtained by adjusting the parameters of the binary classification task through a cost-sensitive loss function.

[0023] According to some embodiments of the first aspect of the present invention, before performing supervised training on each binary classification task based on each dimension of the beauty rating training labels, the method includes:

[0024] By jointly debugging and adjusting each binary classification task, feature sharing between the various binary classification tasks can be achieved.

[0025] According to some embodiments of the first aspect of the present invention, supervised training is performed on each binary classification task based on each dimension of the beauty level training label, and a trained face beauty classification network is obtained after parameter tuning of the binary classification task using a cost-sensitive loss function, including:

[0026] When the face training set is a difficult sample, the shared features between the various binary classification tasks are kept unchanged. Each binary classification task is supervised and trained according to each dimension of the beauty level training label. The parameters of the binary classification task are adjusted by the cost-sensitive loss function to obtain a well-trained face beauty classification network.

[0027] According to some embodiments of the first aspect of the present invention, after training the face beauty classification network, testing is also performed. The testing method for the face beauty classification network includes:

[0028] The face test set is input into the face beauty classification network, where the face test set includes face test images and beauty level test labels;

[0029] Based on the beauty level test labels, an error judgment is made for each category result, and the error result is obtained;

[0030] Based on the error results, the corresponding binary classification task is corrected to obtain the face beauty classification network that has completed the test.

[0031] A second aspect of the present invention provides an electronic device, comprising:

[0032] The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the array-based face beauty prediction method of any one of the first aspects.

[0033] Since the electronic device of the second aspect embodiment applies the array-based face beauty prediction method of any one of the first aspects, it has all the beneficial effects of the first aspect of the present invention.

[0034] According to a third aspect of the present invention, a computer storage medium is provided, storing computer-executable instructions for executing the array-based face beauty prediction method of any one of the first aspects.

[0035] Since the computer storage medium of the third aspect embodiment can execute the array-based face beauty prediction method of any one of the first aspects, it has all the beneficial effects of the first aspect of the present invention.

[0036] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0037] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0038] Figure 1 This is a diagram showing the main steps of the array-based face beauty prediction method according to an embodiment of the present invention;

[0039] Figure 2 yes Figure 1 A schematic diagram illustrating the specific steps of step S100;

[0040] Figure 3 yes Figure 1 A schematic diagram illustrating the specific steps of step S200;

[0041] Figure 4 This is a schematic diagram of the training steps of the face beauty classification network in the array-based face beauty prediction method of this invention.

[0042] Figure 5 This is a schematic diagram of the test steps of the face beauty classification network in the array-based face beauty prediction method of this invention.

[0043] Figure 6 This is a schematic diagram of the structure of the face beauty prediction network model corresponding to the array-based face beauty prediction method in this embodiment of the invention. Detailed Implementation

[0044] In the description of this invention, unless otherwise explicitly defined, terms such as "setting," "installing," and "connecting" should be interpreted broadly. Those skilled in the art can reasonably determine the specific meaning of these terms in this invention based on the specific content of the technical solution. In the description of this invention, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. Furthermore, features specified as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "multiple" means two or more.

[0045] Facial beauty prediction is a cutting-edge topic in the fields of facial learning and computer vision. It involves using machine learning methods to intelligently predict the beauty level of different facial images based on their aesthetic features, thereby enabling machines to possess facial beauty perception intelligence similar to humans. Currently, facial beauty prediction faces challenges such as insufficient supervised information, imbalanced data samples, and the tendency for models to overfit.

[0046] In related technologies, facial beauty prediction involves extracting features from a facial image and then performing corresponding tasks based on those features to obtain a prediction result. Currently, facial beauty prediction suffers from insufficient feature extraction capabilities and inadequate prediction accuracy.

[0047] The lack of a large-scale, effective facial beauty database and the insufficient feature extraction capabilities of the network models used for feature extraction lead to problems such as insufficient supervision information and model overfitting in facial beauty prediction. Furthermore, facial beauty prediction typically uses error rate or accuracy as evaluation metrics, resulting in a uniform cost for predicting each class of samples. In reality, the costs of incorrectly classifying a sample from one class as another are different. For example, in cancer diagnosis, predicting a cancer patient as healthy causes them to miss the optimal treatment window, a cost significantly different from predicting a healthy person as a cancer patient.

[0048] Therefore, in the process of model building, it is necessary to not only focus on the accuracy of the results, but also on the average cost of prediction errors. In the field of facial beauty prediction, since there are more ordinary people than extremely attractive and extremely unattractive people in reality, this leads to an imbalance in the training data samples. The classifier performs well when classifying the category corresponding to the majority of the data, but performs poorly when classifying the category corresponding to the minority of the data.

[0049] The following reference Figures 1 to 6 The present invention describes an array-based face beauty prediction method, device, and storage medium, which has strong feature extraction capabilities and provides accurate and effective face beauty prediction results.

[0050] refer to Figure 1 As shown, an array-based face beauty prediction method according to a first aspect of the present invention includes, but is not limited to, the following steps:

[0051] S100: Extracts multiple facial beauty features at different scales from a face image using multiple feature extractors;

[0052] S200: Multiple facial beauty features of different scales are fused in an array to obtain multiple fused features;

[0053] S300: Multiple fusion features are subjected to multiple binary classification processes through a face beauty classification network to obtain multiple classification results. The face beauty classification network is obtained through supervision and correction by a cost-sensitive loss function. The cost-sensitive loss function is a loss function set according to the cost-sensitive training labels. The cost-sensitive function is used to minimize the average cost when the classification result is wrong.

[0054] S400: Make a decision by combining multiple classification results to obtain a face beauty prediction result. The ensemble decision is used to make a decision by integrating multiple classification results to obtain a face beauty prediction result.

[0055] Extracting facial beauty features from face images at multiple scales using multiple feature extractors effectively improves feature extraction capabilities and provides comprehensive feature data for subsequent prediction. Furthermore, array-based fusion of facial beauty features at different scales enhances information supervision and improves model fitting performance. Optimizing the facial beauty classification network using a cost-sensitive loss function effectively reduces the average cost of classification errors and mitigates the impact of imbalanced training data on the network, thereby improving classification prediction performance. Finally, ensemble decision-making on the classification results of various binary classification tasks allows for the comprehensive analysis of these results to obtain the optimal facial beauty prediction result, thus improving the accuracy of facial beauty prediction.

[0056] It should be noted that the ensemble decision can be set up to vote on the results of each classification in the form of a vote, and output the final face beauty prediction result.

[0057] Understandably, reference Figure 2 As shown, in step S100, multiple facial beauty features at different scales are extracted from the face image using multiple feature extractors, including but not limited to the following steps:

[0058] S110: Construct three feature extractors using a convolutional neural network, a width learning system, and a transformer model, respectively;

[0059] S120: Three feature extractors are used to extract features from the face image to obtain three different scales of facial beauty features.

[0060] It should be noted that, in addition to constructing three feature extractors using convolutional neural networks, a width learning system, and a transformer model, other different network models can be used to construct a corresponding number of feature extractors for extracting features from face images at different scales. This improves feature extraction capabilities, enhances information supervision, and reduces the probability of overfitting. Convolutional neural networks are a type of feedforward neural network that includes convolutional computations and has a deep structure; they are one of the representative algorithms of deep learning. A width learning system is a neural network structure that does not rely on a deep structure; its structure has no coupling between layers, making it very simple. The transformer model is a self-attention network model.

[0061] Understandably, reference Figure 3 As shown, in step S200, multiple facial beauty features of different scales are fused in an array to obtain multiple fused features, including but not limited to the following steps:

[0062] S210: Distribute facial beauty features at multiple scales in an array to obtain a feature array;

[0063] S220: Fuse the facial beauty features of every two people in the feature array to obtain multiple fused features. The aforementioned feature fusion is to fuse the facial beauty features in the feature array in pairs. When fusion is performed on the array-arranged facial beauty features, array-style feature fusion can be easily performed.

[0064] Facial beauty features are distributed in an array, as shown in the feature array below:

[0065] F1=ξ(x)

[0066] F2=ψ(x)

[0067] F3 = θ(x)

[0068] Where x represents a face image, ξ represents the feature extraction function of the convolutional neural network, ψ represents the feature extraction function provided by width learning, θ represents the feature extraction function of the transformer model, and F1, F2 and F3 represent the facial beauty features at the corresponding scales.

[0069] It is understandable that after step S200, that is, after fusing the facial beauty features of every two people in the feature array to obtain multiple fused features, the following steps are also included, but are not limited to:

[0070] S230: Multiple fusion features are fused to obtain secondary fusion features, which are then input into the face beauty classification network for binary classification to obtain the corresponding classification results.

[0071] Multiple fusion features are obtained by fusing facial beauty features. These fusion features are then further fused to obtain secondary fusion features. The individual fusion features and the secondary fusion features are represented as follows:

[0072] F a =F1+F2

[0073] F b =F1+F3

[0074] F c =F2+F3

[0075] F sum =F a +F b +F c

[0076] Among them, F a F b and F c F represents the fused feature obtained by fusing three facial beauty features in two pairs. sum This represents the secondary fusion feature resulting from the fusion of various fusion features.

[0077] Understandably, the face beauty classification network uses both fused features and secondary fused features for face beauty classification. By fusing multi-scale face beauty features in an array and inputting the fused features into the face beauty classification network for classification prediction, it can effectively solve problems such as insufficient supervision information and easy overfitting of the model in face beauty prediction.

[0078] Understandably, reference Figure 4 As shown, the training method for the face beauty classification network includes, but is not limited to, the following steps:

[0079] S301: Input the face training set into the face beauty classification network. The face training set includes multiple sets of corresponding face training images and beauty level training labels. The beauty level training labels have multiple dimensions.

[0080] S302: Each binary classification task in the face beauty classification network classifies the face training images to obtain the classification training results. Among them, the binary classification task is used to perform the corresponding binary classification processing.

[0081] S303: Supervised training is performed on each binary classification task based on each dimension of the beauty level training label, and the trained face beauty classification network is obtained after adjusting the parameters of the binary classification task through the sensitive cost loss function.

[0082] By inputting the face training set into the face beauty classification network for training, and by supervising the training of the face beauty classification network through a cost-sensitive loss function, a well-trained face beauty classification network can be obtained.

[0083] It should be noted that during the training process of the facial beauty classification network, the following settings are used: This represents the test set, which consists of N sets. test There are test samples, let's say Let yi ∈ {1, 2, 3, ... K}, and have a total of N. train There are training samples, among which... Let represent the i-th face training image, and yi represent the beauty level training label of the i-th training sample, used to represent the face beauty level label of the training sample. There are K levels in total. The label of the i-th face training image under the k-th dimension is represented as:

[0084]

[0085] Where, yi∈R K-1 , k∈{1,2,3…K-1}.

[0086] Using the sorting formula described above, we redefine yi as a K-1 dimensional vector, treating each dimension of the K-1 dimensional vector in yi as a label. Then, the i-th face training image... K-1 boolean labels were generated. Assuming the face classification network has K-1 tasks, all of which are binary classification tasks, then in the i-th face training image... The K-1 labels can be used to supervise the aforementioned K-1 binary classification tasks. This successfully transforms the face beauty classification task into multiple binary classification tasks.

[0087] It should be further explained that the cost-sensitive loss function introduces cost sensitivity into the loss function. The defined cost expression is:

[0088]

[0089] Where k∈{1,2,3…K-1}, after transforming the face beauty classification task into K-1 binary classification tasks, a cost-sensitive loss function is introduced for each binary classification task. The cost-sensitive loss function for the Kth binary classification task is expressed as:

[0090]

[0091] Among them, W (k) Let σ(x) represent the parameters of the shared features and task k, and let σ(x) represent the ReLU function. The cost-sensitive loss function for the binary classification task above indicates that the greater the error, the higher the cost; a correct classification incurs zero cost. This cost-sensitive loss function is used for supervised training of the binary classification task.

[0092] It is understandable that, before conducting supervised training on each binary classification task based on each dimension of the beauty level training labels, the following steps are included, but are not limited to:

[0093] By jointly debugging and adjusting each binary classification task, feature sharing between the various binary classification tasks can be achieved.

[0094] Convolutional neural networks can be used to extract shared features from the input fusion features and secondary fusion features, and various binary classification tasks can use the shared features for classification.

[0095] Specifically, joint debugging involves transforming the face beauty classification task into K-1 binary classification tasks, and then training and fine-tuning the face beauty classification network in order from 1 to K-1. Joint debugging reduces the probability of negative transfer between different binary classification tasks because the features involved in the same binary classification task are similar. Joint debugging enables feature sharing between different tasks, and the backpropagation algorithm is used to optimize the binary classification tasks, allowing them to share features.

[0096] By splitting the face beauty classification task into multiple binary classification tasks for joint debugging and optimization, the correlation between the various binary classification tasks can be preserved in the form of shared features. Furthermore, each binary classification task is more specialized, improving the generalization performance of the face beauty classification network. Using shared features also avoids negative transfer and the excessively large network structure caused by trying to accommodate multiple datasets. Introducing cost sensitivity into the loss function effectively addresses the problem of imbalanced samples in the face training set, thereby improving the accuracy of face beauty prediction.

[0097] It is understandable that supervised training is performed on each binary classification task based on each dimension of the beauty level training labels, and the trained face beauty classification network is obtained after parameter tuning of the binary classification task through the sensitive cost loss function, including but not limited to the following steps:

[0098] When the face training set is a difficult sample, the shared features between the various binary classification tasks are kept unchanged. Each binary classification task is supervised and trained according to each dimension of the beauty level training label. The parameters of the binary classification task are adjusted by the sensitive cost loss function to obtain the trained face beauty classification network. Here, the difficult sample means the training sample whose output results of K-1 binary classification tasks are contradictory.

[0099] By introducing difficult samples, the face beauty classification network can be forced to learn deeper features. By fine-tuning the binary classification task with difficult samples while keeping the shared features unchanged and only changing the parameters of the binary classification task, the probability of overfitting can be reduced, while improving the representation ability and generalization ability of the face beauty classification network.

[0100] Understandably, reference Figure 5 As shown, after training the face beauty classification network, it is also tested. The testing method for the face beauty classification network includes, but is not limited to, the following steps:

[0101] S401: Input the face test set into the face beauty classification network, where the face test set includes face test images and beauty level test labels;

[0102] S402: Based on the beauty level test label, perform error judgment on each category result and obtain the error result;

[0103] S403: Based on the error results, correct the corresponding binary classification task to obtain the face beauty classification network that has completed the test.

[0104] During the testing phase, an integrated decision-making approach is used to correct errors in individual binary classification tasks. By comprehensively considering the classification results of multiple binary classification tasks to vote on the final face beauty prediction result, the accuracy and robustness of the decision-making can be improved.

[0105] It's important to note that ensemble decision-making involves voting on the classification results of K-1 binary classification tasks, ultimately outputting a face beauty prediction result. Assuming each binary classification task has an equal probability of error, when a result doesn't belong to any label, the binary classification task with the fewest errors is used as the standard. That is, if some binary classifiers make mistakes, the task requiring the fewest modifications is corrected to obtain the face beauty classification network for testing. When a bottleneck occurs—that is, when the number of binary classification tasks requiring modification is the same—the confidence levels of each task needing modification are compared. The binary classification task with the lower confidence level is deemed to be incorrect, and it is corrected to obtain the face beauty classification network for testing, thus resolving the bottleneck problem.

[0106] Specifically, taking four-class classification as an example, the classification results of each binary classification task are integrated into a vector form. If the test label is [0,1,0], the above-mentioned ensemble decision is used as the criterion for decision-making. If a binary classification task is wrong and the result is [0,0,0] or [1,1,0], then it is necessary to compare the confidence of the first binary classification task and the second binary classification task, and select the binary classification task with lower confidence for correction. For example, if the confidence of the first binary classification task is lower, then the first binary classification task is corrected so that the 1 in its classification result is corrected to 0, and the face beauty classification network that has completed the test is obtained.

[0107] Figure 6 The structure of the face beauty prediction network model corresponding to the array-based face beauty prediction method is shown below. Figure 6 An array-based face beauty prediction method according to a first aspect embodiment of the present invention will be described:

[0108] Feature extractor 1, feature extractor 2, and feature extractor 3 were constructed using convolutional neural network, width learning system, and transformer model, respectively, to extract facial features at different scales, i.e., facial beauty features.

[0109] By fusing face features 1, face features 2, and face features 3 at different scales in pairs, we obtain fused features 1, fused features 2, and fused features 3.

[0110] The fused features 1, fused features 2, and fused features 3 are input into the face beauty classification model. The face beauty classification task of the face beauty classification network is split into multiple binary classification tasks, namely Task 1, Task 2, ..., Task K-1, and optimized through multi-task prediction learning and joint debugging. Three classification results are obtained through ensemble decision, namely Result 1, Result 2, and Result 3. Result 1, Result 2, and Result 3 are fused and input into the face beauty classification model to obtain another classification result, Result 4.

[0111] By integrating decision-making processes and voting on outcomes 1, 2, 3, and 4, the final facial beauty prediction result is obtained. Figure 6 The final result.

[0112] In addition, a second aspect of the present invention provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0113] The processor and memory can be connected via a bus or other means.

[0114] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0115] The non-transitory software program and instructions required to implement the array-based face beauty prediction method of the first aspect embodiment described above are stored in the memory. When executed by the processor, the array-based face beauty prediction method in the above embodiment is executed. For example, the method steps S100 to S400, S110 to S120, S210 and S220, S230, S301 to S303, and S401 to S403 described above are executed.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0117] Furthermore, a third aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by a processor in the above-described device embodiment, causing the processor to perform the array-based face beauty prediction method in the above-described embodiment, for example, performing the method steps S100 to S400, S110 to S120, S210 and S220, S230, S301 to S303, and S401 to S403 described above.

[0118] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0119] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0120] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. An arrayed face beauty prediction method, characterized in that, The method comprises the following steps: extracting multiple different scales of facial beauty features from a facial image through multiple feature extractors; arraying and fusing the multiple different scales of facial beauty features to obtain multiple fused features; performing multiple binary classification processes on the multiple fused features through a facial beauty classification network to obtain multiple classification results, wherein the facial beauty classification network is obtained through supervision and correction by a cost-sensitive loss function, and the cost-sensitive loss function is a loss function set according to cost-sensitive training labels; combining the multiple classification results to obtain a facial beauty prediction result; the arraying and fusing the multiple different scales of facial beauty features to obtain multiple fused features comprises: arraying and distributing the multiple scales of facial beauty features to obtain a feature array; fusing every two facial beauty features in the feature array to obtain multiple fused features; the extracting multiple different scales of facial beauty features from a facial image through multiple feature extractors comprises: respectively constructing three feature extractors in the form of a convolutional neural network, a width learning system and a transformer model; respectively extracting facial beauty features of three different scales from the facial image through the three feature extractors; after the fusing every two facial beauty features in the feature array to obtain multiple fused features, the method further comprises: fusing the multiple fused features to obtain secondary fused features, wherein the secondary fused features are used as input to the facial beauty classification network for binary classification to obtain corresponding classification results.

2. The array-based beautiful face prediction method of claim 1, wherein, The training method of the facial beauty classification network comprises: inputting a facial training set to the facial beauty classification network, wherein the facial training set comprises multiple groups of corresponding facial training images and beauty level training labels, and the beauty level training labels have multiple dimensions; performing classification on the facial training images by each binary classification task in the facial beauty classification network to obtain classification training results; supervising and training each binary classification task according to each dimension in the beauty level training labels, and adjusting the parameters of the binary classification task through a cost-sensitive loss function to obtain a trained facial beauty classification network.

3. The array-based beautiful face prediction method of claim 2, wherein, Before the supervising and training each binary classification task according to each dimension in the beauty level training labels, the method comprises: adjusting each binary classification task through joint debugging to enable feature sharing between the binary classification tasks.

4. The array-based beautiful face prediction method of claim 3, wherein, the supervising and training each binary classification task according to each dimension in the beauty level training labels, and adjusting the parameters of the binary classification task through a cost-sensitive loss function to obtain a trained facial beauty classification network comprises: When the face training set is a difficult sample, the shared features between each of the two classification tasks are kept unchanged, each of the two classification tasks is supervised trained according to each dimension of the beauty level training label, and the two classification tasks are parameter adjusted through a cost-sensitive loss function, so as to obtain the trained face beauty classification network.

5. The array-based method for predicting facial beauty according to any one of claims 2 to 4, characterized in that, After the face beauty classification network is trained, the face beauty classification network is also tested, and the testing method of the face beauty classification network comprises the following steps: inputting a face test set into the face beauty classification network, wherein the face test set comprises a face test image and a beauty level test label; judging an error result of each of the classification results according to the beauty level test label; correcting the corresponding two classification tasks according to the error result, so as to obtain the face beauty classification network that completes the test.

6. An electronic device, comprising: comprise: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the array type face beauty prediction method according to any one of claims 1 to 5.

7. A computer storage medium, characterized in that computer executable instructions are stored, and the computer executable instructions are used to execute the array type face beauty prediction method according to any one of claims 1 to 5.

Images