Face beauty prediction method and apparatus, and device and medium
By classifying and reweighting the training set of facial beauty prediction images, pseudo-labels are generated, the problem of label noise is solved, the accuracy and robustness of the model are improved, and efficient utilization of unlabeled data is achieved, and the data is avoided. Overfitting.
Patent Information
- Application Number
- PCT/CN2023/095555
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-10
- Filing Date
- 2023-05-22
- Publication Date
- 2024-11-14
AI Technical Summary
The existing face beauty prediction method requires a large amount of label data during model training, and the label quality is affected by subjective factors and tool technology of manual or machine data annotation, resulting in label noise problems and affecting model accuracy.
By classifying the training set of the beautiful predicted face images, images with noise labels and non-noise labels are obtained, and reweighted to form the target training set. The self-training model is used to generate pseudo-labels to enhance the utilization of unlabeled data. Adaptively improves model accuracy and reduces the dependence of noise labels.
Improve the classification performance and generalization ability of the model, enhance the utilization effect of unlabeled data, avoid overfitting, improve the data labeling quality and model robustness, ensure the stability and accuracy of the algorithm, and have relatively high running speed. quick.
Smart Images

Figure CN2023095555_14112024_PF_FP_ABST
Abstract
Description
Face beauty prediction method, device, equipment and medium Technical Field
[0001] The embodiments of the present application relate to, but are not limited to, the field of image recognition, and in particular to methods, devices, equipment, and media for predicting facial beauty. Background Art
[0002] Current facial beauty prediction methods typically require a large amount of labeled data for model training. Furthermore, when labeling noisy samples, label quality is affected by subjective factors, tool technology, and other factors, such as manual or machine data annotation, which can also introduce label noise. This label noise can significantly impact model accuracy and reduce facial beauty prediction performance.
[0003] Summary of the Invention
[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
[0005] The embodiments of the present application aim to solve at least one of the technical problems existing in the prior art. The embodiments of the present application provide a method, apparatus, device and medium for predicting facial beauty, which can reduce the model's dependence on noise labels and enhance the utilization of unlabeled data.
[0006] An embodiment of the first aspect of the present invention, a method for predicting facial beauty, includes:
[0007] Classify the training set of facial beauty prediction images to obtain a first image with a noise label and a second image with a non-noise label;
[0008] performing re-weighting processing on the second image;
[0009] The first image and the reweighted second image constitute a target training set, and the target training set is used to train the first model to obtain a second model;
[0010] Using the second model to perform labeling processing on the image data to obtain labeled first data and unlabeled second data;
[0011] Using the second data to train a classifier to generate third data with pseudo labels;
[0012] Training the classifier according to the first data, the second data, and the third data to obtain a target classifier;
[0013] The face beauty prediction model with the target classifier is used to perform face beauty prediction on the image to be predicted, thereby obtaining a face beauty prediction result.
[0014] In certain embodiments of the first aspect of the present invention, classifying a training set of facial beauty prediction images to obtain a first image with a noise label and a second image with a non-noise label includes:
[0015] Performing probability calculation based on the training set of the facial beauty prediction image to obtain a probability value indicating whether the facial beauty prediction image has a noise label;
[0016] Obtaining a joint distribution of the training set according to the probability values;
[0017] The training set is classified according to the joint distribution to obtain a first image with a noise label and a second image with a non-noise label.
[0018] In certain embodiments of the first aspect of the present invention, classifying the training set according to the joint distribution to obtain a first image with a noisy label and a second image with a non-noise label includes:
[0019] According to the joint distribution of the training set, the true distribution of the training set, the weight of the face beauty prediction image and the loss function, the expected risk of classification is obtained;
[0020] The training set is classified according to the expected risk to obtain a first image with a noise label and a second image with a non-noise label.
[0021] In certain embodiments of the first aspect of the present invention, the weight of the facial beauty prediction image is determined by the joint distribution and noise rate of the training set.
[0022] In certain embodiments of the first aspect of the present invention, the noise rate is a minimum value of the joint distribution of the training set within a preset range.
[0023] In certain embodiments of the first aspect of the present invention, when the value of the joint distribution of the training set is not equal to 0, the weight is non-negative; when the value of the joint distribution of the training set is equal to 0, the weight is equal to 0.
[0024] In certain embodiments of the first aspect of the present invention, the step of training the classifier according to the first data, the second data, and the third data to obtain a target classifier includes:
[0025] The classifier is trained according to the first data, the second data, and the third data until the classifier converges, and the converged classifier is used as a target classifier.
[0026] According to an embodiment of the second aspect of the present invention, a facial beauty prediction device includes:
[0027] a reweighting unit configured to: classify a training set of facial beauty prediction images to obtain a first image with a noise label and a second image with a non-noise label, reweight the second image, form a target training set with the first image and the reweighted second image, and train the first model using the target training set to obtain a second model;
[0028] a self-training unit, the self-training unit being configured to: label the image data using the second model to obtain labeled first data and unlabeled second data, train a classifier using the second data to generate pseudo-labeled third data, and train the classifier based on the first data, the second data, and the third data to obtain a target classifier;
[0029] A prediction unit is configured to: perform face beauty prediction on the image to be predicted using a face beauty prediction model having the target classifier to obtain a face beauty prediction result.
[0030] An embodiment of the third aspect of the present invention is an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-described method for predicting facial beauty when executing the computer program.
[0031] An embodiment of the fourth aspect of the present invention is a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the facial beauty prediction method as described above.
[0032] The above scheme has at least the following beneficial effects: it can utilize noisy label reweighting learning to reweight the training set, reducing the model's dependence on noisy labels and improving the model's classification performance and generalization ability. During the model self-training process, samples with uncertain classification are treated as noisy labels and filtered or reweighted, retaining only high-confidence samples for model training. This enhances the utilization of unlabeled data, adaptively improves model accuracy, and avoids overfitting. Furthermore, when making predictions using unlabeled data, high-confidence predictions are treated as new labeled data and added to the training data, thereby increasing sample utilization and improving model performance. During the iterative process, labeled data is continuously added, and the quality of data annotation gradually improves, making the model highly robust. Furthermore, the algorithm can automatically adjust the model to factors such as label anomalies or noise in the data, ensuring its stability and accuracy. The algorithm is based on a simple but efficient algorithm framework and runs relatively quickly. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.
[0034] FIG1 is a diagram showing the steps of a method for predicting facial beauty according to an embodiment of the present application;
[0035] FIG2 is a sub-step diagram of step S100;
[0036] FIG3 is a sub-step diagram of step S130;
[0037] FIG4 is a structural diagram of a facial beauty prediction device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0038] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0039] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and the like in the specification, claims, or accompanying drawings are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.
[0040] The embodiments of the present application are further described below with reference to the accompanying drawings.
[0041] The embodiments of the present application provide a method for predicting facial beauty.
[0042] Referring to FIG1 , the face beauty prediction method includes:
[0043] Step S100, classifying a training set of facial beauty prediction images to obtain a first image with a noise label and a second image with a non-noise label;
[0044] Step S200, performing re-weighting processing on the second image;
[0045] Step S300, the first image and the reweighted second image are used to form a target training set, and the target training set is used to train the first model to obtain a second model;
[0046] Step S400: labeling the image data using the second model to obtain labeled first data and unlabeled second data;
[0047] Step S500, training the classifier using the second data to generate third data with pseudo labels;
[0048] Step S600, training a classifier according to the first data, the second data, and the third data to obtain a target classifier;
[0049] Step S700: performing face beauty prediction on the image to be predicted using a face beauty prediction model with a target classifier to obtain a face beauty prediction result.
[0050] First, obtain a training set, which contains multiple face images, and use the face images as face beauty prediction images.
[0051] For example, a Large Scale Asian Facial Beauty Database (LSAFBD) can be used to obtain a training set of facial beauty prediction images.
[0052] 2 , in step S100 , the training set of facial beauty prediction images is classified to obtain a first image with a noise label and a second image with a non-noise label, which specifically includes but is not limited to the following steps:
[0053] Step S110, performing probability calculation based on the training set of facial beauty prediction images to obtain a probability value indicating whether the facial beauty prediction image has a noise label;
[0054] Step S120, obtaining the joint distribution of the training set according to the probability value;
[0055] Step S130 , classifying the training set according to the joint distribution to obtain a first image with a noise label and a second image with a non-noise label.
[0056] It is understandable that the joint distribution function (JDF) is also called the multidimensional distribution function. Taking the two-dimensional case as an example, let (X, Y) be a two-dimensional random variable, x and y be arbitrary real numbers, and the binary function: F(x, y) = P({X≤x ∩Y≤y}) = P(X≤x, Y≤y) is called the distribution function of the two-dimensional random variable (X, Y), or the joint distribution function of X and Y.
[0057] 3 , in step S130 , the training set is classified according to the joint distribution to obtain a first image with a noise label and a second image with a non-noise label, including:
[0058] Step S131, obtaining the expected risk of classification based on the joint distribution of the training set, the true distribution of the training set, the weight of the facial beauty prediction image, and the loss function;
[0059] Step S132 : Classify the training set according to the expected risk to obtain a first image with a noise label and a second image with a non-noise label.
[0060] For step S131, the expected risk of classification is obtained according to the joint distribution of the training set, the true distribution of the training set, the weight of the facial beauty prediction image and the loss function.
[0061] Furthermore, the noise label data classification method first considers that the expected risk of classification should satisfy the following relationship:
[0062] Among them, l represents the loss function, P D (X,Y) represents the true distribution of the training set, represents the joint distribution of the training set, Represents the weight of the face beauty prediction image.
[0063] Specifically, the weight of the facial beauty prediction image is determined by the joint distribution and noise rate of the training set.
[0064] in, is the weight of the face beauty prediction image.
[0065] The noise rate is the minimum value of the joint distribution of the training set within a preset range. The noise rate can be expressed by the following formula:
[0066] When the value of the joint distribution of the training set is not equal to 0, the weight is non-negative; when the value of the joint distribution of the training set is equal to 0, the weight is equal to 0.
[0067] In step S200 , a re-weighting process is performed on the second image with the non-noise label.
[0068] Generally, reweighting can be to increase the weight of the second image with a non-noise label / the first image with a noisy label, reduce the weight of the second image with a non-noise label / the first image with a noisy label, or filter the second image with a non-noise label / the first image with a noisy label; the weight of the label data is adjusted through reweighting processing.
[0069] Reweighting involves reweighting training data with noisy labels, giving more weight to correct data and improving model accuracy and robustness. Noisy labels are often generated by manual or automated annotation and can negatively impact model performance. Labeled data needs to be inspected and reweighted to minimize their impact on the model. Specific strategies should be selected based on the dataset and model requirements. For example, probabilistic modeling methods, such as deep learning, can effectively minimize the impact of noisy labels and improve model accuracy.
[0070] In addition, a filtering process is performed on the first image with the noise label.
[0071] In step S300 , the first image and the reweighted second image constitute a target training set, and the target training set is used to train the first model to obtain the second model.
[0072] The model can accurately discover the first image with noisy labels and accurately estimate the joint distribution of the first image with noisy labels and the second image with non-noisy labels.
[0073] In step S400 , the image data is labeled using the second model to obtain labeled first data and unlabeled second data.
[0074] In step S500, a classifier is trained using the second data to generate third data with pseudo labels, wherein the classifier is obtained based on the first data with labels.
[0075] For step S600, training a classifier according to the first data, the second data, and the third data to obtain a target classifier includes but is not limited to the following steps:
[0076] A classifier is trained according to the first data, the second data, and the third data until the classifier converges, and the converged classifier is used as a target classifier.
[0077] Based on the semi-supervised learning described above, the model is trained and optimized using unlabeled data. The basic idea is to first train a base model using labeled data, then use that model to make predictions on the unlabeled data and select the highly confident parts as new labeled data. The training process is repeated until the model converges or a preset number of predictions is reached. This algorithm's advantage lies in its ability to fully utilize unlabeled data, improving the model's generalization performance and making it particularly suitable for applications with limited labeled data.
[0078] In step S700 , a face beauty prediction model with a target classifier is used to perform face beauty prediction on the image to be predicted, and a face beauty prediction result is obtained.
[0079] The above-mentioned face beauty prediction method can use noise label reweighting learning to reweight the training set, reduce the model's dependence on noise labels, and improve the model's classification performance and generalization ability. During the model self-training process, samples with uncertain classification are regarded as noise labels, from which they are filtered or reweighted, and only high-confidence samples are retained for model training, thereby enhancing the utilization of unlabeled data, adaptively improving the model's accuracy, and avoiding overfitting problems.
[0080] Furthermore, when using unlabeled data for prediction, highly confident predictions are treated as newly labeled data and added to the training data, thereby increasing sample utilization and improving model performance. During the iterative process, labeled data is continuously added, and the quality of data annotation gradually improves, making the model highly robust.
[0081] Furthermore, the algorithm can automatically adjust the model to account for factors such as label anomalies or noise in the data, ensuring its stability and accuracy. Furthermore, the algorithm is based on a simple yet efficient framework and runs relatively quickly.
[0082] An embodiment of the present invention provides a facial beauty prediction device.
[0083] 4 , the facial beauty prediction apparatus includes a re-weighting unit 10 , a self-training unit 20 and a prediction unit 30 .
[0084] Among them, the reweighting unit 10 is configured to: classify the training set of facial beauty prediction images to obtain a first image with a noise label and a second image with a non-noise label, reweight the second image, and form a target training set with the first image and the reweighted second image, and use the target training set to train the first model to obtain the second model.
[0085] The self-training unit 20 is configured to: use the second model to label the image data to obtain labeled first data and unlabeled second data, use the second data to train the classifier to generate third data with pseudo labels, and train the classifier based on the first data, second data and third data to obtain a target classifier.
[0086] The prediction unit 30 is configured to perform face beauty prediction on the image to be predicted using a face beauty prediction model having a target classifier to obtain a face beauty prediction result.
[0087] It can be understood that the face beauty prediction device in this embodiment adopts the above-mentioned face beauty prediction method. The various units of the face beauty prediction device in this embodiment correspond to the steps of the above-mentioned face beauty prediction method, solve the same technical problems as the above-mentioned face beauty prediction method, and have the same beneficial effects as the above-mentioned face beauty prediction method.
[0088] An embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-described method for predicting facial beauty when executing the computer program.
[0089] The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.
[0090] In general, for the hardware structure of the electronic device, the processor can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.
[0091] The memory can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and is called by the processor to execute the methods of the embodiments of this application.
[0092] The input / output interface is used to realize information input and output.
[0093] The communication interface is used to realize the communication interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0094] The bus transmits information between the various components of the device (such as the processor, memory, input / output interface, and communication interface). The processor, memory, input / output interface, and communication interface communicate with each other within the device through the bus.
[0095] An embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions for executing the above-mentioned face beauty prediction method.
[0096] It should be appreciated that the method steps in the embodiments of the present invention can be implemented or executed by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, for this purpose, the program can be run on a programmed application-specific integrated circuit.
[0097] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that can be executed by one or more processors.
[0098] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a smartphone, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, RAM, ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted over a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.
[0099] The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data that is stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.
[0100] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.
[0101] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Technical personnel familiar with the field can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the present embodiment.
Claims
1. A method for predicting facial beauty, characterized in that: include: Classifying a training set of facial beauty prediction images to obtain a first image with a noise label and a second image with a non-noise label; performing a re-weighting process on the second image; The first image and the reweighted second image constitute a target training set, and the target training set is used to train the first model to obtain a second model; Using the second model to label the image data, to obtain labeled first data and unlabeled second data; Using the second data to train a classifier to generate third data with a pseudo label; Training the classifier according to the first data, the second data and the third data to obtain a target classifier; The face beauty prediction model with the target classifier is used to perform face beauty prediction on the image to be predicted, and a face beauty prediction result is obtained.
2. The method for predicting facial beauty according to claim 1, characterized in that: The method of classifying the training set of facial beauty prediction images to obtain a first image with a noise label and a second image with a non-noise label includes: Performing probability calculation based on the training set of the facial beauty prediction image to obtain a probability value indicating whether the facial beauty prediction image has a noise label; Obtaining a joint distribution of the training set according to the probability values; The training set is classified according to the joint distribution to obtain a first image with a noise label and a second image with a non-noise label.
3. The method for predicting facial beauty according to claim 1, characterized in that: The step of classifying the training set according to the joint distribution to obtain a first image with a noise label and a second image with a non-noise label includes: According to the joint distribution of the training set, the true distribution of the training set, the weight of the face beauty prediction image and the loss function, the expected risk of classification is obtained; The training set is classified according to the expected risk to obtain a first image with a noise label and a second image with a non-noise label.
4. The method for predicting facial beauty according to claim 3, characterized in that: The weight of the facial beauty prediction image is determined by the joint distribution and noise rate of the training set.
5. The method for predicting facial beauty according to claim 4, characterized in that: The noise rate is the minimum value of the joint distribution of the training set within a preset range.
6. The method for predicting facial beauty according to claim 4, characterized in that: When the value of the joint distribution of the training set is not equal to 0, the weight is non-negative; when the value of the joint distribution of the training set is equal to 0, the weight is equal to 0.
7. The method for predicting facial beauty according to claim 1, characterized in that: The step of training the classifier according to the first data, the second data, and the third data to obtain a target classifier includes: The classifier is trained according to the first data, the second data and the third data until the classifier converges, and the converged classifier is used as a target classifier.
8. A facial beauty prediction device, characterized in that: include: A reweighting unit, wherein the reweighting unit is configured to: classify a training set of facial beauty prediction images to obtain a first image with a noise label and a second image with a non-noise label, reweight the second image, form a target training set with the first image and the reweighted second image, and train the first model using the target training set to obtain a second model; a self-training unit, wherein the self-training unit is configured to: use the second model to label the image data to obtain first data with labels and second data without labels, use the second data to train a classifier to generate third data with pseudo labels, and train the classifier according to the first data, the second data, and the third data to obtain a target classifier; A prediction unit is configured to: perform face beauty prediction on the image to be predicted by using a face beauty prediction model having the target classifier to obtain a face beauty prediction result.
9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the face beauty prediction method as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the face beauty prediction method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Method for training multi-target detection model, detection method and related device
CN114445885A
Face recognition model training method and device
CN114463810A
Model training method and related device
CN114611607A
