A method, device and equipment for identifying a beautified image and a storage medium

Abstract

The application provides a method, device and equipment for identifying beautified images, and a storage medium. The method comprises the following steps: obtaining a face image to be processed; generating a face alignment image corresponding to the face image; and identifying whether the face image is a beautified image through a pre-trained neural network model. The face alignment image comprises global features and local features of a face, and more accurate classification results can be obtained by processing the face alignment image through the neural network model. The neural network model is trained in combination with a center loss function, so that the model can learn features with higher distinguishability and better classification of various image categories. In addition, the neural network model can also identify whether a face part is beautified, and the operation is simple and efficient. The neural network model can simultaneously output the probability of the face image belonging to each image category and the probability of the face part being beautified. The neural network model can identify whether each face image in the same image is a beautified image, and the accuracy is high, thereby avoiding the ambiguity of image classification for images comprising multiple faces.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to a method, apparatus, device and storage medium for recognizing beautified images. Background Technology

[0002] Users frequently browse facial images online, many of which may be beautified images created using photo-editing apps or images taken with makeup applied. Users browsing these facial images have a need to know whether a face image is beautified or natural.

[0003] Currently, one method for detecting whether a facial image has been beautified using related technologies is provided. This method uses a face liquefaction detector (FALdetector) to deform the facial image, including operations such as slimming the face, enlarging the eyes, and making a smile. However, deformation is only one aspect of beautification processing. Beautification can also include non-deformation processing such as skin smoothing, whitening, sharpening, and filters. Therefore, a face liquefaction detector alone cannot detect non-deformation beautification processing. Furthermore, face liquefaction detectors can only detect beautified images obtained by image processing applications, and cannot detect beautified images taken after a person has applied makeup. Summary of the Invention

[0004] This application proposes a method, apparatus, device, and storage medium for identifying beautified images. By classifying and processing the face-aligned image corresponding to the face image through a pre-trained neural network model, it is possible to accurately determine whether the face image is a beautified image.

[0005] The first aspect of this application proposes a method for recognizing beautified images, including:

[0006] Obtain the face image to be processed;

[0007] Generate a face-aligned image corresponding to the face image;

[0008] The system uses a pre-trained neural network model to identify whether the facial image is a beautified image.

[0009] In some embodiments of this application, generating the face-aligned image corresponding to the face image includes:

[0010] The face image is scaled up to a first image of a preset size;

[0011] Multiple facial key points were detected in the face image;

[0012] Based on preset standard key points and multiple facial key points of the face image, an alignment operation is performed on the face image to obtain a second image;

[0013] The first image and the second image are merged based on color channels to obtain the face-aligned image corresponding to the face image.

[0014] In some embodiments of this application, the step of identifying whether the face image is a beautified image using a pre-trained neural network model includes:

[0015] The face-aligned image is classified by a pre-trained neural network model to obtain the probability that the face image is a beautified image.

[0016] In some embodiments of this application, the step of identifying whether the face image is a beautified image using a pre-trained neural network model includes:

[0017] The neural network model is used to classify at least one facial region in the face alignment image, and the probability that the image of each facial region is a beautified image is obtained respectively.

[0018] In some embodiments of this application, before identifying whether the face image is a beautified image using a pre-trained neural network model, the method further includes:

[0019] Construct the structure of a neural network model for recognizing beautified images;

[0020] Obtain the training set;

[0021] The neural network model is trained based on the training set.

[0022] In some embodiments of this application, the structure for constructing a neural network model for recognizing beautified images includes:

[0023] A preset number of convolutional modules are connected sequentially, where the preset number is an integer greater than or equal to 3;

[0024] The last convolutional module in the predetermined number of sequentially connected convolutional modules is connected to a predetermined high-efficiency network to obtain the structure of a neural network model for recognizing beautified images.

[0025] In some embodiments of this application, obtaining the training set includes:

[0026] Acquire multiple beautified facial images and multiple unbeautified facial images;

[0027] Generate face alignment images corresponding to each acquired face image;

[0028] The training set is obtained by labeling each face-aligned image with a classification label.

[0029] In some embodiments of this application, the step of labeling each face-aligned image with a classification tag includes:

[0030] Label each beautified face-aligned image with a beautification tag, and label each unbeautified face-aligned image with an unbeautified tag; and / or,

[0031] In the beautified face alignment image, beautification labels are marked on the beautified face parts.

[0032] In some embodiments of this application, training the neural network model based on the training set includes:

[0033] Obtain face-aligned images from the training set;

[0034] The acquired face-aligned image is input into the neural network model to obtain the classification result of the face image corresponding to the face-aligned image;

[0035] Based on the classification results of the face image, the loss value for the current training period is calculated using a preset classification-center loss function.

[0036] In some embodiments of this application, the classification result includes the probability that a face image belongs to each image category, and each image category includes at least beautified images;

[0037] The step of calculating the loss value for the current training period based on the classification result of the face image using a preset classification-center loss function includes:

[0038] Based on the number of image categories and the probability that a face image belongs to each image category, the first classification loss value corresponding to the face image is calculated using the first classification loss function included in the preset classification-center loss function;

[0039] Based on the feature vectors and feature center points of each image category corresponding to the face image, the center loss value corresponding to the face image is calculated using the center loss function included in the classification-center loss function;

[0040] Calculate the loss value for the current training period based on the first classification loss value and the center loss value.

[0041] In some embodiments of this application, the classification result also includes the probability that an image of at least one facial region in a face image belongs to each image category;

[0042] The step of calculating the loss value for the current training period based on the classification result of the face image using a preset classification-center loss function includes:

[0043] Based on the number of the at least one face part, the number of image categories, and the probability that each face part belongs to each image category, the second classification loss value corresponding to the face image is calculated using the second classification loss function included in the classification-center loss function;

[0044] The loss value for the current training period is calculated based on the first classification loss value, the second classification loss value, and the center loss value.

[0045] In some embodiments of this application, the classification result includes the probability that an image of at least one facial feature in a face image belongs to each image category;

[0046] The step of calculating the loss value for the current training period based on the classification result of the face image using a preset classification-center loss function includes:

[0047] Based on the number of the at least one face part, the number of image categories, and the probability that each face part belongs to each image category, the second classification loss value corresponding to the face image is calculated using the second classification loss function included in the classification-center loss function;

[0048] Based on the feature vectors and feature center points of each image category corresponding to the face image, the center loss value corresponding to the face image is calculated using the center loss function included in the classification-center loss function;

[0049] The loss value for the current training period is calculated based on the second classification loss value and the center loss value.

[0050] In some embodiments of this application, obtaining the face image to be processed includes:

[0051] Obtain the image to be processed;

[0052] Identify each face image included in the image to be processed;

[0053] Each face image is extracted from the image to be processed and used as the face image to be processed.

[0054] An embodiment of the second aspect of this application provides an apparatus for recognizing beautified images, comprising:

[0055] The acquisition module is used to acquire the face image to be processed;

[0056] The generation module is used to generate a face-aligned image corresponding to the face image;

[0057] The recognition module is used to identify whether the face image is a beautified image through a pre-trained neural network model.

[0058] An embodiment of the third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0059] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.

[0060] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0061] In this embodiment, a neural network model is used to classify the face image corresponding to the face image, obtaining the probability that the face image belongs to each image category, including beautified images. This accurately determines whether a face image is a beautified image. The face-aligned image includes global and local features of the face region, and processing the face-aligned image can obtain more accurate classification results.

[0062] Furthermore, the neural network model is trained using a central loss function, enabling it to learn more discriminative features and thus better separate image categories for more accurate classification. This model can also identify whether facial features have been beautified, offering simple and efficient operation. It can simultaneously output the probability of a face image belonging to each image category and the probability of facial features being beautified. For images to be processed, including both beautified and unbeautified face images, it can identify whether each face image is beautified with high accuracy, avoiding ambiguity in the classification of images containing multiple faces.

[0063] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0064] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0065] Figure 1 This illustration shows a schematic diagram of the structure of a neural network model for recognizing beautified images according to an embodiment of this application;

[0066] Figure 2 This illustration shows a process for generating a face-aligned image according to an embodiment of this application;

[0067] Figure 3 This illustration shows a first schematic diagram of a neural network model provided in an embodiment of the present application classifying face-aligned images;

[0068] Figure 4 This illustration shows a second schematic diagram of a neural network model provided in an embodiment of this application classifying face-aligned images;

[0069] Figure 5 A third schematic diagram is shown illustrating a neural network model provided in an embodiment of this application for classifying face-aligned images;

[0070] Figure 6 This invention provides a schematic diagram comparing feature classification before and after the introduction of a center loss function according to an embodiment of this application.

[0071] Figure 7 A flowchart illustrating a method for recognizing beautified images according to an embodiment of this application is shown;

[0072] Figure 8 A schematic diagram of the structure of a device for recognizing beautified images according to an embodiment of this application is shown;

[0073] Figure 9 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0074] Figure 10 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0075] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0076] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0077] The following description, in conjunction with the accompanying drawings, describes a method, apparatus, device, and storage medium for recognizing beautified images according to embodiments of this application.

[0078] Currently, related technologies use face liquefaction detectors (FAL detectors) to identify whether facial images have been beautified. These detectors deform facial images, including slimming, eye enlargement, and smiling effects. However, deformation is only one method of beautification. Beautification can also include non-deformation processes such as skin smoothing, whitening, sharpening, and filters. Therefore, face liquefaction detectors alone cannot detect non-deformation beautification processes. Furthermore, face liquefaction detectors can only detect beautified images created through image enhancement applications; they cannot detect beautified images taken after a person has applied makeup.

[0079] Based on this, embodiments of this application provide a method for identifying beautified images. This method trains a neural network model for recognizing beautified images. The neural network model then classifies the face image corresponding to the face image, obtaining the probability that the face image belongs to each image category, including beautified images. This allows for the determination of whether the face image is beautified. Since the face-aligned image includes both global and local features of the face region, processing the face-aligned image yields more accurate classification results. Furthermore, the neural network model is trained using a center loss function, making the learned features more discriminative and enabling the trained neural network model to better separate image categories, resulting in more accurate classification. The neural network model can also identify whether at least one facial feature in the face region is beautified. The operation is simple and efficient; a single neural network model can simultaneously output the probability that a face image belongs to each image category and the probability that a facial feature has undergone beautification processing.

[0080] In addition, when an image contains multiple face images, including both beautified and unbeautiful faces, the aforementioned neural network can identify whether each face image is beautified or not with high accuracy, avoiding the ambiguity in classifying images containing multiple faces.

[0081] Before classifying face images using the aforementioned neural network model, the model is trained using the following steps S1-S3, specifically:

[0082] S1: Construct the structure of a neural network model for recognizing beautified images.

[0083] A preset number of convolutional modules are sequentially connected. The preset number can be an integer greater than or equal to 3. For example, the preset number can be 3 or 4. This application embodiment does not limit the specific value of the preset number; it can be set according to requirements in practical applications.

[0084] The aforementioned convolutional module may include a convolutional layer (CONV), a batch normalization layer (BN), and an activation layer (RELU) connected in sequence.

[0085] The last convolutional module in a predetermined number of sequentially connected convolutional modules is connected to a predetermined high-efficiency network to obtain the structure of a neural network model for recognizing beautified images. The predetermined high-efficiency network can be any of the eight high-efficiency networks B0 to B7 (EfficientNet-B0 to EfficientNet-B7), or any other neural network capable of image classification.

[0086] like Figure 1 As shown, a schematic example of the structure of this neural network model is illustrated. Figure 1 The diagram only shows four convolutional modules; in actual applications, there may be other numbers of convolutional modules. Figure 1 The middle arrow indicates the data flow direction; the output of the previous convolutional module serves as the input of the next convolutional module, and the output of the last convolutional module serves as the input of the preset high-efficiency network. Figure 1 The default high-efficiency network is EfficientNet-B0, but other networks can also be used in practical applications.

[0087] S2: Obtain the training set.

[0088] Acquire a large number of face images, each of which may include one or more face regions. For a face image containing multiple face regions, extract images of each face region separately to obtain multiple face images containing only one face region. When extracting face regions, you can extract face images that include the complete face region above the neck and shoulders.

[0089] The aforementioned large number of facial images, each representing only a single facial region, includes multiple beautified facial images and multiple unbeautiful facial images. The beautified facial images include both those obtained through image enhancement applications and those captured by a camera showing a face after actual makeup application. The beautification processes performed by the image enhancement applications include one or more combinations of distortion techniques such as face slimming, eye enlargement, and smile enhancement, as well as non-distortion techniques such as skin smoothing, whitening, sharpening, and filters. The actual makeup application includes one or more combinations of foundation makeup, eye makeup, eyebrow shaping, blush application, and lipstick application.

[0090] After obtaining multiple face images using the above method, a face-aligned image corresponding to each face image is generated. For each face image, it is first scaled to a preset size, such as 224*224 or 226*226. This embodiment does not limit the specific value of the preset size; it can be set according to requirements in practical applications.

[0091] Then, multiple facial landmarks in the face image are detected using a preset facial landmark detection model. The number of detected facial landmarks can be 106 or 117, etc. Based on the preset standard landmarks and the multiple facial landmarks in the face image, an alignment operation is performed on the face image to obtain the second image.

[0092] The preset standard facial landmarks may include preset standard center points for the left eye, right eye, nose tip, left corner of mouth, and right corner of mouth. The center points for the left eye, right eye, nose tip, left corner of mouth, and right corner of mouth from the detected facial landmarks in the face image are aligned one-to-one with the preset standard facial landmarks. For example, the center point of the left eye in the face image is aligned with the standard center point of the left eye, and the center point of the right eye in the face image is aligned with the standard center point of the right eye, and so on.

[0093] Before performing the alignment operation described above, a facial region image can be extracted from the facial image based on multiple facial key points. This facial region image includes only the complete face region, excluding areas such as the shoulders and neck. Then, the corresponding coordinate points of this facial region image are aligned using the methods described above. Alternatively, after performing the alignment operation, the multiple facial key points of the aligned facial image are redefined, and a facial region image including only the complete face region is extracted from the aligned facial image based on these aligned key points.

[0094] After obtaining an image containing only the complete face region through the above operations, this face region image is scaled to the preset size to obtain a second image. This second image is equivalent to enlarging the facial region in the original face image. The first and second images are then merged based on color channels to obtain a face-aligned image corresponding to the original face image. The process of generating the face-aligned image is as follows: Figure 2 As shown.

[0095] Assuming the preset size is 224*224, the first and second images are both 224*224*3, and the face-aligned image obtained by merging based on color channels is 224*224*6.

[0096] After generating face-aligned images for each face image using the methods described above, each face-aligned image is labeled with a tag indicating its image category, which can include beautified images, unedited images, etc. Specifically, each beautified face-aligned image is labeled with a "beautified" tag, and each unedited face-aligned image is labeled with a "unedited" tag.

[0097] In other embodiments of this application, beautification tags can also be added to the beautified facial areas in the beautified face-aligned image. For example, if foundation makeup is applied in the beautified face-aligned image, a beautification tag corresponding to the foundation makeup on the facial skin is added. If eye makeup is applied, a beautification tag corresponding to the eye makeup is added. If eyeliner is applied, a beautification tag corresponding to the eyeliner is added. If eyebrows are applied, a beautification tag corresponding to the eyebrows is added. If blush is applied, a beautification tag corresponding to the cheek area is added. If lipstick is applied, a beautification tag corresponding to the lips is added.

[0098] The above-mentioned labeled face-aligned images are used to form a training set. Each image in the training set may contain only one label indicating whether the image is beautified. Alternatively, each image may include at least one label indicating whether the facial features are beautified. Alternatively, each image may include both a label indicating whether the image is beautified and at least one label indicating whether the facial features are beautified.

[0099] S3: Train the constructed neural network model based on the training set mentioned above.

[0100] Face-aligned images are acquired from the training set, and the number of face-aligned images acquired from the training set in each training cycle can be multiple. The acquired face-aligned images are input into the constructed neural network model, where a predetermined number of convolutional modules connected sequentially are used to extract features from the face-aligned images. The finally extracted feature images are input into a predetermined high-efficiency network.

[0101] For example, refer to Figure 3-4The first convolutional module takes a 224*224*6 face-aligned image and inputs it into the first convolutional module. This module expands the image's channels to 64, outputting a 224*224*64 feature image. The second convolutional module expands the image's channels to 128, outputting a 224*224*128 feature image. The third convolutional module reduces the image's channels to 3, outputting a 224*224*3 feature image. Finally, the fourth convolutional module maintains the same number of channels and outputs a 224*224*3 feature image to the EfficientNet-B0 network.

[0102] A pre-defined high-efficiency network classifies the input feature image to obtain the classification result of the face image corresponding to the face alignment image. Then, based on the classification result of the face image, the loss value of the current training cycle is calculated using a pre-defined classification-center loss function. The classification result includes the probability that the face image belongs to each image category, and each image category includes at least beautified images.

[0103] In one implementation, the face-aligned image is labeled only with a tag indicating whether it is a beautified image. Correspondingly, the classification result only includes the probability that the face image belongs to a beautified image and the probability that it belongs to a plain image.

[0104] For example, refer to Figure 3 The EfficientNet-B0 network classifies the face image based on this feature image and outputs that the probability of the face image corresponding to the face alignment image is 0.98 for a beautified image and 0.02 for a plain image.

[0105] The loss value for the current period is calculated based on the classification results. Specifically, based on the number of image categories and the probability that the face image belongs to each image category, the first classification loss value corresponding to the face image is calculated using the first classification loss function shown in formula (1), which is part of the preset classification-center loss function. Based on the feature vectors and feature center points of each image category corresponding to the face image, the center loss value corresponding to the face image is calculated using the center loss function shown in formula (2), which is part of the classification-center loss function. Based on the first classification loss value and the center loss value, the loss value for the current training period is calculated using the classification-center loss function shown in formula (3).

[0106]

[0107]

[0108] L=Lsoftmax1+λ1Lcenterloss…(3)

[0109] In formulas (1), (2), and (3) above, L is the loss value of the current training cycle, Lsoftmax1 is the first classification loss value, Lcenterloss is the center loss value, and λ1 is the balance factor controlling the two loss functions. m is the number of image categories, p i x represents the probability that a face image belongs to the i-th image category. i Let c be the feature vector of the i-th image category corresponding to the face image. yi It is the feature center point of the i-th image category corresponding to the face image.

[0110] Where, x i and c yi Both are high-dimensional feature vectors. The L2 distance between them is taken as the optimization index of the center loss. The smaller the center loss value Lcenterloss, the closer the feature is to the center point, and the better the training effect.

[0111] In another implementation, the face-aligned image is labeled with only at least one tag indicating whether a facial feature has been beautified. Correspondingly, the classification result only includes the probability that an image with at least one facial feature belongs to each image category.

[0112] For example, refer to Figure 4 The EfficientNet-B0 network classifies based on this feature image and outputs the probability of beautifying multiple facial features, such as... Figure 3 The probability of applying foundation is 0.04, the probability of applying eye makeup is 0.12, the probability of applying eyeliner is 0.04, the probability of applying eyebrows is 0.02, the probability of applying blush is 0.67, and the probability of applying lipstick is 0.08.

[0113] The loss value for the current period is calculated based on the classification results. Specifically, based on the number of at least one face part, the number of image categories, and the probability that each face part belongs to each image category, the second classification loss value corresponding to the face image is calculated using the second classification loss function shown in formula (4), which includes the classification-center loss function. Based on the feature vectors and feature center points of each image category corresponding to the face image, the center loss value corresponding to the face image is calculated using the center loss function shown in formula (2). Based on the second classification loss value and the center loss value, the loss value for the current training period is calculated using the classification-center loss function shown in formula (5).

[0114]

[0115] L=Lsoftmax2+λ2Lcenterloss…(5)

[0116] In formulas (4) and (5) above, L is the loss value of the current training cycle, Lsoftmax2 is the second classification loss value, Lcenterloss is the center loss value, and λ2 is the balance factor controlling the two loss functions. m is the number of image categories, n is the number of at least one face region mentioned above, and p ij Let be the probability that the image of the j-th face region in a face image belongs to the i-th image category.

[0117] In another implementation, the face-aligned image is labeled with a tag indicating whether the image is beautified and at least one tag indicating whether a facial feature is beautified. Correspondingly, the classification result includes the probability that the face image belongs to a beautified image, the probability that it belongs to a plain image, and the probability that at least one facial feature in the face image belongs to each image category.

[0118] For example, refer to Figure 5 The EfficientNet-B0 network classifies the face image based on this feature image, outputting a probability of 0.98 for a beautified image and a probability of 0.02 for a natural face image. It also outputs the probabilities of beautifying multiple facial features, such as... Figure 3 The probability of applying foundation is 0.04, the probability of applying eye makeup is 0.12, the probability of applying eyeliner is 0.04, the probability of applying eyebrows is 0.02, the probability of applying blush is 0.67, and the probability of applying lipstick is 0.08.

[0119] The loss value for the current period is calculated based on the classification results. Specifically, the first classification loss value is calculated using formula (1) above, the center loss value is calculated using formula (2) above, and the second classification loss value is calculated using formula (4) above. Then, based on the first classification loss value, the second classification loss value, and the center loss value, the loss value for the current training period is calculated using the classification-center loss function shown in formula (6).

[0120] L=Lsoftmax1+λ3Lcenterloss+Lsoftmax2…(6)

[0121] In formula (6), L is the loss value of the current training cycle, Lsoftmax1 is the first classification loss value, Lsoftmax2 is the second classification loss value, Lcenterloss is the center loss value, and λ3 is the balance factor that controls the two loss functions.

[0122] In any of the above implementations, after calculating the loss value of the current training period, it is determined whether the current number of training periods has reached the preset number of training periods. If so, training is stopped, and the model parameters of the training period with the smallest loss value are obtained from the trained periods. The structure of the constructed neural network model and the model parameters corresponding to the training period with the smallest loss value are determined as the trained neural network model. If the current number of training periods has not reached the preset number of training periods, the learning and training process continues in the above manner until a trained neural network model is obtained.

[0123] In this embodiment, considering that classifying beautified and bare-faced images is a relatively difficult metric learning problem, a center loss function is introduced for constraint learning during the training of the neural network model. The center loss function can pull all points of the same category towards the center point of that category. Ultimately, this ensures that the features of all beautified images are relatively close together, and the features of all bare-faced images are also relatively close together. This results in the features of beautified and bare-faced images being relatively far apart, achieving a better classification effect for beautified and bare-faced images.

[0124] To validate the classification results, the final features are dimensionality reduced and output to a two-dimensional plane for visualization. For example... Figure 6 As shown in Figure a, Figure a presents the result of training using only the classification loss function softmax-crossentrop, followed by 2D dimensionality reduction and visualization of the output features. Figure b presents the result of using both classification loss softmax-crossentrop and center loss, followed by 2D dimensionality reduction and visualization of the output features. The comparison in the figures clearly shows that using center loss effectively separates the beauty and natural beauty features, significantly reducing the mixing of the two features.

[0125] After training the neural network model for recognizing beautified images using the above method, as follows: Figure 7 As shown, the neural network model is used to identify whether a face image to be processed is a beautified image through the following steps:

[0126] Step 101: Obtain the face image to be processed.

[0127] The process begins by acquiring an image to be processed, which includes at least one face image. Each face image within the image is then identified using facial recognition technology. Each face image is then cropped from the original image and used as the final face image for processing. The cropped face image can capture the entire face region above the neck and shoulders.

[0128] Step 102: Generate the face alignment image corresponding to the face image.

[0129] For each face image extracted from the image to be processed, a corresponding face-aligned image is generated. The specific generation process is the same as the operation of generating face-aligned images during step S2 above when obtaining the training set, and will not be repeated here.

[0130] Step 103: Identify whether the face image is a beautified image using a pre-trained neural network model.

[0131] For each face image included in the image to be processed, a corresponding face-aligned image is generated. Then, each face-aligned image is input into the neural network model trained through the above steps S1-S3. The neural network model extracts features from each input face-aligned image and classifies them based on the extracted features, outputting the probability that the face image corresponding to each face-aligned image belongs to each image category.

[0132] In one implementation, the trained neural network model outputs only the probability that a face image belongs to each image category, including beautified images. If the probability of belonging to a beautified image is greater than 0.5, the face image is considered beautified; if the probability is less than or equal to 0.5, the face image is not beautified. For example, assuming the image categories only include beautified and plain images, and the probability of belonging to a beautified image is 0.98 while the probability of belonging to a plain image is 0.02, then the face image is determined to be a beautified image.

[0133] In another implementation, the trained neural network model outputs only the probability of at least one facial feature in a face image being beautified. If the sum of the probabilities of at least one facial feature being beautified is greater than 0.5, or if there is a probability greater than 0.5 among the probabilities of at least one facial feature being beautified, then the face image is considered to be beautified. For example, suppose at least one feature includes facial skin, eyes, eyebrows, cheeks, and lips. Suppose the neural network model outputs a probability of 0.04 for applying foundation to facial skin, 0.12 for applying eye makeup to the eyes, 0.04 for applying eyeliner to the eyes, 0.02 for applying eyebrows to the eyebrows, 0.67 for applying blush to the cheeks, and 0.08 for applying lipstick to the lips. The sum of these probabilities is 0.97, or there is a probability greater than 0.5 (0.67), therefore the face image is determined to be beautified.

[0134] In another implementation, the trained neural network model outputs the probability that a face image belongs to each image category and the probability that at least one facial feature has been beautified. Each image category includes beautified images. If the probability of belonging to a beautified image is greater than 0.5, the face image is considered beautified; if the probability is less than or equal to 0.5, the face image is not beautified. Furthermore, the probability of at least one facial feature being beautified clearly identifies which facial feature in the face image has undergone beautification processing.

[0135] Each face image in the image to be processed is identified individually using the above method to determine whether it is a beautified image. The recognition results for each face image can then be labeled on the image to be processed, allowing users to visually determine whether each face image in the image to be processed is a beautified image after the image is displayed.

[0136] This application's embodiments can also be applied to scenarios where the effects of cosmetic surgery are evaluated. When users view evaluations of cosmetic surgery procedures, they typically prefer to see natural, unedited facial images in the reviews to more accurately assess the results. Therefore, upon receiving user evaluations of cosmetic surgery procedures, if the evaluation includes a facial image, the method provided in this application's embodiments can first detect whether the facial image is filtered or not. If it is filtered, the user can be prompted to provide a natural, unedited image. Alternatively, evaluations containing filtered images can be placed at the end of the cosmetic surgery procedure's evaluation list, allowing other users to see evaluations containing natural, unedited images first.

[0137] In this embodiment, a neural network model is used to classify the face image corresponding to the face image, obtaining the probability that the face image belongs to each image category, including beautified images. This accurately determines whether a face image is beautified. The face-aligned image includes global and local features of the face region, and processing the face-aligned image yields more accurate classification results. Furthermore, the neural network model is trained using a center loss function, enabling the model to learn more discriminative features, allowing it to further separate the image categories and classify them more accurately. This neural network model can also identify whether facial features are beautified, is simple and efficient, and can simultaneously output the probability that a face image belongs to each image category and the probability that facial features are beautified. For images to be processed, including beautified and unbeautified face images, it can identify whether each face image is beautified with high accuracy, avoiding ambiguity in the classification of images containing multiple faces.

[0138] This application also provides an apparatus for recognizing beautified images, which is used to perform the method for recognizing beautified images provided in any of the above embodiments. Figure 8 As shown, the device includes:

[0139] The acquisition module 201 is used to acquire the face image to be processed;

[0140] The generation module 202 is used to generate a face-aligned image corresponding to the face image;

[0141] The recognition module 203 is used to identify whether a face image is a beautified image through a pre-trained neural network model.

[0142] The generation module 202 is used to scale the face image to a first image of a preset size; detect multiple facial key points of the face image; perform an alignment operation on the face image according to the preset standard key points and the multiple facial key points of the face image to obtain a second image; and merge the first image and the second image based on the color channel to obtain the face-aligned image corresponding to the face image.

[0143] The recognition module 203 is used to classify the face-aligned image using a pre-trained neural network model to obtain the probability that the face image is a beautified image.

[0144] The recognition module 203 is used to classify at least one facial region in the face alignment image through a neural network model, and obtain the probability that the image of each facial region is a beautified image.

[0145] The device also includes: a model training module for constructing the structure of a neural network model for recognizing beautified images; acquiring a training set; and training the neural network model based on the training set.

[0146] The model training module is used to sequentially connect a preset number of convolutional modules, where the preset number is an integer greater than or equal to 3; the last convolutional module in the sequentially connected preset number of convolutional modules is connected to a preset high-efficiency network to obtain the structure of a neural network model for recognizing beautified images.

[0147] The model training module is used to acquire multiple beautified face images and multiple unbeautified face images; generate face alignment images corresponding to each acquired face image; and label each face alignment image with a classification label to obtain a training set.

[0148] The model training module is used to annotate beautification labels in each beautified face-aligned image and to annotate bare-face labels in each bare-face-aligned image; and / or, to annotate beautified facial features in the beautified face-aligned images.

[0149] The model training module is used to acquire face-aligned images from the training set; input the acquired face-aligned images into the neural network model to obtain the classification results of the face images corresponding to the face-aligned images; and calculate the loss value of the current training cycle based on the classification results of the face images using a preset classification-center loss function.

[0150] The classification results include the probability that a face image belongs to each image category, and each image category includes at least beautified images; the model training module is used to calculate the first classification loss value corresponding to the face image based on the number of image categories and the probability that the face image belongs to each image category, using the first classification loss function included in the preset classification-center loss function; based on the feature vectors and feature center points of each image category corresponding to the face image, the center loss value corresponding to the face image is calculated using the center loss function included in the classification-center loss function; based on the first classification loss value and the center loss value, the loss value of the current training cycle is calculated.

[0151] The classification results also include the probability that an image of at least one facial feature in a face image belongs to each image category; the model training module is also used to calculate the second classification loss value corresponding to the face image through the second classification loss function included in the classification-center loss function, based on the number of at least one facial feature, the number of image categories, and the probability that an image of each facial feature belongs to each image category; and to calculate the loss value of the current training cycle based on the first classification loss value, the second classification loss value, and the center loss value.

[0152] The classification results include the probability that an image with at least one facial feature belongs to each image category; the model training module is used to calculate the second classification loss value corresponding to the facial image using the second classification loss function included in the classification-center loss function, based on the number of at least one facial feature, the number of image categories, and the probability that each facial feature belongs to each image category; based on the feature vectors and feature centers of each image category corresponding to the facial image, the center loss value corresponding to the facial image is calculated using the center loss function included in the classification-center loss function; and based on the second classification loss value and the center loss value, the loss value for the current training cycle is calculated.

[0153] The acquisition module 201 is used to acquire the image to be processed; identify each face image included in the image to be processed; and extract each face image from the image to be processed as a face image to be processed.

[0154] The apparatus for recognizing beautified images provided in the above embodiments of this application and the method for recognizing beautified images provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0155] This application also provides an electronic device for performing the above-described method for recognizing beautified images. Please refer to... Figure 9 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 9 As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected via the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the method for recognizing beautified images provided in any of the foregoing embodiments of this application.

[0156] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0157] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs. After receiving an execution instruction, the processor 800 executes the program. The method for recognizing beautified images disclosed in any of the foregoing embodiments of this application can be applied to the processor 800, or implemented by the processor 800.

[0158] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0159] The electronic device provided in this application embodiment and the method for recognizing beautified images provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0160] This application also provides a computer-readable storage medium corresponding to the method for recognizing beautified images provided in the foregoing embodiments. Please refer to... Figure 10 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the method for recognizing beautified images provided in any of the foregoing embodiments.

[0161] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0162] The computer-readable storage medium provided in the above embodiments of this application and the method for recognizing beautified images provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0163] It should be noted that:

[0164] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0165] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0166] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0167] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for recognizing beautified images, characterized in that, include: Obtain the face image to be processed; Generate a face-aligned image corresponding to the face image; The face alignment image is identified as a beautified image by a pre-trained neural network model. The neural network model is used to classify face images in the training set. The classification results include the probability that an image with at least one facial feature belongs to each image category, and the probability that a face image belongs to each image category, wherein each image category includes at least beautified images. Based on the classification results of the face images in the training set, the loss value of the current training period is calculated using a preset classification-center loss function to train the neural network model, including: Based on the number of image categories and the probability that a face image belongs to each image category, the first classification loss value corresponding to the face image is calculated using the first classification loss function included in the classification-center loss function; Based on the feature vectors and feature center points of each image category corresponding to the face image, the center loss value corresponding to the face image is calculated using the center loss function included in the classification-center loss function; Based on the number of the at least one face part, the number of image categories, and the probability that each face part belongs to each image category, the second classification loss value corresponding to the face image is calculated using the second classification loss function included in the classification-center loss function; The loss value for the current training period is calculated based on the first classification loss value, the second classification loss value, and the center loss value.

2. The method according to claim 1, characterized in that, The process of generating the face-aligned image corresponding to the face image includes: The face image is scaled up to a first image of a preset size; Multiple facial key points were detected in the face image; Based on preset standard key points and multiple facial key points of the face image, an alignment operation is performed on the face image to obtain a second image; The first image and the second image are merged based on color channels to obtain the face-aligned image corresponding to the face image.

3. The method according to claim 1, characterized in that, The step of identifying whether the facial image is a beautified image using a pre-trained neural network model includes: The face-aligned image is classified by a pre-trained neural network model to obtain the probability that the face image is a beautified image.

4. The method according to claim 1 or 3, characterized in that, The step of identifying whether the facial image is a beautified image using a pre-trained neural network model includes: The neural network model is used to classify at least one facial region in the face alignment image, and the probability that the image of each facial region is a beautified image is obtained respectively.

5. The method according to any one of claims 1-3, characterized in that, Before identifying whether the face image is a beautified image using a pre-trained neural network model, the process further includes: Construct the structure of a neural network model for recognizing beautified images; Obtain the training set; The neural network model is trained based on the training set.

6. The method according to claim 5, characterized in that, The structure for constructing a neural network model for recognizing beautified images includes: A preset number of convolutional modules are connected sequentially, where the preset number is an integer greater than or equal to 3; The last convolutional module in the predetermined number of sequentially connected convolutional modules is connected to a predetermined high-efficiency network to obtain the structure of a neural network model for recognizing beautified images.

7. The method according to claim 5, characterized in that, The acquisition of the training set includes: Acquire multiple beautified facial images and multiple unbeautified facial images; Generate face alignment images corresponding to each acquired face image; The training set is obtained by labeling each face-aligned image with a classification label.

8. The method according to claim 7, characterized in that, The step of labeling each face-aligned image with a classification tag includes: Label each beautified face-aligned image with a beautification tag, and label each unbeautified face-aligned image with an unbeautified tag; and / or, In the beautified face alignment image, beautification labels are marked on the beautified face parts.

9. The method according to claim 8, characterized in that, The step of training the neural network model based on the training set includes: Obtain face-aligned images from the training set; The acquired face-aligned image is input into the neural network model to obtain the classification result of the face image corresponding to the face-aligned image; Based on the classification results of the face image, the loss value for the current training period is calculated using a preset classification-center loss function.

10. The method according to any one of claims 1-3, characterized in that, The process of acquiring the face image to be processed includes: Obtain the image to be processed; Identify each face image included in the image to be processed; Each face image is extracted from the image to be processed and used as the face image to be processed.

11. A device for recognizing beautified images, characterized in that, include: The acquisition module is used to acquire the face image to be processed; The generation module is used to generate a face-aligned image corresponding to the face image; The recognition module is used to identify whether the face-aligned image is a beautified image through a pre-trained neural network model; The device is configured for the neural network model to classify face images in a training set. The classification result includes the probability that an image of at least one face part in a face image belongs to each image category, and the probability that a face image belongs to each image category, wherein each image category includes at least beautified images. The model training module is used to calculate the first classification loss value corresponding to the face image based on the number of image categories and the probability that the face image belongs to each image category, through a first classification loss function included in the preset classification-center loss function; Based on the feature vectors and feature center points of each image category corresponding to the face image, the center loss value corresponding to the face image is calculated using the center loss function included in the classification-center loss function; Based on the number of the at least one face part, the number of image categories, and the probability that each face part belongs to each image category, the second classification loss value corresponding to the face image is calculated using the second classification loss function included in the classification-center loss function; The loss value for the current training period is calculated based on the first classification loss value, the second classification loss value, and the center loss value.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-10.

Images