Method for training eyebrow recognition model and related device

Abstract

The embodiment of the application relates to the technical field of image processing, and discloses a training method of an eyebrow recognition model and related devices, which determine eyebrow images corresponding to eyebrow regions through face key points, the eyebrow images include left eyebrow images and right eyebrow images, and feature extraction is performed on the eyebrow images based on a double-channel feature extraction module to obtain feature maps corresponding to the eyebrow images, the eyebrow features can be better extracted by using the prior condition of left and right eyebrow class consistency; a comparison data set is constructed through an image data set, image mapping is performed by using a pre-training feature extraction module, a comparison cross-entropy loss function is constructed to perform model training, the network learns the difference of eyebrow classes on different faces, and the application can improve the recognition accuracy of eyebrow classes.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a training method and related apparatus for an eyebrow recognition model. Background Technology

[0002] Eyebrows are a crucial feature in facial recognition. Due to the specificity of facial features, the same eyebrow type can appear differently on different people. Therefore, the ability to quickly and accurately identify eyebrow types is practical in the beauty and cosmetic industry, enabling clients to customize their eyebrows to match their facial features, and also plays an important role in facial recognition.

[0003] Currently, the eyebrow image is typically segmented from the face image and the eyebrow region is extracted to determine the eyebrow image. The eyebrow image is then converted into a feature vector and input into the test eyebrow image to obtain an approximate vector of the test eyebrow image. The eyebrow category of the test eyebrow is determined by using the feature vector of the eyebrow image and the approximate vector of the test eyebrow image.

[0004] However, this recognition method depends on the image quality of the eyebrow image. When the image quality of the eyebrow image is not high, it is easy to lead to a low accuracy rate in eyebrow category recognition. Summary of the Invention

[0005] The embodiments of this application aim to provide a training method and related apparatus for an eyebrow recognition model, to solve the technical problem of low accuracy in current eyebrow category recognition, and to improve the accuracy of eyebrow classification.

[0006] The embodiments of this application provide the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a training method for an eyebrow recognition model. The eyebrow recognition model includes a dual-channel feature extraction module and a pre-trained feature extraction module. The method includes:

[0008] Obtain an image dataset, which includes human face images;

[0009] Based on the facial key points of the face image, the eyebrow region is determined. The eyebrow region includes the eyebrow image, which includes the left eyebrow image and the right eyebrow image.

[0010] Based on the dual-channel feature extraction module, feature extraction is performed on the eyebrow image to obtain the feature map corresponding to the eyebrow image. The dual-channel feature extraction module includes an upper channel network and a lower channel network. The upper channel network is used to output the predicted value of the eyebrow category of the left eyebrow image, and the lower channel network is used to output the predicted value of the eyebrow category of the right eyebrow image. The dual-channel feature extraction module is used to determine the eyebrow category corresponding to the eyebrow image based on the predicted values ​​of the eyebrow category of the left eyebrow image and the predicted values ​​of the eyebrow category of the right eyebrow image.

[0011] First eyebrow image, second eyebrow image and third eyebrow image are randomly selected from the image dataset to construct a comparison dataset. The first eyebrow image is the eyebrow image input to the dual-channel feature extraction module. The first eyebrow image and the second eyebrow image have the same eyebrow category, and the first eyebrow image and the third eyebrow image have different eyebrow categories.

[0012] Based on the pre-trained feature extraction module, image mapping is performed on the first eyebrow image, the second eyebrow image, and the third eyebrow image to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image.

[0013] A contrastive cross-entropy loss function is constructed, and the eyebrow recognition model is trained based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges to obtain the trained eyebrow recognition model. The contrastive cross-entropy loss function includes dual-channel constrained multi-class cross-entropy loss and contrastive cross loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, and the contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector.

[0014] In some embodiments,

[0015] The upper channel network and the lower channel network have the same structure. Both the upper channel network and the lower channel network include several convolutional layers, fully connected layers and normalization layers connected in sequence.

[0016] The third eyebrow image consists of multiple images, each with an eyebrow category different from that of the first eyebrow image.

[0017] In some embodiments,

[0018] The upper channel network is used to input the left eyebrow image and obtain the first probability value corresponding to several eyebrow categories in the upper channel;

[0019] The lower channel network is used to input the right eyebrow image and obtain the second probability values ​​corresponding to several eyebrow categories in the lower channel;

[0020] The dual-channel feature extraction module is used to sum the first probability value and the second probability value corresponding to each eyebrow category to obtain the probability value corresponding to each eyebrow category. The eyebrow category with the highest probability value is determined as the eyebrow category corresponding to the face image.

[0021] In some embodiments, the contrastive cross-entropy loss function is used to calculate the contrastive cross-entropy loss, wherein the contrastive cross-entropy loss is calculated as follows:

[0022] Loss = L cls +L con

[0023] Where Loss is the contrastive cross-entropy loss, L cls For dual-channel constrained multi-class cross-entropy loss, L con To compare the cross loss.

[0024] In some embodiments, the dual-channel constrained multi-class cross-entropy loss includes:

[0025]

[0026] Among them, L cls For dual-channel constrained multi-class cross-entropy loss, y i Let be the probability value of the i-th eyebrow category. Let be the probability value of the i-th eyebrow category predicted by the upper channel network. Let be the probability value of the i-th eyebrow category predicted by the lower channel network. is the absolute difference between the probability value of the i-th eyebrow category predicted by the lower channel network and the probability value of the i-th eyebrow category predicted by the upper channel network.

[0027] In some embodiments, comparing the cross loss includes:

[0028]

[0029] Among them, L con To compare the cross loss, V K V is the first feature vector corresponding to the first eyebrow image. K+ Let V be the second feature vector corresponding to the second eyebrow image, τ be the scaling parameter, n be the total number of eyebrow categories, and V be the second feature vector. K- This is the third feature vector corresponding to the third eyebrow image.

[0030] Secondly, embodiments of this application provide a method for predicting eyebrow type, the method comprising:

[0031] Obtain the face image to be predicted;

[0032] The face image to be predicted is input into the eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted. The eyebrow recognition model is trained based on the method in the first aspect.

[0033] In some embodiments, the face image to be predicted is input into the eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted, including:

[0034] Facial landmarks are identified from the face image to be predicted.

[0035] Based on facial key points, the eyebrow region is determined. The eyebrow region includes the eyebrow image, which includes the left eyebrow image and the right eyebrow image.

[0036] The left eyebrow image is input into the upper channel network to obtain the first probability value corresponding to several eyebrow categories in the upper channel; and the right eyebrow image is input into the lower channel network to obtain the second probability value corresponding to several eyebrow categories in the lower channel.

[0037] The first probability value and the second probability value corresponding to each eyebrow category are summed to obtain the probability value corresponding to each eyebrow category.

[0038] The eyebrow category with the highest probability value is determined as the eyebrow category corresponding to the face image to be predicted.

[0039] Thirdly, embodiments of this application provide a non-volatile computer-readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the steps of the training method for the eyebrow recognition model as described in the first aspect, or the steps of the eyebrow category prediction method as described in the second aspect.

[0040] Fourthly, embodiments of this application provide an electronic device, including:

[0041] At least one processor, and

[0042] A memory that is communicatively connected to at least one processor, wherein,

[0043] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform the method of the first aspect or the second aspect.

[0044] The beneficial effects of this application's embodiments: Unlike existing technologies, this application's embodiments provide a training method for an eyebrow recognition model. The eyebrow recognition model includes a dual-channel feature extraction module and a pre-trained feature extraction module. The method includes: acquiring an image dataset, wherein the image dataset includes face images; determining an eyebrow region based on facial key points in the face images, the eyebrow region including eyebrow images, the eyebrow images including a left eyebrow image and a right eyebrow image; performing feature extraction on the eyebrow images based on the dual-channel feature extraction module to obtain a feature map corresponding to the eyebrow image, wherein the dual-channel feature extraction module includes an upper channel network and a lower channel network, the upper channel network is used to output the predicted value of the eyebrow category of the left eyebrow image, the lower channel network is used to output the predicted value of the eyebrow category of the right eyebrow image, and the dual-channel feature extraction module is used to determine the eyebrow category corresponding to the eyebrow image based on the predicted values ​​of the eyebrow category of the left eyebrow image and the right eyebrow image; randomly selecting a first eyebrow image, a second eyebrow image, and a third eyebrow image from the image dataset to construct... A comparison dataset is constructed, where the first eyebrow image is the eyebrow image input to the dual-channel feature extraction module. The first eyebrow image and the second eyebrow image have the same eyebrow category, while the first eyebrow image and the third eyebrow image have different eyebrow categories. Based on the pre-trained feature extraction module, image mapping is performed on the first eyebrow image, the second eyebrow image, and the third eyebrow image to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image. A contrastive cross-entropy loss function is constructed, and the eyebrow recognition model is trained based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges to obtain the trained eyebrow recognition model. The contrastive cross-entropy loss function includes dual-channel constrained multi-class cross-entropy loss and contrastive cross loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, while the contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector.

[0045] On the one hand, the eyebrow image corresponding to the eyebrow region is determined by facial key points. The eyebrow image includes the left eyebrow image and the right eyebrow image. Based on the dual-channel feature extraction module, the features of the eyebrow image are extracted to obtain the feature map corresponding to the eyebrow image. It can utilize the prior condition of consistency between the left and right eyebrow categories to better extract eyebrow features.

[0046] On the other hand, a contrast dataset is constructed using an image dataset. Image mapping is performed using a pre-trained feature extraction module, and a contrastive cross-entropy loss function is constructed. This contrastive cross-entropy loss function includes a dual-channel constrained multi-class cross-entropy loss and a contrastive cross loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, while the contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector. This allows the network to learn the differences in eyebrow categories on different faces, thereby improving the accuracy of eyebrow category recognition. Attached Figure Description

[0047] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0048] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0050] Figure 3 This is a flowchart illustrating a training method for an eyebrow recognition model provided in an embodiment of this application;

[0051] Figure 4 This is a flowchart illustrating a method for normalizing facial images with multiple eyebrow types, as provided in an embodiment of this application.

[0052] Figure 5 This is a schematic diagram of facial key points provided in an embodiment of this application;

[0053] Figure 6 This is a schematic diagram of the structure of an eyebrow recognition model provided in an embodiment of this application;

[0054] Figure 7 This is a schematic diagram illustrating the training process of an eyebrow recognition model provided in an embodiment of this application;

[0055] Figure 8 This is a flowchart illustrating a method for predicting eyebrow categories provided in an embodiment of this application;

[0056] Figure 9 yes Figure 8 A detailed flowchart of step S802 in the process;

[0057] Figure 10This is a schematic diagram of the structure of a training device for an eyebrow recognition model provided in an embodiment of this application;

[0058] Figure 11 This is a schematic diagram of the structure of an eyebrow category prediction device provided in an embodiment of this application. Detailed Implementation

[0059] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," and "third" used herein do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0062] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0063] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0064] Before providing a detailed description of this application, the nouns and terms used in the embodiments of this application are explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows:

[0065] (1) A neural network, also known as a neural network (NNs) or connection model, is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks to perform distributed parallel information processing. Neural networks rely on the complexity of the system to process information by adjusting the interconnections between a large number of internal nodes. Specifically, a neural network can be composed of neural units, which can be understood as a neural network with an input layer, hidden layers, and an output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the intermediate layers are hidden layers. Neural networks with many hidden layers are called deep neural networks (DNNs). The work of each layer in a neural network can be described by the mathematical expression y = a(W·x + b). From a physical perspective, the work of each layer in a neural network can be understood as transforming the input space (the set of input vectors) to the output space (i.e., from the row space to the column space of a matrix) through five operations on the input space: 1. Dimensional increase / decrease; 2. Magnification / reduction; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are performed by "W·x", operation 4 by "+b", and operation 5 by "a()". The term "space" is used here because the objects being classified are not individual things, but a class of things; space refers to the set of all individuals within that class. W is the weight matrix of each layer in the neural network, where each value represents the weight of a neuron in that layer. This matrix W determines the spatial transformation from the input space to the output space, meaning that the W of each layer in the neural network controls how the space is transformed. The purpose of training the neural network is to ultimately obtain the weight matrices of all layers in the trained neural network. Therefore, the training process of a neural network is essentially learning how to control spatial transformation, more specifically, learning the weight matrix.

[0066] It should be noted that in the embodiments of this application, the models used for machine learning tasks are essentially neural networks. Common components in neural networks include convolutional layers, pooling layers, normalization layers, and deconvolutional layers. By assembling these common components in neural networks, a model is designed. When the model parameters (weight matrices of each layer) are determined such that the model error meets a preset condition or the number of model parameters is adjusted to reach a preset threshold, the model converges.

[0067] The convolutional layer is configured with multiple convolutional kernels, each with a corresponding stride, to perform convolution operations on the image. The purpose of convolution is to extract different features from the input image. The first convolutional layer may only extract some low-level features such as edges, lines, and corners, while deeper convolutional layers can iteratively extract more complex features from low-level features.

[0068] A deconvolutional layer is used to map a low-dimensional space to a high-dimensional space while preserving the connections / patterns between them (the connections during convolution). A deconvolutional layer is configured with multiple convolutional kernels, each with a corresponding stride, to perform deconvolution operations on the image. Generally, framework libraries used for designing neural networks (such as the PyTorch library) have a built-in `upsumple()` function, which allows for low-dimensional to high-dimensional spatial mapping.

[0069] Pooling layers mimic the human visual system, allowing for dimensionality reduction or representing images with higher-level features. Common pooling operations include max pooling, mean pooling, random pooling, median pooling, and combined pooling. Typically, pooling layers are periodically inserted between convolutional layers in neural networks to achieve dimensionality reduction.

[0070] The normalization layer is used to normalize all neurons in the intermediate layer to prevent gradient explosion and gradient vanishing.

[0071] (2) A loss function is a function that maps the values ​​of a random event or its related random variables to non-negative real numbers to represent the "risk" or "loss" of that random event. A loss function is a non-negative real number function used to quantify the difference between the predicted label and the true label. In applications, the loss function is often used as a learning criterion in relation to optimization problems, i.e., solving and evaluating the model by minimizing the loss function. For example, it is used for parametric estimation in statistics and machine learning. During the training of a neural network, because we want the output of the neural network to be as close as possible to the actual predicted value, we can compare the current network's predicted value with the actual target value, and then update the weight matrix of each layer of the neural network based on the difference between the two (there is usually an initialization process before the first update, i.e., pre-configuring the parameters for each layer in the neural network). For example, if the network's predicted value is too high, the weight matrix is ​​adjusted to make it predict lower, and this adjustment continues until the neural network can predict the actual target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value," which is the loss function or objective function. These are important equations used to measure the difference between the predicted value and the target value. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, so training the neural network becomes the process of minimizing this loss as much as possible.

[0072] The technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0073] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0074] like Figure 1 As shown, the application environment 100 includes an electronic device 10 and a server 20. The electronic device 10 is connected to the server 20 via network communication, wherein the network includes wired networks and / or wireless networks. It is understood that the network includes wireless networks such as 2G, 3G, 4G, 5G, wireless LAN, and Bluetooth, and may also include wired networks such as serial cables and network cables.

[0075] In this embodiment, the electronic device 10 is communicatively connected to the server 20 and is used to acquire image datasets, construct feature extraction modules and multi-level image block encoding modules. For example, those skilled in the art can download facial images of various eyebrow categories onto the electronic device. It is understood that the electronic device 10 can also be used to acquire facial images to be predicted. For example, a user inputs a facial image to be predicted through an input interface, and after input, the electronic device automatically acquires the facial image to be predicted. Alternatively, the electronic device 10 may have a camera to capture facial images, or the electronic device 10 may store a facial image library from which the user can select a facial image to be predicted. The electronic device in this embodiment includes, but is not limited to, various terminals with computing capabilities such as laptops, desktop computers, or mobile devices. Preferably, the electronic device is a smartphone.

[0076] In this embodiment, the server 20 is communicatively connected to the electronic device 10 and is used to train an eyebrow recognition model, or to acquire a face image to be predicted input by the user on the electronic device 10, and to call the built-in eyebrow recognition model to process the face image to be predicted, thereby obtaining the eyebrow category corresponding to the face image to be predicted, and then sending the eyebrow category to the electronic device 10. The number of servers 20 can also be multiple, and multiple servers can form a server cluster. For example, the server cluster includes: a first server, a second server, ..., an Nth server; or, the server cluster can be a cloud computing service center, which includes several servers. The servers in this embodiment include, but are not limited to: tower servers, rack servers, blade servers, and cloud servers. Preferably, the server is a cloud server (Elastic Compute Service, ECS).

[0077] It is understood that, in this embodiment of the application, the electronic device 10 is also used to display the eyebrow category on its own display interface after receiving the predicted eyebrow category sent by the server, so as to inform the user, or to locally execute the training method of the eyebrow recognition model or the prediction method of the eyebrow category provided in this embodiment of the application.

[0078] Example 1

[0079] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0080] like Figure 2 As shown, the electronic device 200 includes one or more processors 201 and a memory 202. Wherein, Figure 2 Take a processor 201 as an example.

[0081] The processor 201 and the memory 202 can be connected via a bus or other means. Figure 2 Taking the example of a connection between China and Israel via a bus.

[0082] Processor 201 provides computing and control capabilities to control electronic device 200 to perform corresponding tasks, such as controlling electronic device 200 to perform a training method for an eyebrow recognition model in any of the following method embodiments, including: acquiring an image dataset, wherein the image dataset includes face images; determining an eyebrow region based on facial key points of the face images, the eyebrow region including eyebrow images, the eyebrow images including a left eyebrow image and a right eyebrow image; performing feature extraction on the eyebrow images based on a dual-channel feature extraction module to obtain a feature map corresponding to the eyebrow images, wherein the dual-channel feature extraction module includes an upper channel network and a lower channel network, the upper channel network is used to output a predicted value of the eyebrow category of the left eyebrow image, the lower channel network is used to output a predicted value of the eyebrow category of the right eyebrow image, and the dual-channel feature extraction module is used to determine the eyebrow category corresponding to the eyebrow image based on the predicted values ​​of the eyebrow category of the left eyebrow image and the predicted values ​​of the eyebrow category of the right eyebrow image; randomly selecting a first eyebrow image, a second eyebrow image, and a third eyebrow image from the image dataset to construct... The dataset is compared, where the first eyebrow image is the eyebrow image input to the dual-channel feature extraction module. The first eyebrow image and the second eyebrow image have the same eyebrow category, while the first eyebrow image and the third eyebrow image have different eyebrow categories. Based on the pre-trained feature extraction module, image mapping is performed on the first eyebrow image, the second eyebrow image, and the third eyebrow image to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image. A contrastive cross-entropy loss function is constructed, and the eyebrow recognition model is trained based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges to obtain the trained eyebrow recognition model. The contrastive cross-entropy loss function includes dual-channel constrained multi-class cross-entropy loss and contrastive cross loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, while the contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector.

[0083] Alternatively, the electronic device 200 may execute the eyebrow category prediction method in any of the following method embodiments, including: acquiring a face image to be predicted; inputting the face image to be predicted into an eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted, wherein the eyebrow recognition model is trained based on the training method of the eyebrow recognition model.

[0084] Processor 201 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0085] The memory 202, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the eyebrow recognition model training method or the eyebrow category prediction method in the embodiments of this application. The processor 201, by running the non-transitory software programs, instructions, and modules stored in the memory 202, can implement the eyebrow recognition model training method or the eyebrow category prediction method in any of the following method embodiments. Specifically, the memory 202 may include volatile memory (VM), such as random access memory (RAM); the memory 202 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; the memory 202 may also include combinations of the above types of memory.

[0086] Memory 202 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 202 may optionally include memory remotely located relative to processor 201, and these remote memories may be connected to processor 201 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0087] One or more modules are stored in memory 202. When executed by one or more processors 201, they perform the eyebrow recognition model training method or eyebrow category prediction method in any of the following method embodiments. For example, they perform the following described... Figure 3 The steps shown, or perform the following descriptions. Figure 8 The steps shown.

[0088] In the embodiments of this application, the electronic device 200 may also have wired or wireless network interfaces, input / output interfaces and other components to perform input and output. The electronic device 200 may also include other components for implementing device functions, which will not be described in detail here.

[0089] The following describes the training method of the eyebrow recognition model provided in this application embodiment, with reference to the exemplary application and implementation of the server provided in the embodiments of this application.

[0090] Please see Figure 3 , Figure 3 This is a flowchart illustrating a training method for an eyebrow recognition model provided in an embodiment of this application;

[0091] The training method of the eyebrow recognition model is applied to electronic devices, such as terminals and servers. Specifically, the execution subject of the training method of the eyebrow recognition model is one or at least two processors in the electronic device. The following uses a server as an example to illustrate the training method of the eyebrow recognition model.

[0092] Specifically, the eyebrow recognition model includes a dual-channel feature extraction module and a pre-trained feature extraction module;

[0093] like Figure 3 As shown, the training method for this eyebrow recognition model includes:

[0094] Step S301: Obtain an image dataset, wherein the image dataset includes face images;

[0095] Specifically, facial images belonging to different eyebrow categories can be downloaded from the Internet using electronic devices to form an image dataset. This image dataset includes facial images of various eyebrow categories, and the number of facial images included in the image dataset can be in the tens of thousands, for example, 20,000. Alternatively, the number of facial images can be determined by those skilled in the art based on the actual situation.

[0096] It is understandable that there are many ways to classify eyebrows, such as horizontal eyebrows, arched eyebrows, upward-sloping eyebrows, downward-sloping eyebrows, willow-leaf eyebrows, etc., which will not be listed here. It is also understandable that those skilled in the art can determine the eyebrow categories that can be included in the image dataset according to actual needs, such as including face images corresponding to 10 eyebrow categories, so that the trained eyebrow recognition model can identify these 10 eyebrow categories.

[0097] Furthermore, after acquiring the image dataset, each face image in the dataset is labeled with its true eyebrow category, and the labeled image dataset is uploaded to the server. It can be understood that this true eyebrow category is the actual category of the eyebrows in the corresponding face image. The true eyebrow category is equivalent to a label, and it can be encoded using a hot-coding method.

[0098] Please see Figure 4 , Figure 4 This is a flowchart illustrating a method for normalizing facial images with multiple eyebrow types, as provided in an embodiment of this application.

[0099] like Figure 4 As shown, after obtaining the image dataset, the method also includes:

[0100] Step S401: Adjust the size of the face image to obtain a face image of a fixed size;

[0101] Specifically, the size of facial images with various eyebrow types is adjusted to a fixed size, for example, a width of 112 pixels and a height of 56 pixels. This is equivalent to unifying the size of each facial image, for example, to a fixed size.

[0102] Step S402: Normalize the fixed-size face image to obtain the normalized face image.

[0103] Specifically, a normalization function is used to normalize face images of various eyebrow categories adjusted to a fixed size, resulting in pixel values ​​of the face images of various eyebrow categories after normalization. The normalization function is used to transform the pixel value range of the face image from 0-255 to 0-1.

[0104] The above normalization functions include:

[0105]

[0106] Where norm is the pixel value of the face image after normalization, and x i Let be the value of the i-th pixel in the face image, max(x) be the maximum value of the face image, and min() be the minimum value of the face image.

[0107] In this embodiment of the application, normalizing facial images of various eyebrow types helps to accelerate the training of subsequent eyebrow recognition models.

[0108] In this embodiment, the number of training and testing images for each eyebrow category is set according to specific needs, for example: 10,000 training images and 1,000 testing images. Furthermore, to expand the data sample, this embodiment also performs data augmentation operations, including illumination transformation.

[0109] In this embodiment of the application, the method further includes:

[0110] After obtaining the face image, facial landmark recognition is performed to obtain the facial landmarks. Specifically, a facial landmark algorithm is used to recognize facial landmarks in the face image, for example:

[0111] Please see Figure 5 , Figure 5 This is a schematic diagram of facial key points provided in an embodiment of this application;

[0112] like Figure 5 As shown, facial landmarks are obtained by processing the facial landmarks in the face region using a facial landmark algorithm. Each facial landmark is identified by a unique number.

[0113] Step S302: Determine the eyebrow region based on the facial key points of the face image. The eyebrow region includes the eyebrow image, which includes the left eyebrow image and the right eyebrow image.

[0114] Specifically, based on the eyebrow key points in the facial landmarks, the eyebrow region is determined, and then the corresponding eyebrow image is determined. The eyebrow image includes a left eyebrow image and a right eyebrow image. In other words, the eyebrow region includes a left eyebrow region and a right eyebrow region; the left eyebrow region corresponds to the left eyebrow image, and the right eyebrow region corresponds to the right eyebrow image.

[0115] like Figure 5As shown, the x-coordinate value of the facial key point numbered 17, the y-coordinate value of the facial key point numbered 19, the x-coordinate value of the facial key point numbered 21, and the y-coordinate value of the facial key point numbered 37 are obtained, thereby obtaining the coordinate values ​​of the upper left and lower right corner rectangles of the eyebrow to determine the left eyebrow area.

[0116] Similarly, obtain the x-coordinate of facial key point number 22, the y-coordinate of facial key point number 24, the x-coordinate of facial key point number 26, and the y-coordinate of facial key point number 44 to obtain the coordinate values ​​of the upper left and lower right corner rectangles of the eyebrow, thereby determining the right eyebrow area.

[0117] After determining the left and right eyebrow regions, preprocess the left and right eyebrow regions, for example, by adjusting their sizes to 112*56*3.

[0118] Step S303: Based on the dual-channel feature extraction module, perform feature extraction on the eyebrow image to obtain the feature map corresponding to the eyebrow image;

[0119] Specifically, the dual-channel feature extraction module includes a dual-channel feature extraction network, which consists of an upper channel network and a lower channel network. Specifically, the upper channel network is used to output the predicted value of the eyebrow category of the left eyebrow image, and the lower channel network is used to output the predicted value of the eyebrow category of the right eyebrow image. The dual-channel feature extraction module is used to determine the eyebrow category corresponding to the eyebrow image based on the predicted values ​​of the eyebrow category of the left eyebrow image and the right eyebrow image.

[0120] In this embodiment, the eyebrow image feature extraction and eyebrow recognition model utilize the prior condition of consistency between the left and right eyebrow categories. By leveraging the symmetry of the left and right eyebrow features of the face, the trained eyebrow recognition model can more accurately predict the eyebrow category.

[0121] The upper channel network and the lower channel network have the same structure, both including several convolutional layers, fully connected layers and normalization layers connected in sequence.

[0122] For example, construct a dual-channel feature extraction network, which is a dual-channel deep neural network. Each channel uses a 3*3 kernel with a stride of 2 and the ReLU activation function. After multiple layers of convolution operations, a feature map of size 7*4*128 is obtained. Finally, a fully connected layer and a softmax layer are added. This step does not limit the network structure for feature extraction and can be replaced according to actual needs. The ultimate goal is to obtain the feature map of the input image.

[0123] The basic feature extraction network consists of convolutional layers, activation function layers, and instance normalization layers, and the corresponding mathematical expression is as follows:

[0124]

[0125] in, This represents the m-th feature map of the l-th layer. Let represent the m-th feature map of the (l+l)-th layer, W represent the convolution kernel, and the kernel size is set to 3×3 to help reduce the aliasing effect after feature map fusion. B represents the bias term, σ(·) represents the ReLU activation function, and IN represents instance normalization.

[0126] Understandably, since downsampling and upsampling would lead to information loss, average stride convolution is used instead, with the number of convolution kernels in each layer set to 16, 32, 64, and 128.

[0127] In this embodiment, the dual-channel feature extraction module is used to extract features from each cropped eyebrow image. By extracting features from each cropped eyebrow image, the dual-channel feature extraction module can learn the eyebrow features corresponding to each eyebrow category. These eyebrow features can be used to predict the eyebrow category.

[0128] For example, the upper channel network and the lower channel network include an input layer, a convolutional layer, a fully connected layer and a normalization layer connected in sequence. The convolutional layer includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer connected in sequence.

[0129] Specifically, the upper and lower channel networks each use 3x3 convolutional kernels. The input layer can directly operate on the original input data; if the input is an image, the input data is the image's pixel values. The first convolutional layer has 32 kernels, and its output feature map size is 256x256, with 32 output feature maps. The second convolutional layer has 64 kernels, and its output feature map size is 128x128, with 64 output feature maps. The third convolutional layer has 128 kernels, and its output feature map size is 64x64, with 128 output feature maps. The fourth convolutional layer... The number of convolutional kernels is 256, and the size of its output feature map is 32*32, with a total of 256 output feature maps; the number of convolutional kernels in the fifth convolutional layer is 256, and the size of its output feature map is 16*16, with a total of 256 output feature maps; the number of convolutional kernels in the sixth convolutional layer is 512, and the size of its output feature map is 8*8, with a total of 512 output feature maps; the size of the fully connected layer is 1*1024; the size of the normalization layer is 1*N, where N is the total number of eyebrow categories in the face images in the image dataset, and N can be 10, or can be set by those skilled in the art according to the actual situation.

[0130] In this embodiment of the application, the upper channel network and the lower channel network are also used to predict the eyebrow category, specifically, to predict the category probability value of the eyebrow image.

[0131] Specifically, the output of each layer of the upper or lower channel network serves as the input to the next layer connected to it. This upper or lower channel network is used to extract features and classify each input eyebrow image. Finally, the normalization layer outputs an N-dimensional vector, which represents the probability value corresponding to the category of the eyebrow image. For example, if the eyebrow categories include horizontal eyebrows, arched eyebrows, upward sloping eyebrows, downward sloping eyebrows, and willow-leaf eyebrows, then the N-dimensional vector [0.01, 0.02, 0.01, 0.93, 0.03] represents the predicted probability value of the eyebrow images input to the upper or lower channel network for each eyebrow category, namely horizontal eyebrows, arched eyebrows, upward sloping eyebrows, downward sloping eyebrows, and willow-leaf eyebrows. The higher the value, the closer the eyebrow image is to that eyebrow category. For example, 0.93 indicates that the probability of the eyebrow image being a downward sloping eyebrow is 93%.

[0132] In this embodiment, the upper channel network is used to predict the eyebrow category corresponding to the left eyebrow image, and the lower channel network is used to predict the eyebrow category corresponding to the right eyebrow image. The upper and lower channel networks predict the eyebrow categories of the left and right eyebrow images respectively, and the eyebrow category in the face image is determined based on the eyebrow categories corresponding to the left and right eyebrow images.

[0133] Specifically, the upper channel network is used to input the left eyebrow image and obtain the first probability value corresponding to several eyebrow categories in the upper channel;

[0134] The lower channel network is used to input the right eyebrow image and obtain the second probability values ​​corresponding to several eyebrow categories in the lower channel;

[0135] The dual-channel feature extraction module is used to sum the first probability value and the second probability value corresponding to each eyebrow category to obtain the probability value corresponding to each eyebrow category. The eyebrow category with the highest probability value is determined as the eyebrow category corresponding to the face image.

[0136] Assuming there are five eyebrow types: horizontal, arched, upward-sloping, downward-sloping, and willow-leaf, and the first probability value output by the upper channel network is [0.1, 0.2, 0.3, 0.2, 0.2], and the second probability value output by the lower channel network is [0.2, 0.2, 0.1, 0.3, 0.2], then the first and second probability values ​​for each eyebrow type are summed to obtain the probability value for each eyebrow type as [0.1+0.2, 0.2+0.2, 0.3+0.1, 0.2+0.3, 0.2+0.2] = [0.3, 0.4, 0.4, 0.5, 0.4]. The eyebrow type with the highest probability value is the downward-sloping eyebrow, therefore, the eyebrow type corresponding to the face image is determined to be downward-sloping.

[0137] Step S304: Randomly select the first eyebrow image, the second eyebrow image, and the third eyebrow image from the image dataset to construct a comparison dataset;

[0138] Specifically, a comparison dataset is constructed. For example, during the training phase, a first eyebrow image, a second eyebrow image, and a third eyebrow image are randomly selected. The first eyebrow image is the eyebrow image input to the dual-channel feature extraction module. The first eyebrow image and the second eyebrow image have the same eyebrow type, while the first eyebrow image and the third eyebrow image have different eyebrow types. For example, the data of the i-th eyebrow category is selected and labeled as V, and data of the same eyebrow category as V are randomly selected and labeled as V. + And randomly select a set of eyebrow category data that is different from the eyebrow category of V and label it as

[0139] Specifically, let's assume the first eyebrow image is X. i For the first eyebrow image X i Randomly crop the first eyebrow image X to a size of H*H (H<200). i The corresponding first eyebrow image block is labeled as V. If multiple first eyebrow images are cropped separately, then each first eyebrow image is X. i The corresponding image patch is labeled V iIt is understandable that the length and width of the first eyebrow image block obtained by randomly cropping the first eyebrow image are the same.

[0140] After determining the first eyebrow image, at least two second eyebrow images are determined, wherein the second eyebrow images belong to the same eyebrow category as the first eyebrow image. For example: select the second eyebrow image X that is the same as the first eyebrow image. i Eyebrow images C belonging to the same eyebrow category i There can be multiple second eyebrow images. It is understood that the number of second eyebrow images is greater than or equal to the number of first eyebrow images; for example, one first eyebrow image corresponds to one or at least two second eyebrow images.

[0141] After determining at least two second eyebrow images, each second eyebrow image is cropped to obtain a second eyebrow image block corresponding to each second eyebrow image. For example, for second eyebrow image C... i Randomly crop the second eyebrow image C to size H*H (H<200). i The corresponding second eyebrow image block is labeled as V. + If multiple second eyebrow images are cropped separately, then each second eyebrow image C i The corresponding second eyebrow image block is labeled V. i + It is understandable that the length and width of the second eyebrow image block obtained by randomly cropping the second eyebrow image are the same.

[0142] After determining the first eyebrow image, at least two third eyebrow images are determined, wherein the third eyebrow images do not belong to the same eyebrow category as the first eyebrow image. Specifically, select the image X that is closest to the first eyebrow image. i Different categories of eyebrow images as third eyebrow images D t Where t can be 1, 2, ..., n, and n is the number of third eyebrow images. It can be understood that the number of third eyebrow images is greater than or equal to the number of first eyebrow images. Each third eyebrow image comprises multiple images, and the eyebrow category of each image is different from that of the first eyebrow images.

[0143] For example, one first eyebrow image corresponds to one or at least two third eyebrow images. Then, each third eyebrow image is cropped to obtain a corresponding third eyebrow image block. For example, suppose the third eyebrow image is D. t Then for the third eyebrow image D t Randomly crop a third eyebrow image D of size H*H (H<200) t The corresponding third eyebrow image block is labeled as V. -If multiple third eyebrow images are cropped separately, then each third eyebrow image D t The corresponding image patch is labeled V t - .

[0144] Step S305: Based on the pre-trained feature extraction module, perform image mapping on the first eyebrow image, the second eyebrow image, and the third eyebrow image to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image.

[0145] Specifically, the pre-trained feature extraction module includes a pre-trained feature extraction network, which includes a VGG network. For example, the pre-trained feature extraction network performs image mapping on the first eyebrow image, the second eyebrow image, and the third eyebrow image to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image. The first eyebrow image corresponds to the first feature vector, the second eyebrow image corresponds to the second feature vector, and the third eyebrow image corresponds to the third feature vector. For example, V, V... + and V - Mapped to K-dimensional feature vector V K V K+ V K- .

[0146] Specifically, based on the pre-trained feature extraction module, image mapping is performed on the first eyebrow image, the second eyebrow image, and the third eyebrow image to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image, including:

[0147] Input at least two first eyebrow images into the pre-trained feature extraction module to obtain the first feature vector corresponding to each first eyebrow image;

[0148] Input at least two second eyebrow images into the pre-trained feature extraction module to obtain the second feature vector corresponding to each second eyebrow image;

[0149] Input at least two third eyebrow images into the pre-trained feature extraction module to obtain the third feature vector corresponding to each third eyebrow image.

[0150] In this embodiment of the application, the pre-trained feature extraction module includes a multi-level image block coding network, wherein the multi-level image block coding network includes a first convolutional layer, a second convolutional layer, a third convolutional layer and a fourth convolutional layer.

[0151] Specifically, this multi-layer image patch coding network mainly uses 3*3 convolutional kernels. The first convolutional layer has 32 kernels, and its output feature map has a size of (H / 2)*(H / 2) and a number of output feature maps of 32, where H is the size of the image patch obtained above. The second convolutional layer has 64 kernels, and its output feature map has a size of (H / 4)*(H / 4) and a number of output feature maps of 64. The third convolutional layer has 128 kernels, and its output feature map has a size of (H / 8)*(H / 8) and a number of output feature maps of 128. The fourth convolutional layer has a size of 1*1.

[0152] It is understandable that the first, second, third, and fourth convolutional layers of this multi-level image block coding network are connected sequentially, with the output of each layer serving as the input of the next layer. The output of this multi-level image block coding network is a K-dimensional feature vector, where K is a positive integer. During the training of the eyebrow recognition model, the value of K can be dynamically adjusted according to the recognition accuracy of the eyebrow recognition model, or it can be adjusted manually.

[0153] It is understandable that, since the second eyebrow image and the first eyebrow image belong to the same eyebrow category, the first feature vector V corresponding to block V of the first eyebrow image is... K Regarding the second eyebrow image block V + The corresponding second feature vector V K + This is a positive sample. Since the third eyebrow image and the first eyebrow image belong to the same eyebrow category, the first feature vector V corresponding to block V of the first eyebrow image is... K In terms of the third eyebrow image block V - The corresponding third feature vector V K- This is a negative sample.

[0154] Step S306: Construct a contrastive cross-entropy loss function, and train the eyebrow recognition model based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges to obtain the trained eyebrow recognition model.

[0155] The contrastive cross-entropy loss function includes dual-channel constrained multi-class cross-entropy loss and contrastive cross loss. Dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, while contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector.

[0156] Specifically, the contrastive cross-entropy loss function is used to calculate the contrastive cross-entropy loss, which is calculated as follows:

[0157] Loss = Lcls +L con

[0158] Where Loss is the contrastive cross-entropy loss, L cls For dual-channel constrained multi-class cross-entropy loss, L con To compare the cross loss.

[0159] Specifically, firstly, leveraging the prior knowledge of the consistency between the left and right eyebrows, a dual-channel constrained multi-class cross-entropy loss is constructed. This dual-channel constrained multi-class cross-entropy loss characterizes the loss between the true probability and the predicted probability of the eyebrow category. The contrastive cross-loss characterizes the correlation between feature vectors corresponding to image patches of the same eyebrow category, and the difference between feature vectors corresponding to image patches of different eyebrow categories. Specifically, the dual-channel constrained multi-class cross-entropy loss characterizes the difference between the left and right eyebrow images, while the contrastive cross-loss characterizes the correlation between the first and second feature vectors, and the difference between the first and third feature vectors.

[0160] In some embodiments, the dual-channel constrained multi-class cross-entropy loss includes:

[0161]

[0162] Among them, L cls For dual-channel constrained multi-class cross-entropy loss, y i Let be the probability value of the i-th eyebrow category. Let be the probability value of the i-th eyebrow category predicted by the upper channel network. Let be the probability value of the i-th eyebrow category predicted by the lower channel network. is the absolute difference between the probability value of the i-th eyebrow category predicted by the lower channel network and the probability value of the i-th eyebrow category predicted by the upper channel network.

[0163] In this embodiment of the application, the output of the last layer (softmax layer) of the dual-channel deep neural network is used as the input of the cross-entropy to calculate the softmax loss. Assuming that there are n eyebrow categories, the softmax layer has n output vectors. or This represents the value of the i-th output vector, which is the probability value of the i-th class. Additionally, L1 loss is used to constrain the features of both the upper and lower layers.

[0164] In some embodiments, comparing the cross loss includes:

[0165]

[0166] Among them, L con To compare the cross loss, VK V is the first feature vector corresponding to the first eyebrow image. K+ Let V be the second feature vector corresponding to the second eyebrow image, τ be the scaling parameter, n be the total number of eyebrow categories, and V be the second feature vector. K- This is the third feature vector corresponding to the third eyebrow image.

[0167] Understandably, V K V is the first feature vector corresponding to the first eyebrow image block of the first eyebrow image. K+ V is the second feature vector corresponding to the second eyebrow image patch in the second eyebrow image. K- This is the third feature vector corresponding to the third eyebrow image block of the third eyebrow image.

[0168] Furthermore, the method for calculating the contrastive cross loss involves mapping image patches to K-dimensional feature vectors, and calculating the first feature vector V corresponding to the first eyebrow image patch V. K With the second eyebrow image block V + The corresponding second feature vector V K+ The correlation between them, where exp(V K *V K+ *τ) represents the first feature vector V corresponding to the first eyebrow image block V. K With the second eyebrow image block V + The corresponding second feature vector V K+ The correlation between them, and the calculation of the first feature vector V corresponding to the first eyebrow image block V. K With the third eyebrow image block V - The corresponding third feature vector V K- The difference, that is, the contrast difference, in which, The first feature vector V represents the first eyebrow image patch V. K With the third eyebrow image block V - The corresponding third feature vector V K- The difference, the scaling parameter τ represents the second eyebrow image block V + The corresponding second feature vector V K+ The probability of being selected, for example: τ is set to 0.5, 1, etc.

[0169] Understandably, this contrastive cross-loss integrates the first feature vector V corresponding to the first eyebrow image patch V. K With the second eyebrow image block V + The corresponding second feature vector V K+ The correlation between them and the first feature vector V corresponding to the first eyebrow image block V K With the third eyebrow image block V - The corresponding third feature vector V K-The differences between feature vectors can be compared to comprehensively identify the correlations and differences between them, enabling the pre-trained feature extraction module to better learn the eyebrow features of the same eyebrow category as well as the differences between different eyebrow categories.

[0170] Please refer to the following: Figure 6 , Figure 6 This is a schematic diagram of the structure of an eyebrow recognition model provided in an embodiment of this application;

[0171] The eyebrow recognition model includes a dual-channel feature extraction network and a pre-trained feature extraction network. The dual-channel feature extraction network includes an upper channel network and a lower channel network.

[0172] like Figure 6 As shown, the upper and lower channel networks have the same structure. Both the upper and lower channel networks include several convolutional layers, fully connected layers, and normalization layers (softmax layers) connected in sequence. The output of each layer of the upper or lower channel network serves as the input to the next layer connected to it. The upper or lower channel network is used to extract features and classify the input left or right eyebrow image, respectively. Finally, the normalization layer outputs the predicted eyebrow category value of the left or right eyebrow image. The predicted eyebrow category value is an N-dimensional vector, which represents the probability value corresponding to the category of the eyebrow image. For example, if the eyebrow categories include horizontal eyebrow, arched eyebrow, upward slanting eyebrow, downward slanting eyebrow, and willow leaf eyebrow, then the N-dimensional vector [0.01, 0.02, 0.01, 0.93, 0.03] represents the predicted probability value of the left or right eyebrow image input to the upper or lower channel network for each eyebrow category, namely horizontal eyebrow, arched eyebrow, upward slanting eyebrow, downward slanting eyebrow, and willow leaf eyebrow.

[0173] Subsequently, the features of the upper and lower layers are constrained by the dual-channel constrained multi-class cross-entropy loss. That is, the dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image.

[0174] Furthermore, after constructing the comparison dataset, it is input into a pre-trained feature extraction network. The pre-trained feature extraction network performs image mapping on the first, second, and third eyebrow images in the comparison dataset to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image. The correlation between the first and second feature vectors and the difference between the first and third feature vectors are characterized by the contrastive cross-entropy loss. Thus, the contrastive cross-entropy loss function is constructed using the dual-channel constrained multi-class cross-entropy loss and the contrastive cross-entropy loss. This contrastive cross-entropy loss function is used to calculate the contrastive cross-entropy loss, which is the sum of the dual-channel constrained multi-class cross-entropy loss and the contrastive cross-entropy loss.

[0175] Finally, the eyebrow recognition model is trained by comparing the cross-entropy loss function until the cross-entropy loss function converges, so as to obtain the trained eyebrow recognition model, which can then be used to predict the eyebrow category.

[0176] In the actual eyebrow category prediction process, such as Figure 6 As shown, the left eyebrow image is input into the upper channel network, and the right eyebrow image is input into the lower channel network. After being processed by several convolutional layers, fully connected layers, and normalization layers of the upper channel network, the eyebrow category of the left eyebrow image is predicted to obtain the upper channel eyebrow category prediction value. The upper channel eyebrow category prediction value is an N-dimensional vector, which represents the probability value corresponding to the eyebrow category of the left eyebrow image, that is, the first probability value corresponding to each eyebrow category.

[0177] Furthermore, after processing through several convolutional layers, fully connected layers, and normalization layers of the lower channel network, eyebrow category prediction is performed on the right eyebrow image to obtain the lower channel eyebrow category prediction value. The lower channel eyebrow category prediction value is an N-dimensional vector, which represents the probability value corresponding to the eyebrow category of the right eyebrow image, that is, the second probability value corresponding to each eyebrow category.

[0178] Finally, the first probability value and the second probability value corresponding to each eyebrow category are summed to obtain the probability value corresponding to each eyebrow category. The eyebrow category with the highest probability value is determined as the eyebrow category corresponding to the face image to be predicted.

[0179] Specifically, the eyebrow recognition model is trained based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges, so as to obtain the trained eyebrow recognition model.

[0180] Specifically, a contrastive cross-entropy loss function is constructed and used to train the eyebrow recognition model. The parameters of the eyebrow recognition model are adjusted until the contrastive cross-entropy loss function converges. The eyebrow recognition model at this point is taken as the eyebrow recognition model that has been trained. It can be understood that convergence means that, under a certain model parameter, the sum of the differences between each real eyebrow category in the image dataset and the eyebrow category predicted based on the eyebrow recognition model is less than a preset threshold or fluctuates within a certain range.

[0181] For details, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram illustrating the training process of an eyebrow recognition model provided in an embodiment of this application;

[0182] like Figure 7 As shown, the training process of this eyebrow recognition model includes:

[0183] Step S701: Construct an eyebrow recognition model;

[0184] Specifically, the eyebrow recognition model includes a dual-channel feature extraction module and a pre-trained feature extraction module. For details, please refer to the above embodiments, which will not be repeated here.

[0185] Step S702: Construct the contrastive cross-entropy loss function;

[0186] Specifically, the contrastive cross-entropy loss function includes the dual-channel constrained multi-class cross-entropy loss and the contrastive cross-entropy loss. This contrastive cross-entropy loss function is used to calculate the contrastive cross-entropy loss, which is the sum of the dual-channel constrained multi-class cross-entropy loss and the contrastive cross-entropy loss.

[0187] Step S703: Iteratively train the eyebrow recognition model based on the contrastive cross-entropy loss function;

[0188] Specifically, the contrastive cross-entropy loss function includes multi-class cross-entropy loss and contrastive cross-entropy loss. By utilizing the image dataset, the image dataset is processed, and then the eyebrow recognition model is iteratively trained based on the constructed contrastive cross-entropy loss function.

[0189] Step S704: Is the number of iterations greater than the threshold?

[0190] This application uses the Adam algorithm (Adaptive Moment Estimation Algorithm) to optimize model parameters. For example, the number of iterations is set to 500, the initial learning rate is set to 0.001, the weight decay is set to 0.0005, and the learning rate decays to 1 / 10 of its original value every 50 iterations.

[0191] Specifically, the Adam algorithm is used to optimize the model parameters. For example, the number of iterations is set to 10,000, the initial learning rate is set to 0.001, and the learning rate weight decay is set to 0.0005. Every 1,000 iterations, the learning rate decays to 1 / 10 of its original value. The learning rate and the difference between the real eyebrow categories and the corresponding predicted eyebrow categories in the image dataset can be input into the Adam algorithm to obtain the adjusted model parameters output by the Adam algorithm. These adjusted model parameters are then used for the next training iteration until the cross-entropy loss function converges.

[0192] It is understandable that the convergence of the contrastive cross-entropy loss function refers to the convergence of the sum of the dual-channel constrained multi-class cross-entropy loss and the contrastive cross-loss.

[0193] For dual-channel constrained multi-class cross-entropy loss, the smaller the product of the logarithmic operation of the true probability value and the predicted probability value, the smaller the value of the multi-class cross-entropy loss.

[0194] For contrastive cross loss, it makes The larger the value of V, the smaller the crossover loss, that is, making V... K *V K- The larger the value of τ, the greater the comparison cross-loss L. con The smaller the value, the better, since the scaling parameter τ is a fixed value, the smaller the value of the first feature vector V corresponding to the first eyebrow image block. K The third feature vector V corresponding to the third eyebrow image patch K- The larger the vector product, the greater the comparison cross-loss L. con The smaller the value.

[0195] If the cross-entropy loss function converges, it indicates that the eyebrow recognition model has been trained successfully. The model parameters of the eyebrow recognition model at this point are then output, thus obtaining the trained eyebrow recognition model.

[0196] Understandably, the Adam algorithm (Adaptive Moment Estimation Algorithm) can be seen as a combination of the momentum method and the RMSprop algorithm. It not only uses momentum as a parameter to update the direction, but also can adaptively adjust the learning rate.

[0197] Specifically, it checks whether the number of iterations is greater than a threshold, which is preset, for example, 500 iterations. If the number of iterations is greater than the threshold, proceed to step S706: training complete; if the number of iterations is not greater than the threshold, proceed to step S705: whether the loss of the eyebrow recognition model is less than the loss threshold.

[0198] It is understandable that this threshold number can be set according to specific needs, and no limit is set here.

[0199] Step S705: Is the loss of the eyebrow recognition model less than the loss threshold?

[0200] Specifically, determine whether the loss of the eyebrow recognition model is less than the loss threshold. If yes, proceed to step S706: Training complete; otherwise, return to step S703: Iteratively train the eyebrow recognition model based on the comparative cross-entropy loss function.

[0201] In the embodiments of this application, the loss threshold can be set to 0.0005 or 0.001. It is understood that the loss threshold can be set according to specific needs and is not limited here.

[0202] Step S706: Training complete;

[0203] Understandably, after training is complete, a trained eyebrow recognition model is obtained. At this point, the eyebrow category corresponding to the face image to be predicted can be predicted by calling this trained eyebrow recognition model.

[0204] In this embodiment, by constructing a contrastive cross-entropy loss function and training the eyebrow recognition model based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges, the eyebrow recognition model can better learn eyebrow classification and eyebrow difference comparison, especially strengthening the learning of eyebrow differential features and improving the accuracy of eyebrow classification.

[0205] After training an eyebrow recognition model using the training method provided in this application, the eyebrow recognition model can be used to predict the eyebrow category of an eyebrow image. The eyebrow category prediction method provided in this application can be implemented by various types of electronic devices with computing capabilities, such as smart terminals and servers.

[0206] In this embodiment, a training method for an eyebrow recognition model is provided. The eyebrow recognition model includes a dual-channel feature extraction module and a pre-trained feature extraction module. The method includes: acquiring an image dataset, wherein the image dataset includes face images; determining the eyebrow region based on the facial key points of the face images, the eyebrow region including eyebrow images, and the eyebrow images including a left eyebrow image and a right eyebrow image; performing feature extraction on the eyebrow images based on the dual-channel feature extraction module to obtain the feature map corresponding to the eyebrow image, wherein the dual-channel feature extraction module includes an upper channel network and a lower channel network, the upper channel network is used to output the predicted value of the eyebrow category of the left eyebrow image, the lower channel network is used to output the predicted value of the eyebrow category of the right eyebrow image, and the dual-channel feature extraction module is used to determine the eyebrow category corresponding to the eyebrow image based on the predicted values ​​of the eyebrow category of the left eyebrow image and the predicted values ​​of the eyebrow category of the right eyebrow image; and randomly selecting a first eyebrow image, a second eyebrow image, and a third eyebrow image from the image dataset to construct a comparison dataset. The first eyebrow image is the eyebrow image input to the dual-channel feature extraction module. The first eyebrow image and the second eyebrow image have the same eyebrow category, while the first eyebrow image and the third eyebrow image have different eyebrow categories. Based on the pre-trained feature extraction module, image mapping is performed on the first eyebrow image, the second eyebrow image, and the third eyebrow image to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image. A contrastive cross-entropy loss function is constructed, and the eyebrow recognition model is trained based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges to obtain the trained eyebrow recognition model. The contrastive cross-entropy loss function includes dual-channel constrained multi-class cross-entropy loss and contrastive cross loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, while the contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector.

[0207] On the one hand, the eyebrow image corresponding to the eyebrow region is determined by facial key points. The eyebrow image includes the left eyebrow image and the right eyebrow image. Based on the dual-channel feature extraction module, the features of the eyebrow image are extracted to obtain the feature map corresponding to the eyebrow image. It can utilize the prior condition of consistency between the left and right eyebrow categories to better extract eyebrow features.

[0208] On the other hand, a contrast dataset is constructed using an image dataset. Image mapping is performed using a pre-trained feature extraction module, and a contrastive cross-entropy loss function is constructed. This contrastive cross-entropy loss function includes a dual-channel constrained multi-class cross-entropy loss and a contrastive cross loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, while the contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector. This allows the network to learn the differences in eyebrow categories on different faces, thereby improving the accuracy of eyebrow category recognition.

[0209] Example 2

[0210] After training the eyebrow recognition model using the method provided in the above embodiments, a trained eyebrow recognition model is obtained, which can be used to predict eyebrow categories.

[0211] It is understood that the above embodiment one is the training stage of the eyebrow recognition model, and embodiment two of this application is the prediction stage of the eyebrow recognition model. The prediction stage requires the use of a dual-channel feature extraction module to predict the eyebrow category in order to obtain the eyebrow category corresponding to the face image to be predicted.

[0212] The following describes the eyebrow category prediction method provided in this application embodiment, with reference to exemplary applications and implementations of the terminal provided in the embodiments of this application.

[0213] Please see Figure 8 , Figure 8 This is a flowchart illustrating a method for predicting eyebrow categories provided in an embodiment of this application;

[0214] like Figure 8 As shown, the prediction method for this eyebrow category includes:

[0215] Step S801: Obtain the face image to be predicted;

[0216] Specifically, the electronic device acquires the face image to be predicted. For example, the user inputs the face image to be predicted through an input interface, and after the input is completed, the electronic device automatically acquires the face image to be predicted. Alternatively, the electronic device has a camera that captures face images. Or, the electronic device stores a face image library, from which the user can select the face image to be predicted. Or, the electronic device receives the face image to be predicted uploaded by the user via the network.

[0217] Step S802: Input the face image to be predicted into the eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted. The eyebrow recognition model is trained based on the above-mentioned eyebrow recognition model training method.

[0218] Specifically, the face image to be predicted is input into the trained eyebrow recognition model to obtain the prediction result output by the eyebrow recognition model, which includes the eyebrow category corresponding to the face image to be predicted.

[0219] It is understood that the eyebrow recognition model is trained using the same training method as the eyebrow recognition model in the above embodiments, and has the same structure and function as the eyebrow recognition model in the above embodiments, which will not be described in detail here.

[0220] In some embodiments, the dual-channel feature extraction module includes a dual-channel feature extraction network, which includes an upper channel network and a lower channel network.

[0221] Please see Figure 9 , Figure 9 yes Figure 8 A detailed flowchart of step S802 in the process;

[0222] like Figure 9 As shown, step S802 includes:

[0223] Step S8021: Perform facial landmark recognition on the face image to be predicted to obtain facial landmarks;

[0224] Step S8022: Based on the facial key points, determine the eyebrow image corresponding to the eyebrow region, wherein the eyebrow image corresponding to the eyebrow region includes the left eyebrow image and the right eyebrow image;

[0225] Step S8023: Input the left eyebrow image into the upper channel network to obtain the first probability value corresponding to several eyebrow categories in the upper channel; and input the right eyebrow image into the lower channel network to obtain the second probability value corresponding to several eyebrow categories in the lower channel.

[0226] Step S8024: Sum the first probability value and the second probability value corresponding to each eyebrow category to obtain the probability value corresponding to each eyebrow category;

[0227] Specifically, assuming that eyebrow categories include five types: horizontal eyebrows, arched eyebrows, upward slanting eyebrows, downward slanting eyebrows, and willow leaf eyebrows, the first probability value output by the upper channel network is [0.1, 0.2, 0.3, 0.2, 0.2], and the second probability value output by the lower channel network is [0.2, 0.2, 0.1, 0.3, 0.2]. Then, the first probability value and the second probability value corresponding to each eyebrow category are summed to obtain the probability value corresponding to each eyebrow category as [0.1+0.2, 0.2+0.2, 0.3+0.1, 0.2+0.3, 0.2+0.2] = [0.3, 0.4, 0.4, 0.5, 0.4].

[0228] Step S8025: Determine the eyebrow category with the highest probability value as the eyebrow category corresponding to the face image to be predicted.

[0229] Specifically, assuming that the eyebrow categories include five types: horizontal eyebrow, arched eyebrow, upward slanting eyebrow, downward slanting eyebrow, and willow leaf eyebrow, and the probability value corresponding to each eyebrow category is [0.3, 0.4, 0.4, 0.5, 0.4], then the eyebrow category with the highest probability value is the downward slanting eyebrow. Therefore, the eyebrow category corresponding to the face image to be predicted is determined to be the downward slanting eyebrow.

[0230] It should be noted that the relevant content on training the eyebrow recognition model can be found in the content provided in Example 1 above, and will not be repeated here.

[0231] In this embodiment, a method for predicting eyebrow categories is provided. The method includes: acquiring a face image to be predicted; inputting the face image to be predicted into an eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted. The eyebrow recognition model is trained based on the training method of the eyebrow recognition model in Embodiment 1. By training the eyebrow recognition model and inputting the face image to be predicted into the trained eyebrow recognition model, this application can better predict the eyebrow category corresponding to the face.

[0232] Example 3

[0233] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a training device for an eyebrow recognition model provided in an embodiment of this application;

[0234] The eyebrow recognition model training device is applied to an electronic device; specifically, the eyebrow recognition model training device is applied to one or at least two processors of the electronic device.

[0235] The eyebrow recognition model includes a dual-channel feature extraction module and a pre-trained feature extraction module.

[0236] like Figure 10 As shown, the training device 101 for the eyebrow recognition model includes:

[0237] The acquisition unit 1011 is used to acquire an image dataset, wherein the image dataset includes face images;

[0238] The eyebrow image unit 1012 is used to determine the eyebrow region based on the facial key points of the face image. The eyebrow region includes an eyebrow image, which includes a left eyebrow image and a right eyebrow image.

[0239] The feature extraction unit 1013 is used to extract features from the eyebrow image based on the dual-channel feature extraction module to obtain the feature map corresponding to the eyebrow image. The dual-channel feature extraction module includes an upper channel network and a lower channel network. The upper channel network is used to output the predicted value of the eyebrow category of the left eyebrow image, and the lower channel network is used to output the predicted value of the eyebrow category of the right eyebrow image. The dual-channel feature extraction module is used to determine the eyebrow category corresponding to the eyebrow image based on the predicted value of the eyebrow category of the left eyebrow image and the predicted value of the eyebrow category of the right eyebrow image.

[0240] The comparison dataset unit 1014 is used to randomly select a first eyebrow image, a second eyebrow image, and a third eyebrow image from the image dataset to construct a comparison dataset. The first eyebrow image is the eyebrow image input to the dual-channel feature extraction module. The first eyebrow image and the second eyebrow image have the same eyebrow category, while the first eyebrow image and the third eyebrow image have different eyebrow categories.

[0241] The image mapping unit 1015 is used to perform image mapping on the first eyebrow image, the second eyebrow image, and the third eyebrow image based on the pre-trained feature extraction module, and to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image.

[0242] The model training unit 1016 is used to construct the contrastive cross-entropy loss function. The eyebrow recognition model is trained based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges to obtain the trained eyebrow recognition model. The contrastive cross-entropy loss function includes dual-channel constrained multi-class cross-entropy loss and contrastive cross loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image. The contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector.

[0243] In the embodiments of this application, the training device for the eyebrow recognition model can also be built from hardware devices. For example, the training device for the eyebrow recognition model can be built from one or more chips, and the chips can work together to complete the training method for the eyebrow recognition model described in the above embodiments. As another example, the training device for the eyebrow recognition model can also be built from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM processors (Advanced RISC Machines) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0244] The training device for the eyebrow recognition model in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0245] The training device for the eyebrow recognition model in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit the specific operating system used.

[0246] The eyebrow recognition model training device provided in this application embodiment can achieve Figure 3 To avoid repetition, the various processes involved will not be described in detail here.

[0247] It should be noted that the above-described eyebrow recognition model training device can execute the eyebrow recognition model training method provided in the above embodiments, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in the eyebrow recognition model training device embodiments can be found in the eyebrow recognition model training method provided in Embodiment 1 above.

[0248] In this embodiment, a training device for an eyebrow recognition model is provided. The eyebrow recognition model includes a dual-channel feature extraction module and a pre-trained feature extraction module. The training device includes: an acquisition unit for acquiring an image dataset, wherein the image dataset includes face images; an eyebrow image unit for determining the eyebrow region based on facial key points in the face images, the eyebrow region including eyebrow images, and the eyebrow images including a left eyebrow image and a right eyebrow image; a feature extraction unit for extracting features from the eyebrow images based on the dual-channel feature extraction module to obtain a feature map corresponding to the eyebrow image, wherein the dual-channel feature extraction module includes an upper channel network and a lower channel network, the upper channel network outputting a predicted value for the eyebrow category of the left eyebrow image, and the lower channel network outputting a predicted value for the eyebrow category of the right eyebrow image; the dual-channel feature extraction module determining the eyebrow category corresponding to the eyebrow image based on the predicted values ​​of the eyebrow category of the left eyebrow image and the right eyebrow image; and a comparison dataset unit for randomly selecting a first eyebrow image, a second eyebrow image, and a third eyebrow image from the image dataset. The system comprises three parts: an image processing unit and a model training unit. The first eyebrow image is the eyebrow image input to the dual-channel feature extraction module. The first eyebrow image and the second eyebrow image have the same eyebrow category, while the first eyebrow image and the third eyebrow image have different eyebrow categories. An image mapping unit performs image mapping on the first, second, and third eyebrow images based on the pre-trained feature extraction module, determining the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image. A model training unit constructs a contrastive cross-entropy loss function and trains the eyebrow recognition model based on this function until it converges, yielding the trained eyebrow recognition model. The contrastive cross-entropy loss function includes a dual-channel constrained multi-class cross-entropy loss and a contrastive cross-loss. The dual-channel constrained multi-class cross-entropy loss characterizes the difference between the left and right eyebrow images, while the contrastive cross-loss characterizes the correlation between the first and second feature vectors and the difference between the first and third feature vectors.

[0249] On the one hand, the eyebrow image corresponding to the eyebrow region is determined by facial key points. The eyebrow image includes the left eyebrow image and the right eyebrow image. Based on the dual-channel feature extraction module, the features of the eyebrow image are extracted to obtain the feature map corresponding to the eyebrow image. It can utilize the prior condition of consistency between the left and right eyebrow categories to better extract eyebrow features.

[0250] On the other hand, a contrast dataset is constructed using an image dataset. Image mapping is performed using a pre-trained feature extraction module, and a contrastive cross-entropy loss function is constructed. This contrastive cross-entropy loss function includes a dual-channel constrained multi-class cross-entropy loss and a contrastive cross loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, while the contrastive cross loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector. This allows the network to learn the differences in eyebrow categories on different faces, thereby improving the accuracy of eyebrow category recognition.

[0251] Example 4

[0252] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of an eyebrow type prediction device provided in an embodiment of this application;

[0253] like Figure 11 As shown, the eyebrow category prediction device 110 includes:

[0254] Image acquisition unit 111 is used to acquire the face image to be predicted;

[0255] The eyebrow category prediction unit 112 is used to input the face image to be predicted into the eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted. The eyebrow recognition model is trained based on the training method of the eyebrow recognition model in the above embodiment 1.

[0256] In this embodiment, the eyebrow category prediction device can also be constructed from hardware components. For example, the eyebrow category prediction device can be constructed from one or more chips, which can work in coordination to complete the eyebrow category prediction method described in Embodiment 2. Furthermore, the eyebrow category prediction device can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0257] The eyebrow category prediction device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0258] The eyebrow category prediction device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0259] The eyebrow category prediction device provided in this application embodiment can achieve Figure 8 To avoid repetition, the various processes involved will not be described in detail here.

[0260] It should be noted that the eyebrow category prediction device described above can execute the eyebrow category prediction method provided in the above embodiments, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the eyebrow category prediction device embodiments can be found in the eyebrow category prediction method provided in Embodiment 2 above.

[0261] In this embodiment of the application, an eyebrow category prediction device is provided, comprising: an image acquisition unit for acquiring a face image to be predicted; and an eyebrow category prediction unit for inputting the face image to be predicted into an eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted, wherein the eyebrow recognition model is trained based on the method of the first aspect.

[0262] By acquiring a face image to be predicted and inputting it into an eyebrow recognition model, the eyebrow category corresponding to the face image is obtained. The eyebrow recognition model is trained using the training method described in Example 1. By training the eyebrow recognition type, the eyebrow category corresponding to the face can be predicted more effectively.

[0263] Example 5

[0264] This application embodiment also provides a non-volatile computer-readable storage medium storing a program or instructions for causing an electronic device to execute the eyebrow recognition model training method provided in Embodiment 1 above, for example, such as... Figure 3 The training method for the eyebrow recognition model shown, or the eyebrow category prediction method provided in Embodiment 2 above, for example, Figure 8 The method for predicting eyebrow categories is shown. The program includes a computer program, and the instructions include computer-executable instructions.

[0265] In the embodiments of this application, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk or CD-ROM; or it may be a device including one or any combination of the above-mentioned memories.

[0266] In the embodiments of this application, the executable instructions may take the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as a standalone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0267] As an example, executable instructions may, but do not necessarily, correspond to files in the file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0268] As an example, executable instructions can be deployed to execute on a single computing device (including devices such as smart terminals and servers), or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0269] This application also provides a computer program product comprising one or more lines of program code stored in a non-volatile computer-readable storage medium. The processor of an electronic device reads the program code from the non-volatile computer-readable storage medium and executes the program code to complete the method steps of the eyebrow recognition model training method or eyebrow category prediction method provided in the above embodiments.

[0270] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program or program code related to hardware. The program can be stored in a non-volatile computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0271] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0272] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above. For the sake of brevity, they are not provided in detail; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A training method for an eyebrow recognition model, characterized in that, The eyebrow recognition model includes a dual-channel feature extraction module and a pre-trained feature extraction module, and the method includes: Obtain an image dataset, wherein the image dataset includes face images; Based on the facial key points of the face image, the eyebrow region is determined, the eyebrow region includes an eyebrow image, and the eyebrow image includes a left eyebrow image and a right eyebrow image; Based on the dual-channel feature extraction module, feature extraction is performed on the eyebrow image to obtain a feature map corresponding to the eyebrow image. The dual-channel feature extraction module includes an upper channel network and a lower channel network. The upper channel network outputs a predicted value for the eyebrow category of the left eyebrow image, and the lower channel network outputs a predicted value for the eyebrow category of the right eyebrow image. The dual-channel feature extraction module determines the eyebrow category corresponding to the eyebrow image based on the predicted values ​​of the eyebrow category in the left and right eyebrow images. The predicted value of the eyebrow category in the left eyebrow image is the first probability value corresponding to the eyebrow category in the left eyebrow image, and the predicted value of the eyebrow category in the right eyebrow image is the second probability value corresponding to the eyebrow category in the right eyebrow image. Based on the predicted values ​​of the eyebrow categories in the left and right eyebrow images, the eyebrow category corresponding to the eyebrow image is determined, including: summing the first and second probability values ​​corresponding to each eyebrow category to obtain the probability value corresponding to each eyebrow category, and determining the eyebrow category with the highest probability value as the eyebrow category corresponding to the eyebrow image. A first eyebrow image, a second eyebrow image, and a third eyebrow image are randomly selected from the image dataset to construct a comparison dataset. The first eyebrow image is the eyebrow image used as input to the dual-channel feature extraction module. The first eyebrow image and the second eyebrow image have the same eyebrow category, while the first eyebrow image and the third eyebrow image have different eyebrow categories. Based on the pre-trained feature extraction module, image mapping is performed on the first eyebrow image, the second eyebrow image, and the third eyebrow image to determine the first feature vector corresponding to the first eyebrow image, the second feature vector corresponding to the second eyebrow image, and the third feature vector corresponding to the third eyebrow image. A contrastive cross-entropy loss function is constructed, and the eyebrow recognition model is trained based on the contrastive cross-entropy loss function until the contrastive cross-entropy loss function converges to obtain the trained eyebrow recognition model. The contrastive cross-entropy loss function includes a dual-channel constrained multi-class cross-entropy loss and a contrastive cross-loss. The dual-channel constrained multi-class cross-entropy loss is used to characterize the difference between the left eyebrow image and the right eyebrow image, and the contrastive cross-loss is used to characterize the correlation between the first feature vector and the second feature vector, as well as the difference between the first feature vector and the third feature vector.

2. The method according to claim 1, characterized in that, The upper channel network and the lower channel network have the same structure. Both the upper channel network and the lower channel network include several convolutional layers, fully connected layers and normalization layers connected in sequence. The third eyebrow image includes multiple images, and the eyebrow category of each image is different from that of the first eyebrow image.

3. The method according to claim 1 or 2, characterized in that, The upper channel network is used to input the left eyebrow image and obtain the first probability value corresponding to several eyebrow categories in the upper channel; The lower channel network is used to input the right eyebrow image and obtain the second probability value corresponding to several eyebrow categories in the lower channel; The dual-channel feature extraction module is used to sum the first probability value and the second probability value corresponding to each eyebrow category to obtain the probability value corresponding to each eyebrow category, and to determine the eyebrow category with the highest probability value as the eyebrow category corresponding to the face image.

4. The method according to claim 1 or 2, characterized in that, The contrastive cross-entropy loss function is used to calculate the contrastive cross-entropy loss, which is calculated as follows: in, To compare cross-entropy loss, For dual-channel constrained multi-class cross-entropy loss, To compare the cross loss.

5. The method according to claim 4, characterized in that, The dual-channel constrained multi-class cross-entropy loss includes: in, For dual-channel constrained multi-class cross-entropy loss, Let be the probability value of the i-th eyebrow category. Let be the probability value of the i-th eyebrow category predicted by the upper channel network. Let be the probability value of the i-th eyebrow category predicted by the lower channel network. is the absolute difference between the probability value of the i-th eyebrow category predicted by the lower channel network and the probability value of the i-th eyebrow category predicted by the upper channel network.

6. The method according to claim 4, characterized in that, The contrastive cross loss includes: in, To compare the cross loss, This is the first feature vector corresponding to the first eyebrow image. This is the second feature vector corresponding to the second eyebrow image. For proportional parameters, This represents the total number of eyebrow categories. This is the third feature vector corresponding to the third eyebrow image.

7. A method for predicting eyebrow type, characterized in that, The method includes: Obtain the face image to be predicted; The face image to be predicted is input into the eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted, wherein the eyebrow recognition model is trained based on the method described in any one of claims 1-6.

8. The method according to claim 7, characterized in that, The step of inputting the face image to be predicted into the eyebrow recognition model to obtain the eyebrow category corresponding to the face image to be predicted includes: Facial landmarks are obtained by performing facial landmark recognition on the face image to be predicted. Based on the facial key points, the eyebrow region is determined, the eyebrow region includes an eyebrow image, and the eyebrow image includes a left eyebrow image and a right eyebrow image; The left eyebrow image is input into the upper channel network to obtain a first probability value corresponding to several eyebrow categories in the upper channel; and the right eyebrow image is input into the lower channel network to obtain a second probability value corresponding to several eyebrow categories in the lower channel. The first probability value and the second probability value corresponding to each eyebrow category are summed to obtain the probability value corresponding to each eyebrow category. The eyebrow category with the highest probability value is determined as the eyebrow category corresponding to the face image to be predicted.

9. A non-volatile computer-readable storage medium, characterized in that, The non-volatile computer-readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the training method for the eyebrow recognition model as described in any one of claims 1-6, or, when executed by a processor, implement the steps of the eyebrow category prediction method as described in claim 7 or 8.

10. An electronic device, characterized in that, include: At least one processor, and The memory communicatively connected to the at least one processor, wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method according to any one of claims 1-8.

Images