Skin color recognition model training method, skin color recognition method, device and electronic equipment

Abstract

The application provides a skin color recognition model training method, a skin color recognition method, a device and an electronic device, and relates to the technical field of image processing. The method comprises the following steps: acquiring a sample face image, wherein the sample face image comprises a plurality of sample images of different angles, and a real skin color value is pre-labeled in each sample image; identifying the plurality of sample images of different angles by using a network model to be trained to obtain recognition results under the plurality of angles, wherein the recognition result under each angle comprises a predicted skin color value under the angle; calculating a skin color prediction loss value according to the real skin color value in the plurality of sample images of different angles and the predicted skin color value under the corresponding angle; calculating a total loss value according to the skin color prediction loss value; updating parameters of the network model according to the total loss value to obtain a skin color recognition model. The application can realize convenient and fine multi-angle skin color recognition.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a skin color recognition model training method, skin color recognition method, device, and electronic device. Background Technology

[0002] With the development of image processing technology, skin color detection through facial images has become an important means to help users understand their skin color.

[0003] Existing skin color recognition algorithms select a frontal view of the face, use facial recognition technology to determine the skin area, calculate the LAB value of the skin area, and use the international standard color difference formula CIEDE2000 and standard color number (Pantone 110) to match the skin color and obtain the final skin color value.

[0004] However, this LAB calculation method is not convenient enough and can only perform skin color recognition on the main view image, and cannot achieve fine skin color recognition from multiple perspectives. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by providing a skin color recognition model training method, skin color recognition method, device, and electronic device, so as to achieve convenient and precise multi-view skin color recognition.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, embodiments of this application provide a method for training a skin color recognition model, the method comprising:

[0008] Acquire sample facial images, which include: sample images from multiple viewpoints, each of which is pre-labeled with the true skin color value;

[0009] The network model to be trained is used to identify the sample images from the multiple viewpoints to obtain the identification results from the multiple viewpoints. The identification result from each viewpoint includes the predicted skin color value from each viewpoint.

[0010] The skin color prediction loss value is calculated based on the real skin color value in the sample images from the multiple viewpoints and the predicted skin color value under the corresponding viewpoints.

[0011] Calculate the total loss value based on the skin color prediction loss value;

[0012] Based on the total loss value, the parameters of the network model are updated to obtain the skin color recognition model.

[0013] Optionally, the recognition result for each viewpoint further includes: the predicted viewpoint corresponding to the sample image for each viewpoint, and the method further includes:

[0014] Calculate the view category loss value based on the multiple viewpoints and the corresponding predicted viewpoints;

[0015] The step of calculating the total loss value based on the predicted loss value of skin color includes:

[0016] The total loss value is calculated based on the skin color prediction loss value and the viewpoint category loss value.

[0017] Optionally, the recognition result for each viewpoint further includes: a predicted skin region mask for each viewpoint, and the method further includes:

[0018] Obtain the true skin region mask of the sample image under each viewpoint;

[0019] Based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, calculate the region mask loss value;

[0020] The step of calculating the total loss value based on the predicted loss value of skin color includes:

[0021] The total loss value is calculated based on the skin color prediction loss value, the viewpoint category loss value, and the region mask loss value.

[0022] Optionally, calculating the region mask loss value based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints includes:

[0023] Based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, the first region mask loss value is calculated using the first hyperparameter.

[0024] Based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, the second hyperparameter is used to calculate the second region mask loss value;

[0025] The mask loss values ​​of the first region and the second region are weighted according to the target weight to obtain the region mask loss value.

[0026] Optionally, before weighting the first region mask loss value and the second region mask loss value according to the target weight to obtain the region mask loss value, the method further includes:

[0027] The target weight is calculated based on the preset loop parameters, the current training round, and the total training rounds.

[0028] Optionally, acquiring the sample facial image includes:

[0029] Acquire initial sample images from multiple perspectives;

[0030] Key point recognition is performed on the initial sample images from the multiple perspectives to generate a set of facial key points;

[0031] Based on the set of facial key points, initial facial images from the multiple perspectives are obtained from the initial sample images from the multiple perspectives.

[0032] Based on the target key points in the set of facial key points, calculate the rotation angle of the face in the initial sample images from the multiple viewpoints;

[0033] Based on the rotation angle, the coordinates of each key point in the set of facial key points are transformed;

[0034] Based on the coordinates of the converted set of facial key points, the initial facial images from the multiple perspectives are aligned and adjusted to obtain sample images from the multiple perspectives.

[0035] Secondly, embodiments of this application also provide a skin color recognition method, the method comprising:

[0036] Acquire the facial image to be identified;

[0037] A skin color recognition model is used to perform skin color recognition on the facial image to be recognized, and the skin color recognition result is output. The skin color recognition model is trained using the method described in any of the first aspects.

[0038] Thirdly, embodiments of this application also provide a skin color recognition model training device, the device comprising:

[0039] The sample image acquisition module is used to acquire sample facial images, which include: sample images from multiple perspectives, each of which is pre-labeled with the real skin color value;

[0040] The sample image recognition module is used to recognize the sample images from the multiple perspectives using the network model to be trained, and to obtain the recognition results from the multiple perspectives. The recognition result from each perspective includes the predicted skin color value from each perspective.

[0041] The skin color prediction loss calculation module is used to calculate the skin color prediction loss value based on the real skin color value in the sample images from the multiple viewpoints and the predicted skin color value under the corresponding viewpoint.

[0042] The total loss calculation module is used to calculate the total loss value based on the predicted loss value of the skin color.

[0043] The parameter update module is used to update the parameters of the network model based on the total loss value to obtain the skin color recognition model.

[0044] Optionally, the recognition result for each viewpoint further includes: the predicted viewpoint corresponding to the sample image for each viewpoint, and the device further includes:

[0045] The visual category loss calculation module is used to calculate the view category loss value based on the multiple viewpoints and the corresponding predicted viewpoints;

[0046] The total loss calculation module is also used to calculate the total loss value based on the skin color prediction loss value and the view category loss value.

[0047] Optionally, the recognition result for each viewpoint further includes: a predicted skin region mask for each viewpoint, and the device further includes:

[0048] The skin region mask acquisition module is used to acquire the real skin region mask of the sample image under each viewpoint;

[0049] The region mask loss calculation module is used to calculate the region mask loss value based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints.

[0050] The total loss calculation module is further configured to calculate the total loss value based on the skin color prediction loss value, the viewpoint category loss value, and the region mask loss value.

[0051] Optionally, the region mask loss calculation module includes:

[0052] The first region mask loss calculation unit is used to calculate the first region mask loss value based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, using the first hyperparameter.

[0053] The second region mask loss calculation unit is used to calculate the second region mask loss value based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, using the second hyperparameter.

[0054] The weighting unit is used to weight the mask loss value of the first region and the mask loss value of the second region according to the target weight to obtain the mask loss value of the region.

[0055] Optionally, prior to the weighting unit, the apparatus further includes:

[0056] The target weight calculation unit is used to calculate the target weight based on preset loop parameters, the current training round, and the total training rounds.

[0057] Optionally, the sample image acquisition module includes:

[0058] The initial sample image acquisition unit is used to acquire initial sample images from multiple viewpoints;

[0059] A key point recognition unit is used to perform key point recognition on the initial sample images from the multiple perspectives to generate a set of facial key points.

[0060] An initial facial image acquisition unit is used to acquire initial facial images from the initial sample images from the multiple perspectives based on the set of facial key points.

[0061] The rotation angle calculation unit is used to calculate the rotation angle of the face in the initial sample images from the multiple viewpoints based on the target key points in the set of facial key points.

[0062] The coordinate transformation unit is used to transform the coordinates of each key point in the set of facial key points according to the rotation angle;

[0063] The image alignment unit is used to align and adjust the initial facial images from multiple perspectives based on the coordinates of the converted set of facial key points, so as to obtain sample images from the multiple perspectives.

[0064] Fourthly, embodiments of this application also provide a skin color recognition device, the device comprising:

[0065] The image acquisition module is used to acquire facial images to be recognized.

[0066] The image recognition module is used to perform skin color recognition on the facial image to be recognized using a skin color recognition model and output the skin color recognition result. The skin color recognition model is trained using the method described in the first aspect.

[0067] Fifthly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the skin color recognition model training method as described in any of the first aspects, or the steps of the skin color recognition method as described in the second aspect.

[0068] In a sixth aspect, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the skin color recognition model training method as described in any of the first aspects, or the steps of the skin color recognition method as described in the second aspect.

[0069] The beneficial effects of this application are:

[0070] This application provides a skin color recognition model training method, skin color recognition method, device, and electronic device. By training a network model, a skin color recognition model is obtained, so as to perform convenient skin color recognition. At the same time, the network model is trained using sample images from multiple perspectives, so that the trained skin color recognition model can achieve refined multi-view skin color recognition. Attached Figure Description

[0071] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 A flowchart illustrating the skin color recognition model training method provided in this application embodiment. Figure 1 ;

[0073] Figure 2 This is a schematic diagram of the network model provided in the embodiments of this application;

[0074] Figure 3 This is a schematic diagram of the skin color feature extraction module provided in an embodiment of this application;

[0075] Figure 4 A flowchart illustrating the skin color recognition model training method provided in this application embodiment. Figure 2 ;

[0076] Figure 5 A flowchart illustrating the skin color recognition model training method provided in this application embodiment. Figure 3 ;

[0077] Figure 6 A flowchart illustrating the skin color recognition model training method provided in this application embodiment. Figure 4 ;

[0078] Figure 7 A flowchart illustrating the skin color recognition model training method provided in this application embodiment. Figure 5 ;

[0079] Figure 8 A schematic flowchart illustrating the skin color recognition method provided in this application embodiment;

[0080] Figure 9 This is a schematic diagram of the structure of the skin color recognition model training device provided in the embodiments of this application;

[0081] Figure 10 This is a schematic diagram of the structure of the skin color recognition device provided in the embodiments of this application;

[0082] Figure 11 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0083] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0084] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0085] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0086] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0087] Before describing the skin color recognition model training method and skin color recognition method provided in the embodiments of this application, the electronic equipment used in the embodiments of this application will be described first.

[0088] The electronic device that performs the skin color recognition training method and the electronic device that performs the skin color recognition method can be the same electronic device or different electronic devices. If they are different electronic devices, the electronic device that performs the skin color recognition model training method can be a computer device. After the skin color recognition model is trained, it is deployed on the electronic device that performs the skin color recognition method. The electronic device that performs the skin color recognition method can be a skin detector or a smart terminal. The skin detector and the smart terminal have camera functions and computing capabilities. The skin detector or the smart terminal acquires the facial image to be recognized, inputs the facial image to be recognized into the deployed skin color recognition model for image recognition, and outputs the skin color recognition result.

[0089] Please refer to Figure 1 This is a flowchart illustrating the skin color recognition model training method provided in the embodiments of this application. Figure 1 ,like Figure 1 As shown, the method may include:

[0090] S101: Obtain sample facial images, which include sample images from multiple viewpoints, each of which is pre-labeled with the true skin color value.

[0091] In this embodiment, a skin detector or smart terminal is used to acquire sample images of multiple faces from multiple viewpoints as a training dataset. Artificial skin tone annotation is performed on the sample images from each viewpoint to determine the true skin tone value of the faces in each viewpoint sample image. The multiple viewpoints may include at least two of the following: a main viewpoint, a left viewpoint, and a right viewpoint; other viewpoints may also be included, but this embodiment does not impose any limitations on this.

[0092] In one possible implementation, the true skin color value can be labeled as follows: convert the sample image in the RGB color space to the Lab color space, determine the Lab value of the sample image, and use the Lab value as the true skin color value.

[0093] In some embodiments, since there are areas on the face such as spots, beards, and hair that affect the brightness value L, it is necessary to remove these areas from the sample image. The specific removal method is as follows:

[0094] The brightness range is divided into M levels. The brightness values ​​L of the sample image are statistically analyzed to determine the level to which the brightness value L of each pixel belongs. The top N levels with the most pixels are determined. The brightness values ​​L of the pixels corresponding to the top N levels are summed and averaged to obtain the target brightness value of the sample image.

[0095] Using this brightness value calculation method is more effective than setting a single brightness threshold in removing areas such as spots, beards, and hair. For people with dark skin, it can also filter out abnormal areas such as white beards and white spots.

[0096] For example, Table 1 shows the Lab values ​​of a multi-view sample image. As shown in Table 1, the real skin color values ​​of the multi-view sample images are labeled using the above-mentioned labeling method for real skin color values. The real skin color values ​​of sample images from different viewpoints are different.

[0097] Table 1. Lab values ​​of a multi-view sample image.

[0098] L a b Front view 53.98 14.58 18.85 Left view 55.77 14.76 19.82 Right view 52.48 13.89 18.81

[0099] In some embodiments, data gain can be used to add moles, hair, and spots to the sample images, enabling the network model to be trained more deeply.

[0100] S102: The network model to be trained is used to identify sample images from multiple perspectives to obtain recognition results from multiple perspectives. The recognition results from each perspective include the predicted skin color value from each perspective.

[0101] In this embodiment, the network model to be trained includes a skin color feature extraction module, which is used to extract features from sample images from multiple perspectives to obtain predicted skin color values ​​corresponding to sample images from multiple perspectives.

[0102] For example, please refer to Figure 2 This is a schematic diagram of the network model provided in the embodiments of this application, such as... Figure 2 As shown, after the sample images from each viewpoint are input into the skin color feature extraction module for feature extraction, the predicted skin color value corresponding to the sample images from each viewpoint is output after convolution through a 3*3 convolutional layer (Conv3*3).

[0103] Please refer to Figure 3 This is a schematic diagram of the skin color feature extraction module provided in an embodiment of this application, as shown below. Figure 3As shown, the sample image from each viewpoint first passes through a 1*1 convolutional layer (Conv1*1) and an activation function (Leaky ReLU, LReLU) to double the number of channels. If downsampling is required, the doubled-channel sample image is processed through multiple 3*3 convolutional layers (Conv3*3) and an activation function (LReLU) for feature extraction, outputting multiple feature maps. The stride of the first 3*3 convolutional layer is 2. Simultaneously, the doubled-channel sample image also passes through a global average pooling layer (AVG POOL) to output a feature map. If downsampling is not required, the doubled-channel sample image is processed through multiple 3*3 convolutional layers (Conv3*3) and an activation function (LReLU) for feature extraction, outputting multiple feature maps F. The stride of the first 3*3 convolutional layer is 1. The multiple feature maps F are then processed through a feature map channel filtering weight module (spatial). The network uses a weighted merging process (choice) to assign different weights to different feature maps, thereby improving the feature discrimination ability of the network model.

[0104] like Figure 3 As shown, the feature map channel selection weight module (spatial choice) performs a global average pooling layer (AVG POOL) on each feature map F along the channel dimension to obtain a feature map Fc based on channel description. The feature map Fc is then input into a two-layer perceptron consisting of two 1*1 convolutional layers (Conv1*1) and an activation function (LReLU). After that, the weight coefficients Mc of each feature map are obtained through the activation function (Sigmoid). Then, the channel size of the feature map is restored by passing through a 1*1 convolutional layer (Conv1*1) and the activation function (LReLU), thus avoiding the increase in computational overhead caused by module stacking.

[0105] Wherein, the input size of the feature map F is F∈R H*W*channels The feature map Fc after the global average pooling layer (AVG POOL) has a size of Fc∈R. 1*1*channels The size of the activation function (LReLU) in the two-layer perceptron is set to R. channels / r*1*1 Where r is the percentage reduction in the number of channels, which can be set to 8 in this embodiment to reduce network parameter overhead. The output channels of the first convolutional layer (Conv1*1) of the two-layer perceptron are channels / r, and the output channels of the second convolutional layer (Conv1*1) are restored to channels. The weight coefficient Mc∈R channels*1*1 .

[0106] S103: Calculate the skin color prediction loss value based on the real skin color values ​​in sample images from multiple viewpoints and the predicted skin color values ​​at the corresponding viewpoints.

[0107] In this embodiment, for the predicted skin color value output from the sample image at each viewpoint, the skin color prediction loss value L for each viewpoint is calculated based on the corresponding true skin color value. Color .

[0108] Among them, the skin color prediction loss value L Color The calculation formula can be expressed as:

[0109]

[0110] Where G represents the network model, G(F) input V represents the predicted skin color value output by the network model. target Represents the true skin color value, where i corresponds to L, a, or b.

[0111] S104: Calculate the total loss value based on the predicted loss value for skin color.

[0112] In this embodiment, in each training epoch of training the network model, sample images from multiple perspectives are used as the training dataset. In each training epoch, some or all of the sample images in the training dataset need to be identified and the skin color prediction loss value L needs to be calculated. Color For each training round, the skin color prediction loss value L is calculated for all sample images participating in the training. Color Sum the results to obtain the total loss value.

[0113] S105: Update the parameters of the network model based on the total loss value to obtain the skin color recognition model.

[0114] In this embodiment, the loss gradient is calculated based on the total loss value, the network parameter change value is calculated based on the loss gradient through the back gradient parameter, and the network parameters of the network model are updated based on the network parameter change value.

[0115] For the network model with updated network parameters, continue to train it multiple times using the training methods S101-S105 mentioned above until the network model converges, at which point the training ends. The converged network model is the skin color recognition model, which is used to perform skin color recognition on the facial image to be recognized.

[0116] The skin color recognition model training method provided in the above embodiments obtains a skin color recognition model by training a network model, so as to use the skin color recognition model for convenient skin color recognition; at the same time, the network model is trained using sample images from multiple perspectives, so that the trained skin color recognition model can achieve refined multi-view skin color recognition.

[0117] In one possible implementation, the recognition result for each viewpoint also includes: the predicted viewpoint corresponding to the sample image for each viewpoint. Please refer to [reference needed]. Figure 4 This is a flowchart illustrating the skin color recognition model training method provided in the embodiments of this application. Figure 2 ,like Figure 4 As shown, the method may further include:

[0118] S151: Calculate the view category loss value based on multiple viewpoints and their corresponding prediction viewpoints.

[0119] The calculation of the total loss value based on the skin color prediction loss value in S105 above may include:

[0120] S152: Calculate the total loss value based on the skin color prediction loss value and the view category loss value.

[0121] In this embodiment, in order to improve the training accuracy of the network model for skin color recognition under multiple perspectives, the perspective of the input sample image can be predicted, and the parameters of the network model can be updated according to the perspective prediction results.

[0122] Specifically, such as Figure 2 As shown, in addition to the labeled true skin color values, the sample images also label the true viewpoints. Besides outputting predicted skin color values, the network model also outputs predicted viewpoints through another 3x3 convolutional layer (Conv3*3). For each viewpoint sample image, based on the true and predicted viewpoints, the viewpoint classification loss value L is calculated for each viewpoint. Label .

[0123] Based on the skin color prediction loss value L of all sample images participating in the training in the current training round. Color Summing these values ​​yields the total skin color prediction loss value L. Color The viewpoint classification loss L is based on the viewpoint classification values ​​of all training sample images in the current training round. Label Summing yields the total perspective classification loss value L. Label ; Loss value for total skin color prediction L Color Total perspective classification loss value L Label Perform a weighted summation to obtain the total loss value.

[0124] In some embodiments, since the network model's prediction of viewpoints is a classification process, the cross-entropy loss function can be used to calculate the viewpoint classification loss value for each viewpoint based on the real viewpoint and the predicted viewpoint.

[0125] For example, the view classification loss value L Label The calculation formula can be expressed as:

[0126]

[0127] Where y = 1 indicates that the predicted viewpoint output by the network model is consistent with the real viewpoint, and p ∈ [0, 1] is the probability of the predicted viewpoint output by the network model.

[0128] It should be noted that when the network model predicts the viewpoint of a sample image, the output result is the probability that the sample image has multiple viewpoints. After one-hot encoding, the viewpoint with the highest probability is used as the predicted viewpoint.

[0129] The skin color recognition model training method provided in the above embodiments calculates the loss function based on the predicted viewpoint output by the network model in order to update the network parameters of the network model, so that the trained skin color recognition model can achieve more refined multi-view skin color recognition.

[0130] In one possible implementation, the recognition results for each viewpoint also include: a predicted skin region mask for each viewpoint. Please refer to [reference needed]. Figure 5 This is a flowchart illustrating the skin color recognition model training method provided in the embodiments of this application. Figure 3 ,like Figure 5 As shown, the method may further include:

[0131] S153: Obtain the real skin region mask of the sample image under each viewpoint.

[0132] S154: Calculate the region mask loss value based on the predicted skin region mask under multiple viewpoints and the real skin region mask under the corresponding viewpoints.

[0133] The calculation of the total loss value based on the skin color prediction loss value in S105 above may include:

[0134] S155: Calculate the total loss value based on the skin color prediction loss value, view category loss value, and region mask loss value.

[0135] The extraction of skin regions by the network model can ensure that the network model can perform skin color recognition more accurately. Therefore, in this embodiment, the network parameters of the network model are further updated based on the recognition results of the skin regions by the network model, so as to ensure that the network model can achieve more accurate skin color recognition while extracting the skin regions more accurately.

[0136] Specifically, skin region detection is performed on sample images from each viewpoint to obtain a mask of the real skin region, such as... Figure 2As shown, in addition to outputting the predicted skin color value, the network model also concatenates the feature maps output by the skin color feature extraction model with the feature maps output by the residual module ResBloceks and the upsampling model to output a predicted skin region mask. For the sample images of each viewpoint, the region mask loss value L is calculated based on the predicted skin region mask and the real skin region mask. Mask For example, a human face segmentation technique can be used to obtain a mask of the real skin region from sample images from each viewpoint.

[0137] In some embodiments, the pixel value of the skin region mask is 1, and the pixel value of the non-skin region mask is 0. Therefore, the prediction of the skin region mask by the network model also belongs to classification. Thus, the cross-entropy loss function can be used to calculate the region mask loss value for each viewpoint based on the predicted skin region mask and the real skin region mask.

[0138] Based on the skin color prediction loss value L of all sample images participating in the training in the current training round. Color Summing these values ​​yields the total skin color prediction loss value L. Color The viewpoint classification loss L is based on the viewpoint classification values ​​of all training sample images in the current training round. Label Summing yields the total perspective classification loss value L. Label Based on the region mask loss value L of all sample images participating in the training in the current training round. Mask Summing yields the total area masking loss value L. Mask ; Loss value for total skin color prediction L Color Total perspective classification loss value L Label and area mask loss value L Mask Perform a weighted summation to obtain the total loss value.

[0139] For example, the formula for calculating the total loss value can be expressed as:

[0140] L Total =αL Color +βL Mask +γL Label

[0141] Where α, β, and γ are the loss weights, respectively.

[0142] The skin color recognition model training method provided in the above embodiments calculates the loss function based on the predicted skin region mask output by the network model, so as to update the network parameters of the network model, enabling the network model to focus on more accurate skin regions, and ensuring that the trained skin color recognition model can achieve more accurate skin color recognition while extracting skin regions more accurately.

[0143] In one possible implementation, please refer to Figure 6 This is a flowchart illustrating the skin color recognition model training method provided in the embodiments of this application. Figure 4 ,like Figure 6 As shown, S154 above calculates the region mask loss value based on the predicted skin region mask under multiple viewpoints and the real skin region mask under the corresponding viewpoints, which may include:

[0144] S154a: Based on the predicted skin region mask under multiple viewpoints and the real skin region mask under the corresponding viewpoints, the first hyperparameter is used to calculate the first region mask loss value.

[0145] S154b: Based on the predicted skin region mask under multiple viewpoints and the real skin region mask under the corresponding viewpoints, the second hyperparameter is used to calculate the second region mask loss value.

[0146] S154c: The mask loss values ​​of the first region and the second region are weighted according to the target weight to obtain the region mask loss value.

[0147] In this embodiment, an initial region mask loss value is calculated based on the predicted skin region mask and the actual skin region mask. A first region mask loss value is calculated based on a first hyperparameter and the initial region mask loss value. A second region mask loss value is calculated based on a second hyperparameter and the initial region mask loss value. The first hyperparameter and the second hyperparameter are used to assign different confidence levels to the initial region mask loss value. The confidence level assigned to the initial region mask loss value by the first hyperparameter is greater than the confidence level assigned to the initial region mask loss value by the second hyperparameter.

[0148] The first and second region mask loss values ​​are weighted according to the target weights to obtain the region mask loss value. The target weights vary in different training epochs, causing the network model to learn different feature richness from the region mask loss values ​​in different training epochs. For example, in the early stages of training, the network parameters are unstable, and to ensure the predicted skin region mask output by the network model is closer to the real skin region mask, the weight of the high-confidence first region mask loss value needs to be greater than that of the low-confidence second region mask loss value. In the middle stages of training, as the network parameters gradually stabilize, since the real skin region mask may have missed or over-labeled instances, to avoid incorrect learning by the network model, the learning of the real skin region mask needs to be reduced, requiring the weight of the high-confidence first region mask loss value to be less than that of the low-confidence second region mask loss value. Furthermore, in the later stages of training, the weight of the high-confidence first region mask loss value is again greater than that of the low-confidence second region mask loss value, allowing the network model to converge better and achieve the best training effect.

[0149] In one possible implementation, before weighting the first region mask loss value and the second region mask loss value according to the target weight in S154c to obtain the region mask loss value, the method may further include:

[0150] Calculate the target weight based on the preset loop parameters, the current training round, and the total training rounds.

[0151] In this embodiment, preset loop parameters are set, and the target weight is calculated based on the current training round and the total training round, so that the target weight changes as the training round increases.

[0152] Example, region masking loss value L Mask The calculation formula can be expressed as:

[0153] L Mask =λL high +(1-λ)L low

[0154] Where λ is the target weight, and the formula for how λ changes with training rounds is:

[0155]

[0156] L high L represents the mask loss value for the first region. low The mask loss value for the second region can be calculated using the following formulas:

[0157] L high =-(1+p) t ) ξhigh log(p t )

[0158] L low =-(1-p t ) ξlow log(p t )

[0159] ξ high Let ξ be the first hyperparameter. low This is the second hyperparameter.

[0160] Where rc is a preset loop parameter, rc ≥ 1. For example, when rc = 1, λ decreases from 1 to 0; when rc = 2, λ decreases from 1 to 0 in the first half of the training epochs and increases from 0 to 1 in the second half; when rc = 4, λ decreases from 1 to 0 in the first quarter of the training epochs and increases from 0 to 1 in the last three-quarters, and so on, epoch number. c For the current training epoch, all This refers to the total number of training rounds.

[0161] The skin color recognition model training method provided in the above embodiments calculates the first region mask loss value and the second region mask loss value using the first hyperparameter and the second hyperparameter, respectively. The first region mask loss value and the second region mask loss value are weighted according to the target weight to obtain the region mask loss value. This makes the network model pay different attention to the real skin region mask in different training rounds, ensuring that the network model can learn the real skin region mask while avoiding learning mislabeled or missed areas in the real skin region mask, thereby improving the accuracy of the network model.

[0162] Please refer to Figure 7 This is a flowchart illustrating the skin color recognition model training method provided in the embodiments of this application. Figure 5 ,like Figure 7 As shown, the above-mentioned step S101, which acquires a sample facial image, may include:

[0163] S111: Obtain initial sample images from multiple perspectives.

[0164] S112: Perform key point recognition on initial sample images from multiple perspectives to generate a set of facial key points.

[0165] S113: Based on the set of facial key points, obtain initial facial images from multiple perspectives from initial sample images from multiple perspectives.

[0166] S114: Calculate the rotation angle of the face in the initial sample images from multiple perspectives based on the target key points in the set of facial key points.

[0167] S115: Based on the rotation angle, transform the coordinates of each key point in the set of facial key points.

[0168] S116: Based on the coordinates of the converted set of facial key points, the initial facial images from multiple perspectives are aligned and adjusted to obtain sample images from multiple perspectives.

[0169] In this embodiment, the initial sample image is an image acquired using a skin detector or a smart terminal. The initial sample image may be skewed during the acquisition process, so it needs to be rotated to ensure the accuracy of model training.

[0170] Specifically, by performing keypoint detection on the initial sample image, a set of facial keypoints (FP) is determined. Based on the coordinates of each keypoint in the FP set, the region corresponding to the largest bounding rectangle is determined as the initial facial image. For example, the largest bounding rectangle can be extended outward to obtain a face cropping box, and the initial facial image is cropped from the initial sample image based on the face cropping box. The outward extension can be 30%.

[0171] Select target key points from the set of facial key points, and calculate the rotation angle of the face in the initial sample image based on the position of the target key points. For example, you can select the center key points of both eyes, and calculate the rotation angle based on the angle between the line connecting the center key points of both eyes and the horizontal axis.

[0172] Based on the rotation angle, the coordinates of each key point in the facial key point set are rotated, and based on the coordinates of the rotated facial key point set FP*, the initial facial images from multiple perspectives are straightened and adjusted to obtain sample images from multiple perspectives.

[0173] The skin color recognition model training method provided in this application uses the Adam optimizer. The network input is a 3-channel LAB image, and the output has 3 branches. The main branch outputs the predicted skin color LAB value, the second branch outputs the predicted skin color region mask, and the third branch outputs the predicted viewpoint. The initial training learning rate is 0.0005, the number of iterations is 700K, and after actual training and adjustment, the loss weights are α=1, β=1, and γ=0.5.

[0174] Based on the aforementioned skin color recognition model, this application also provides a skin color recognition method. Please refer to... Figure 8 This is a flowchart illustrating the skin color recognition method provided in an embodiment of this application, as shown below. Figure 8 As shown, the method may include:

[0175] S201: Obtain the facial image to be recognized.

[0176] S202: Use a skin color recognition model to identify the facial image to be identified and output the skin color recognition result.

[0177] The skin color recognition model is trained using the skin color recognition model training method described in the above embodiments.

[0178] In this embodiment, the skin color recognition model trained using the above embodiments is deployed to a skin detector or smart terminal. The skin detector or smart terminal executes the skin color recognition method through the skin color recognition model to determine the skin color value of the face in the facial image to be recognized.

[0179] Based on the embodiments of the above-described skin color recognition model training method, this application also provides a skin color recognition model training device. Please refer to... Figure 9 This is a schematic diagram of the structure of the skin color recognition model training device provided in the embodiments of this application, as shown below. Figure 9 As shown, the device may include:

[0180] The sample image acquisition module 101 is used to acquire sample facial images, which include: sample images from multiple perspectives, each of which is pre-labeled with a real skin color value.

[0181] The sample image recognition module 102 is used to recognize the sample images from the multiple perspectives using a network model to be trained, and to obtain the recognition results from the multiple perspectives. The recognition result from each perspective includes the predicted skin color value from each perspective.

[0182] The skin color prediction loss calculation module 103 is used to calculate the skin color prediction loss value based on the real skin color value in the sample images from the multiple viewpoints and the predicted skin color value under the corresponding viewpoint.

[0183] The total loss calculation module 104 is used to calculate the total loss value based on the predicted loss value of the skin color;

[0184] The parameter update module 105 is used to update the parameters of the network model according to the total loss value to obtain the skin color recognition model.

[0185] Optionally, the recognition result for each viewpoint further includes: the predicted viewpoint corresponding to the sample image for each viewpoint, and the device may further include:

[0186] The visual category loss calculation module is used to calculate the view category loss value based on the multiple viewpoints and the corresponding predicted viewpoints;

[0187] The total loss calculation module 104 is further configured to calculate the total loss value based on the skin color prediction loss value and the viewpoint category loss value.

[0188] Optionally, the recognition result for each viewpoint further includes: a predicted skin region mask for each viewpoint, and the device may further include:

[0189] The skin region mask acquisition module is used to acquire the real skin region mask of the sample image under each viewpoint;

[0190] The region mask loss calculation module is used to calculate the region mask loss value based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints.

[0191] The total loss calculation module 104 is further configured to calculate the total loss value based on the skin color prediction loss value, the view category loss value, and the region mask loss value.

[0192] Optionally, the region mask loss calculation module includes:

[0193] The first region mask loss calculation unit is used to calculate the first region mask loss value based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, using the first hyperparameter.

[0194] The second region mask loss calculation unit is used to calculate the second region mask loss value based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, using the second hyperparameter.

[0195] The weighting unit is used to weight the mask loss value of the first region and the mask loss value of the second region according to the target weight to obtain the mask loss value of the region.

[0196] Optionally, prior to the weighting unit, the device may further include:

[0197] The target weight calculation unit is used to calculate the target weight based on preset loop parameters, the current training round, and the total training rounds.

[0198] Optionally, the sample image acquisition module 101 includes:

[0199] The initial sample image acquisition unit is used to acquire initial sample images from multiple viewpoints;

[0200] A key point recognition unit is used to perform key point recognition on the initial sample images from the multiple perspectives to generate a set of facial key points.

[0201] An initial facial image acquisition unit is used to acquire initial facial images from the initial sample images from the multiple perspectives based on the set of facial key points.

[0202] The rotation angle calculation unit is used to calculate the rotation angle of the face in the initial sample images from the multiple viewpoints based on the target key points in the set of facial key points.

[0203] The coordinate transformation unit is used to transform the coordinates of each key point in the set of facial key points according to the rotation angle;

[0204] The image alignment unit is used to align and adjust the initial facial images from multiple perspectives based on the coordinates of the converted set of facial key points, so as to obtain sample images from the multiple perspectives.

[0205] Based on the embodiments of the skin color recognition method described above, this application also provides a skin color recognition device. Please refer to... Figure 10 This is a schematic diagram of the skin color recognition device provided in the embodiments of this application, as shown below. Figure 10 As shown, the device may include:

[0206] Image acquisition module 201 is used to acquire the face image to be recognized;

[0207] Image recognition module 202 is used to perform skin color recognition on the face image to be recognized using a skin color recognition model and output the skin color recognition result. The skin color recognition model is trained using the method described in the first aspect.

[0208] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0209] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0210] Please refer to Figure 11 This is a schematic diagram of the electronic device provided in the embodiments of this application, such as... Figure 11 As shown, the electronic device 100 includes a processor 110, a storage medium 120, and a bus. The storage medium 120 stores program instructions executable by the processor 110. When the electronic device 100 is running, the processor 110 communicates with the storage medium 120 via the bus. The processor 110 executes the program instructions to perform an embodiment of the skin color recognition model training method described above, or to perform an embodiment of the skin color recognition method described above. The specific implementation and technical effects are similar and will not be described again here.

[0211] Optionally, embodiments of this application also provide a computer-readable storage medium storing a computer program. When the computer program is run by a processor, it executes an embodiment of the skin color recognition model training method described above, or executes an embodiment of the skin color recognition method described above. The specific implementation and technical effects are similar and will not be repeated here.

[0212] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0213] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0214] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0215] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0216] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for training a skin color recognition model, characterized in that, The method includes: Acquire sample facial images, which include: sample images from multiple viewpoints, each of which is pre-labeled with the true skin color value; The network model to be trained is used to identify the sample images from the multiple viewpoints to obtain the identification results from the multiple viewpoints. The identification result from each viewpoint includes the predicted skin color value from each viewpoint. The skin color prediction loss value is calculated based on the real skin color value in the sample images from the multiple viewpoints and the predicted skin color value under the corresponding viewpoints. Calculate the total loss value based on the skin color prediction loss value; Based on the total loss value, the parameters of the network model are updated to obtain the skin color recognition model; The recognition result for each viewpoint also includes: the predicted viewpoint corresponding to the sample image for each viewpoint, and the method further includes: Calculate the view category loss value based on the multiple viewpoints and the corresponding predicted viewpoints; The step of calculating the total loss value based on the predicted loss value of skin color includes: The total loss value is calculated based on the skin color prediction loss value and the viewpoint category loss value.

2. The method as described in claim 1, characterized in that, The recognition result for each viewpoint also includes: a predicted skin region mask for each viewpoint, and the method further includes: Obtain the true skin region mask of the sample image under each viewpoint; Based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, calculate the region mask loss value; The step of calculating the total loss value based on the predicted loss value of skin color includes: The total loss value is calculated based on the skin color prediction loss value, the viewpoint category loss value, and the region mask loss value.

3. The method as described in claim 2, characterized in that, The step of calculating the region mask loss value based on the predicted skin region mask under the multiple viewpoints and the actual skin region mask under the corresponding viewpoints includes: Based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, the first region mask loss value is calculated using the first hyperparameter. Based on the predicted skin region mask under the multiple viewpoints and the real skin region mask under the corresponding viewpoints, the second hyperparameter is used to calculate the second region mask loss value; The mask loss values ​​of the first region and the second region are weighted according to the target weight to obtain the region mask loss value.

4. The method as described in claim 3, characterized in that, Before weighting the first region mask loss value and the second region mask loss value according to the target weight to obtain the region mask loss value, the method further includes: The target weight is calculated based on the preset loop parameters, the current training round, and the total training rounds.

5. The method as described in claim 1, characterized in that, The acquisition of sample facial images includes: Acquire initial sample images from multiple perspectives; Key point recognition is performed on the initial sample images from the multiple perspectives to generate a set of facial key points; Based on the set of facial key points, initial facial images from the multiple perspectives are obtained from the initial sample images from the multiple perspectives. Based on the target key points in the set of facial key points, calculate the rotation angle of the face in the initial sample images from the multiple viewpoints; Based on the rotation angle, the coordinates of each key point in the set of facial key points are transformed; Based on the coordinates of the converted set of facial key points, the initial facial images from the multiple perspectives are aligned and adjusted to obtain sample images from the multiple perspectives.

6. A skin color recognition method, characterized in that, The method includes: Acquire the facial image to be identified; A skin color recognition model is used to perform skin color recognition on the facial image to be recognized, and the skin color recognition result is output. The skin color recognition model is trained using the method described in any one of claims 1-5.

7. A skin color recognition model training device, characterized in that, The device includes: The sample image acquisition module is used to acquire sample facial images, which include: sample images from multiple perspectives, each of which is pre-labeled with the real skin color value; The sample image recognition module is used to recognize the sample images from the multiple perspectives using the network model to be trained, and to obtain the recognition results from the multiple perspectives. The recognition result from each perspective includes the predicted skin color value from each perspective. The skin color prediction loss calculation module is used to calculate the skin color prediction loss value based on the real skin color value in the sample images from the multiple viewpoints and the predicted skin color value under the corresponding viewpoint. The total loss calculation module is used to calculate the total loss value based on the predicted loss value of the skin color. The parameter update module is used to update the parameters of the network model based on the total loss value to obtain the skin color recognition model.

8. A skin color recognition device, characterized in that, The device includes: The image acquisition module is used to acquire facial images to be recognized. An image recognition module is used to perform skin color recognition on the facial image to be recognized using a skin color recognition model and output the skin color recognition result. The skin color recognition model is trained using the method described in any one of claims 1-5.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus. The processor executes the program instructions to perform the steps of the skin color recognition model training method as described in any one of claims 1 to 5, or the steps of the skin color recognition method as described in claim 6.

Images