Skin sensitivity identification method and system based on conditional generative adversarial network

Abstract

The invention discloses a skin sensitivity identification method and system based on a conditional generative adversarial network, and the method comprises the steps: constructing a pixel-to-pixel mapping relation data set according to a preset image mapping relation; inputting the collected real skin original image into a generative network structure Unet, and generating a false image with the same size as the skin original image; forming an image pair by the generated false image, the real skin original image and the real skin effect image, and inputting the image pair into a discriminator network PatchGAN to obtain a true and false image probability; calculating the loss Loss according to the output of the generator and the discriminator, and updating the network parameters of the generator and the discriminator by adopting a gradient descent method to obtain an image migration model; inputting a to-be-detected skin image into the image migration model to obtain a skin sensitive effect picture, and calculating a sensitive region area of the skin sensitive effect picture; and designing a skin sensitivity score mapping relation according to the area of the sensitive region, and evaluating the skin health state. The real-time performance is high, and the quality of the generated image is high.

Description

Technical Field

[0001] The present invention relates to the technical field of image migration through generative adversarial networks, and in particular discloses a skin sensitivity recognition method and system based on conditional generative adversarial networks. Background Art

[0002] Skin is the largest organ in the human body. It is composed of different layers, including the epidermis, dermis and subcutaneous tissue. The skin plays a protective role on our body and is also an important part of body aesthetics. Having healthy skin can increase self-confidence and personal charm. Skin sensitivity is one of the important indicators to measure skin health. Its main symptoms on the skin are: redness of the skin, fine scales, erythema, capillary dilation, etc. Normal skin and sensitive skin are similar in color, and it is difficult to produce a large visual difference, and it cannot reflect the sensitivity of the skin well. Therefore, being able to clearly distinguish normal skin from sensitive skin from a visual perspective is of great significance for skin health analysis.

[0003] The traditional method is to adjust based on the color card, and map the brightness value in the A channel after color space conversion to the color value of the corresponding level in the color card. This traditional method of adjusting through color cards has defects, such as being greatly affected by light, being unstable, having poor generalization ability, and having relatively high maintenance costs in the later stage.

[0004] In recent years, deep learning has made significant breakthroughs in many fields such as computer vision and natural language processing. The advantage of deep learning lies in the powerful feature extraction and feature learning capabilities of its deep convolutional neural network, which reduces human intervention compared to traditional manual feature extraction methods. In the method of finding skin sensitive areas based on convolutional neural networks, there are often problems such as long time consumption and the generated image effect does not meet expectations. Therefore, how to propose a method for finding human facial sensitivity and scalp sensitivity with strong real-time performance, good generalization ability, and high-quality generated images is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a skin sensitivity recognition method and system based on conditional generative adversarial network, aiming to solve at least one of the defects of the above-mentioned existing skin sensitivity recognition methods.

[0006] One aspect of the present invention relates to a skin sensitivity recognition method based on a conditional generative adversarial network, comprising the following steps:

[0007] According to the preset image mapping relationship, a pixel-to-pixel mapping relationship dataset is constructed;

[0008] The collected real skin original image is input into the generative network structure Unet to generate a fake image with the same size as the original skin image;

[0009] The generated fake image, the real skin original image and the real skin effect image are combined into image pairs, and the image pairs are input into the discriminator network PatchGAN to obtain the probability of true and false images;

[0010] According to the output of the generator and the discriminator, the loss Loss is calculated, and the network parameters of the generator and the discriminator are updated using the gradient descent method to obtain the image migration model;

[0011] Input the skin image to be detected into the image migration model to obtain a skin sensitivity effect map, and calculate the sensitive area of ​​the skin sensitivity effect map;

[0012] According to the area of ​​sensitive areas, a skin sensitivity score mapping relationship is designed to evaluate the skin health status.

[0013] Furthermore, according to the output of the generator and the discriminator, the loss Loss is calculated. In the step of updating the network parameters of the generator and the discriminator using the gradient descent method, the loss function uses the conditional CGAN as the loss function, and the calculation formula is as follows:

[0014] L cGAN (G,D)=E x,y [logD(x,y)]+E x [log(1-D(x,G(x))]

[0015] L L1 (G) = E x,y [||yG(x)|| 1 ]

[0016] L = argminmaxL cGAN (G,D)+λL L1 (G)

[0017] Among them, L cGAN (G, D) is the adversarial loss, which represents the loss of the generator and discriminator in the conditional generative adversarial network model; E x,y [logD(x,y)] represents the expected value of the discriminator’s probability of distinguishing the input original image from the real effect image, E x [log(1-D(x,G(x))] represents the expected value of the discriminator's discrimination probability between the input original image and the fake image generated by the generator, D(x,y) represents the discriminator's discrimination probability between the input real skin original image and the skin sensitive effect image processed by the traditional algorithm, D(x,G(x)) represents the discriminator's discrimination probability between the input real skin original image and the fake image generated by the generator, G represents the generator, D represents the discriminator; x is the input real skin original image, y is the skin sensitive effect image processed by the traditional algorithm;

[0018] L L1 (G) is the L1 loss, which represents the loss of the image generated after the G network in the conditional generative adversarial network model and y, using L L1 (G) is used to evaluate the distance between the generated image and the real skin sensitivity effect image; yG(x) represents the distance between the real effect image and the generated image;

[0019] L is the total loss of the conditional generative adversarial network model, λ is a constant, argminmaxL cGAN (G, D) means maximizing the loss of the discriminator and minimizing the loss of the generator.

[0020] Furthermore, the steps of inputting the skin image to be detected into the image migration model to obtain a skin sensitivity effect map, and calculating the sensitive area of ​​the skin sensitivity effect map include:

[0021] Input the skin image to be detected into the image migration model to obtain the face sensitivity effect map and the scalp sensitivity effect map;

[0022] The face sensitivity effect map and the scalp sensitivity effect map are converted into HSV color space; the red area is determined according to the hue, saturation and brightness range of red;

[0023] Separate the R channel of the image converted into the HSV color space, count the distribution of the R value of each pixel in the red area in the R channel, classify the area with an R value less than 140 as the dark-sensitive area, and classify the area with an R value greater than or equal to 140 as the light-sensitive area, and count the area proportion of the dark-sensitive area and the area proportion of the light-sensitive area;

[0024] The sum of the dark-sensitive area ratio and the light-sensitive area ratio to the power of 0.3 is scaled to obtain the scaled sensitivity score, and then the scaled sensitivity score is mapped to the corresponding skin health rating by comparing it with the threshold score of each rating.

[0025] Furthermore, the sum of the area ratio of the dark sensitive area and the area ratio of the light sensitive area to the power of 0.3 is scaled to obtain the scaled sensitivity score. Then, by comparing the scaled sensitivity score with the threshold score of each rating, it can be mapped to the corresponding skin health rating. In the above step, the calculation method of face and scalp sensitivity is as follows:

[0026] value 敏感 = light color sensitive area ratio + dark color sensitive area ratio 0.3

[0027] Among them, value 敏感 A score representing skin sensitivity.

[0028] Furthermore, the sum of the 0.3 power of the dark-sensitive area ratio and the light-sensitive area ratio is scaled to obtain the scaled sensitivity score, and then the scaled sensitivity score is mapped to the corresponding skin health rating by comparing the scaled sensitivity score with the threshold scores of each rating. The scaled skin sensitivity score distribution is evenly divided into 8 equal parts and mapped to skin sensitivity health ratings of 0 to 8, where 0 to 1 is moderate, 1 to 3 is mild, 3 to 5 is moderate, 5 to 7 is severe, and 7 to 8 is emergency; the mild threshold score is the score where rating 1 is located, the moderate threshold score is the score where rating 3 is located, the severe threshold score is the score where rating 5 is located, and the emergency threshold score is the score where rating 7 is located.

[0029] Another aspect of the present invention relates to a skin sensitivity recognition system based on a conditional generative adversarial network, comprising:

[0030] A data construction module is used to construct a pixel-to-pixel mapping relationship data set according to a preset image mapping relationship;

[0031] The image generation module is used to input the collected real skin original image into the generation network structure Unet to generate a fake image with the same size as the original skin image;

[0032] The image discrimination module is used to form image pairs with the generated fake images, the real skin original images and the real skin effect images, and input the image pairs into the discriminator network PatchGAN to obtain the probability of real and fake images;

[0033] The network training module is used to calculate the loss according to the output of the generator and the discriminator, and use the gradient descent method to update the network parameters of the generator and the discriminator to obtain the image migration model;

[0034] The model application module is used to input the skin image to be detected into the image migration model to obtain a skin sensitivity effect map and calculate the sensitive area of ​​the skin sensitivity effect map;

[0035] The evaluation module is used to design a skin sensitivity score mapping relationship according to the area of ​​sensitive areas and evaluate the skin health status.

[0036] Furthermore, in the network training module, the loss function uses conditional CGAN as the loss function, and the calculation formula is as follows:

[0037] L cGAN (G,D)=E x,y [logD(x,y)]+E x [log(1-D(x,G(x))]

[0038] L L1 (G) = Ex,y [||yG(x)|| 1 ]

[0039] L = argminmaxL cGAN (G,D)+λL L1 (G)

[0040] Among them, L cGAN (G, D) is the adversarial loss, which represents the loss of the generator and discriminator in the conditional generative adversarial network model; E x,y [logD(x,y)] represents the expected value of the discriminator’s probability of distinguishing the input original image from the real effect image, E x [log(1-D(x,G(x))] represents the expected value of the discriminator's discrimination probability between the input original image and the fake image generated by the generator, D(x,y) represents the discriminator's discrimination probability between the input real skin original image and the skin sensitive effect image processed by the traditional algorithm, D(x,G(x)) represents the discriminator's discrimination probability between the input real skin original image and the fake image generated by the generator, G represents the generator, D represents the discriminator; x is the input real skin original image, y is the skin sensitive effect image processed by the traditional algorithm;

[0041] L L1 (G) is the L1 loss, which represents the loss of the image generated after the G network in the conditional generative adversarial network model and y, using L L1 (G) is used to evaluate the distance between the generated image and the real skin sensitivity effect image; yG(x) represents the distance between the real effect image and the generated image;

[0042] L is the total loss of the conditional generative adversarial network model, λ is a constant, argminmaxL cGAN (G, D) means maximizing the loss of the discriminator and minimizing the loss of the generator.

[0043] Furthermore, the model application module includes:

[0044] An acquisition unit, used for inputting the skin image to be detected into the image migration model to obtain a face sensitivity effect map and a scalp sensitivity effect map;

[0045] A determination unit is used to convert the face sensitivity effect map and the scalp sensitivity effect map into HSV color space; and to determine the red area according to the hue, saturation and brightness range of red;

[0046] A conversion unit is used to separate the R channel of the image converted into the HSV color space, count the distribution of the R value of each pixel in the red area in the R channel, classify the area with an R value less than 140 as a dark-sensitive area, classify the area with an R value greater than or equal to 140 as a light-sensitive area, and count the area ratio of the dark-sensitive area and the area ratio of the light-sensitive area;

[0047] The rating unit is used to scale the sum of the area ratio of the dark-sensitive area and the 0.3 power of the area ratio of the light-sensitive area to obtain a scaled sensitivity score, and then map it to the corresponding skin health rating by comparing the scaled sensitivity score with the threshold score of each rating.

[0048] Furthermore, in the rating unit, the calculation method of face and scalp sensitivity is as follows:

[0049] value 敏感 = light color sensitive area ratio + dark color sensitive area ratio 0.3

[0050] Among them, value 敏感 A score representing skin sensitivity.

[0051] Furthermore, in the rating unit, the scaled skin sensitivity score distribution is divided into 8 equal parts and mapped to a skin sensitivity health rating of 0 to 8, where 0 to 1 is moderate, 1 to 3 is mild, 3 to 5 is moderate, 5 to 7 is severe, and 7 to 8 is emergency; the mild threshold score is the score where the rating is 1, the moderate threshold score is the score where the rating is 3, the severe threshold score is the score where the rating is 5, and the emergency threshold score is the score where the rating is 7.

[0052] The beneficial effects achieved by the present invention are:

[0053] The present invention provides a skin sensitivity recognition method and system based on a conditional generative adversarial network. According to a preset image mapping relationship, a pixel-to-pixel mapping relationship data set is constructed; the collected real skin original image is input into a generative network structure Unet to generate a fake image with the same size as the skin original image; the generated fake image, the real skin original image and the real skin effect image form an image pair, and the image pair is input into a discriminator network PatchGAN to obtain the probability of true and false images; according to the output of the generator and the discriminator, the loss Loss is calculated, and the network parameters of the generator and the discriminator are updated by the gradient descent method to obtain an image migration model; the skin image to be detected is input into the image migration model to obtain a skin sensitivity effect image, and the sensitive area of ​​the skin sensitivity effect image is calculated; according to the sensitive area, a skin sensitivity score mapping relationship is designed to evaluate the skin health status. The skin sensitivity recognition method and system based on conditional generative adversarial network provided by the present invention can stably generate one-to-one corresponding high-quality renderings by constructing a large number of pixel-to-pixel mapping data sets and combining with generative adversarial networks; the present invention distinguishes between sensitive areas and non-sensitive areas more accurately than traditional algorithms; the network structure of the present invention is relatively simple, and the calculation amount and complexity of the model are reduced by adjusting the image input size and the number of convolution kernels in the generative network; the Instance Normalization layer with higher complexity is abandoned, and the Batch Normalization layer with lower complexity is used, so that the model can be deployed in real time on various terminal devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of an embodiment of a method for skin sensitivity recognition based on conditional generative adversarial network of the present invention;

[0055] Figure 2 This is a schematic diagram of the network training process;

[0056] Figure 3 To generate the network structure Unet;

[0057] Figure 4 PatchGAN is the discriminator network;

[0058] Figure 5 is the sensitivity score distribution;

[0059] Figure 6 This is a comparison of the reasoning effect of the present invention on scalp pictures;

[0060] Figure 7 Comparison of the inference effect between the traditional algorithm and the present invention;

[0061] Figure 8The figure compares the reasoning effects of the present invention, deep convolutional neural networks, and traditional algorithms on face images. DETAILED DESCRIPTION

[0062] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0063] like Figure 1 and Figure 2 As shown, the first embodiment of the present invention proposes a skin sensitivity recognition method based on a conditional generative adversarial network, comprising the following steps:

[0064] Step S100: construct a pixel-to-pixel mapping relationship data set according to a preset image mapping relationship.

[0065] In actual application, the following data processing is performed specifically.

[0066] (1) Contrast enhancement. First, the EnhanceContras method is used to perform high contrast operations on the original face and scalp images. The enhancement factor aipha is set to 1.3 to increase the color and brightness differences between sensitive skin and normal skin. Then, the color space is converted from RGB (red, green, and blue) to LAB (color model). According to the display effects of the L, A, and B channel images, the channel image with the largest contrast between normal skin and sensitive skin is selected. Among them, the LAB color model consists of three elements, one element is brightness (L), and A and B are two color channels. A includes colors from dark green (low brightness value) to gray (medium brightness value) to bright pink (high brightness value). B is from bright blue (low brightness value) to gray (medium brightness value) and then to yellow (high brightness value).

[0067] (2) Segment the hair and skin regions. In order to reduce the impact of hair on the post-mapping effect, the hair and skin regions need to be segmented. Perform binary segmentation on the channel image in (1), use the OTSU method to adaptively segment the hair and skin regions, and extract the segmented hair and skin region images.

[0068] (3) Smooth image. Create a single-channel image, fill it with black, and keep the size the same as the image in (1). Then randomly fill each pixel of the single-channel image with a pixel value of 1 to 10, and add it to the channel image in (1) to get a smooth channel image.

[0069] (4) Skin area mapping. Based on the hair and skin areas extracted in (2), the hair and skin areas of the smooth channel image in (3) are extracted, and then different color cards are mapped to these two areas to eliminate hair interference. At the same time, sensitive skin of different degrees is mapped into different shades of red according to brightness, and sensitive skin and normal skin are given different colors and brightness.

[0070] (5) Fine-tuning. For the renderings in (4), if there are some mapping errors that are difficult to solve with algorithms, you can use Photoshop image processing software, using the brush or clone stamp tool, to correct the mapping errors in the renderings to make the effect closer to our expectations.

[0071] Step S200: input the collected real skin original image into the generation network structure Unet to generate a fake image with the same size as the original skin image.

[0072] The generated network structure Unet in step S200 is as follows: Figure 3 As shown, the characteristic is that the structure is like a U shape. It is divided into two parts, the left side is the encoder composed of 8 convolutional activation modules, and the right side is the decoder composed of 8 upsampling modules. In the encoder, each module contains a convolution layer and a rectified linear unit activation function; the size of the convolution kernel of the convolution layer is set to 4, the padding is 1, and the step size is 2; except for the last module which uses the Relu activation function, the other seven modules use the LeakReLU activation function. In the decoder, except for the first module which consists of a deconvolution layer and a normalization layer, and the last module which consists of a ReLU activation function, a deconvolution layer and a Tanh activation function, the other modules are composed of a ReLU activation function, a deconvolution layer and a normalization layer; the size of the deconvolution kernel of the deconvolution layer is set to 4, the padding is 1, and the step size is 2; the normalization layer uses BatchNormalization, which is a normalization operation performed on the batch dimension with low complexity.

[0073] The picture input passes through the first convolution activation module, the second convolution activation module, the third convolution activation module, the fourth convolution activation module, the fifth convolution activation module, the sixth convolution activation module, the seventh convolution activation module, the eighth convolution activation module, the first deconvolution normalization module, the first activated deconvolution normalization module, the second activated deconvolution normalization module, the third activated deconvolution normalization module, the fourth activated deconvolution normalization module, the fifth activated deconvolution normalization module, the sixth activated deconvolution normalization module, and the first activated deconvolution activation module.

[0074] Among them, the output end of the first convolution activation module is simultaneously connected to the output end of the sixth activated deconvolution normalization module. For the feature map output by the first convolution activation module, the feature map is updated by adding the feature map output by the sixth activated deconvolution normalization module in the channel dimension, and then the feature map is transmitted to the input end of the first activated deconvolution activation module; the output end of the second convolution activation module is simultaneously connected to the output end of the fifth activated deconvolution normalization module. For the feature map output by the second convolution activation module, the feature map is updated by adding the feature map output by the fifth activated deconvolution normalization module in the channel dimension. The feature map is updated, and the feature map is then transmitted to the input end of the sixth activated deconvolution normalization module; the output end of the third convolution activation module is simultaneously connected to the output end of the fourth activated deconvolution normalization module, and for the feature map output by the third convolution activation module, the feature map is updated by adding the feature map and the feature map output by the fourth activated deconvolution normalization module in the channel dimension, and then the feature map is transmitted to the input end of the fifth activated deconvolution normalization module; the output end of the fourth convolution activation module is simultaneously connected to the output end of the third activated deconvolution normalization module, and for the feature map output by the fourth convolution activation module, the feature map is updated by adding the feature map and the feature map output by the fourth activated deconvolution normalization module in the channel dimension. The feature map output by the third activated deconvolution normalization module is added to the channel dimension to update the feature map, and then the feature map is transmitted to the input end of the fourth activated deconvolution normalization module; the output end of the fifth convolution activation module is simultaneously connected to the output end of the second activated deconvolution normalization module, and for the feature map output by the fifth convolution activation module, the feature map is added to the feature map output by the second activated deconvolution normalization module in the channel dimension to update the feature map, and then the feature map is transmitted to the input end of the third activated deconvolution normalization module; the output end of the sixth convolution activation module is simultaneously connected to the output end of the first activated deconvolution normalization module. At the output end of the deconvolution normalization module, for the feature map output by the sixth convolution activation module, the feature map is updated by adding the feature map output by the first activated deconvolution normalization module in the dimension of the channel, and then the feature map is transmitted to the input end of the second activated deconvolution normalization module; the output end of the seventh convolution activation module is simultaneously connected to the output end of the first deconvolution normalization module, and for the feature map output by the seventh convolution activation module, the feature map is updated by adding the feature map output by the first deconvolution normalization module in the dimension of the channel, and then the feature map is transmitted to the input end of the first activated deconvolution normalization module.

[0075] In order to improve the inference speed of model deployment, the present invention sets the number of convolution kernels to 20 in the first module of the encoder to reduce the parameters of the model; in the decoder, the normalization layer uses the Batch Normalization layer instead of the Instance Normalization layer to reduce the complexity of the model; at the same time, the input size of the image is designed to be 768x768, which further reduces the amount of calculation without affecting the generated image effect, thereby achieving fast inference.

[0076] Step S300: The generated fake image, the real skin original image and the real skin effect image are combined into an image pair, and the image pair is input into the discriminator network PatchGAN to obtain the probability of the real and fake images.

[0077] The discriminator network PatchGAN in the above step S300 is composed of some convolution stacks, such as Figure 4 As shown in the figure, the discriminator processes the input image size into an NxN matrix (the size of N is related to the input size and the number of downsampling in the discriminator network). Each point in the matrix represents a small area Patch of the input image. The authenticity of each Patch is judged by each value in the matrix, and then the average value is taken as the authenticity probability of the entire image.

[0078] Regarding the design of the above PatchGAN network, except for the last module which is composed of a convolutional layer, the other four modules are composed of a convolutional layer, a normalization layer and a LeakReLU activation function; among them, the convolution kernel size of the first three modules is set to 4, the padding is 1, and the step size is 2, and the convolution kernel size of the last two modules is set to 4, the padding is 1, and the step size is 1; the normalization layer uses Batch Normalization.

[0079] In practical applications, the "fake" image generated by the generative network Unet forms an image pair with the original image and the effect image. The input passes through the first convolution normalization activation module, the second convolution normalization activation module, the third convolution normalization activation module, the fourth convolution normalization activation module, and the first convolution module in sequence.

[0080] Step S400: Calculate the loss Loss according to the outputs of the generator and the discriminator, and use the gradient descent method to update the network parameters of the generator and the discriminator to obtain the image migration model.

[0081] In the above step S400, during the network training process, relevant parameters are set until the network converges and the network training weights are saved, as follows:

[0082] (1) Set the initial learning rate for training, the number of iterations of the initial learning rate, the number of iterations at which the initial learning rate linearly decays to zero, and the number of training batches.

[0083] (2) During the training process, data augmentation (flipping, cropping, and scaling) is used.

[0084] (3) Adan is used as the optimizer. During the optimization process, conditional CGAN is used as the loss function. The calculation formula is as follows:

[0085] L cGAN (G,D)=E x,y [logD(x,y)]+E x [log(1-D(x,G(x))]

[0086] L L1 (G) = E x,y [||yG(x)|| 1 ]

[0087] L = argminmaxL cGAN (G,D)+λL L1 (G)

[0088] Among them, L cGAN (G, D) is the adversarial loss, which represents the loss of the generator and discriminator in the conditional generative adversarial network model; E x,y [logD(x,y)] represents the expected value of the discriminator’s probability of distinguishing the input original image from the real effect image, E x [log(1-D(x,G(x))] represents the expected value of the discriminator's discrimination probability between the input original image and the fake image generated by the generator, D(x,y) represents the discriminator's discrimination probability between the input real skin original image and the skin sensitive effect image processed by the traditional algorithm, D(x,G(x)) represents the discriminator's discrimination probability between the input real skin original image and the fake image generated by the generator, G represents the generator, D represents the discriminator; x is the input real skin original image, y is the skin sensitive effect image processed by the traditional algorithm;

[0089] L L1 (G) is the L1 loss, which represents the loss of the image generated after the G network in the conditional generative adversarial network model and y, using L L1 (G) is used to evaluate the distance between the generated image and the real skin sensitivity effect image; yG(x) represents the distance between the real effect image and the generated image;

[0090] L is the total loss of the conditional generative adversarial network model, λ is a constant, argminmaxL cGAN (G, D) means maximizing the loss of the discriminator and minimizing the loss of the generator.

[0091] (4) Repeat steps (2) and (3) until the network converges and the image migration model is obtained.

[0092] Step S500: input the skin image to be detected into the image migration model to obtain a skin sensitivity effect map, and calculate the sensitive area of ​​the skin sensitivity effect map.

[0093] The specific operations are as follows:

[0094] (1) Input the skin image to be detected into the image migration model to obtain the face sensitivity effect map and the scalp sensitivity effect map.

[0095] (2) The face sensitivity effect map and the scalp sensitivity effect map are converted into HSV color space; and the red area is determined according to the hue, saturation and brightness range of red.

[0096] (3) Separate the R channel of the image converted to the HSV color space, and count the distribution of the R value of each pixel in the red area in the R channel. Classify the area with an R value less than 140 as a dark-sensitive area, and classify the area with an R value greater than or equal to 140 as a light-sensitive area. Count the area ratio of the dark-sensitive area and the area ratio of the light-sensitive area.

[0097] Step S600: design a skin sensitivity score mapping relationship according to the area of ​​the sensitive region to evaluate the skin health status.

[0098] In the above step S600, the face and scalp sensitivity calculation method is as follows:

[0099] value 敏感 = light color sensitive area ratio + dark color sensitive area ratio 0.3

[0100] Among them, value 敏感 Represents the score of skin sensitivity. 敏感 Scaling is performed to obtain the final sensitivity score. The present invention scales the sensitivity score to between 0 and 100, such as Figure 5 As shown, it conforms to the normal distribution. The distribution is divided into 8 equal parts and mapped to the skin sensitivity health rating of 0 to 8, where 0 to 1 is moderate, 1 to 3 is mild, 3 to 5 is moderate, 5 to 7 is severe, and 7 to 8 is emergency. The mild threshold score is the score where the rating is 1, the moderate threshold score is the score where the rating is 3, the severe threshold score is the score where the rating is 5, and the emergency threshold score is the score where the rating is 7. By comparing the scaled sensitivity score with the threshold score of each rating, it can be mapped to the corresponding health rating.

[0101] By applying the above design in practice, the present invention can stably generate high-quality renderings with one-to-one correspondence. Figure 6 As shown in the figure, (a) is the actual scalp image, and (b) is the inference image of the present invention. It can be seen that the effect image and the sensitive area in the original image are basically one-to-one corresponding. Compared with the traditional algorithm, the generalization of the present invention is better. Figure 7 As shown, (c) is the inference diagram of the traditional image algorithm, (d) is the actual scalp shot, and (e) is the inference diagram of the present invention. It can be seen that the effect generated by the present invention is more stable and more generalizable. As for the performance of sensitivity in the thermal map, the effect of the present invention is also more delicate and accurate than that of the traditional algorithm. In some subtle places, such as the sensitive area ① and the non-sensitive area ② in the actual scalp shot (g), the distinction between the sensitive area and the non-sensitive area of ​​the present invention is more accurate than that of the traditional algorithm.

[0102] Reasoning about face images, such as Figure 8 As shown, compared with convolutional neural networks and traditional algorithms, the images generated by the model in the present invention are better.

[0103] The present invention relates to a skin sensitivity recognition system based on a conditional generative adversarial network, comprising a data construction module, an image generation module, an image discrimination module, a network training module, a model application module and an evaluation module, wherein the data construction module is used to construct a pixel-to-pixel mapping relationship data set according to a preset image mapping relationship; the image generation module is used to input the collected real skin original image into a generation network structure Unet to generate a fake image with the same size as the skin original image; the image discrimination module is used to form an image pair with the generated fake image, the real skin original image and the real skin effect image, and input the image pair into a discriminator network PatchGAN to obtain the probability of true and false images; the network training module is used to calculate the loss Loss according to the output of the generator and the discriminator, and use the gradient descent method to update the network parameters of the generator and the discriminator to obtain an image migration model; the model application module is used to input the skin image to be detected into the image migration model to obtain a skin sensitivity effect image, and calculate the sensitive area of ​​the skin sensitivity effect image; the evaluation module is used to design a skin sensitivity score mapping relationship according to the sensitive area to evaluate the skin health status.

[0104] Further, see Figures 1 to 8 In the skin sensitivity recognition system based on conditional generative adversarial network provided in this embodiment, in the network training module, the loss function uses conditional CGAN as the loss function, and the calculation formula is as follows:

[0105] L cGAN (G,D)=E x,y [logD(x,y)]+E x[log(1-D(x,G(x))]

[0106] L L1 (G) = E x,y [||yG(x)|| 1 ]

[0107] L = arg min max L cGAN (G,D)+λL L1 (G)

[0108] Among them, L cGAN (G, D) is the adversarial loss, which represents the loss of the generator and discriminator in the conditional generative adversarial network model; E x,y [logD(x,y)] represents the expected value of the discriminator’s probability of distinguishing the input original image from the real effect image, E x [log(1-D(x,G(x))] represents the expected value of the discriminator's discrimination probability between the input original image and the fake image generated by the generator, D(x,y) represents the discriminator's discrimination probability between the input real skin original image and the skin sensitive effect image processed by the traditional algorithm, D(x,G(x)) represents the discriminator's discrimination probability between the input real skin original image and the fake image generated by the generator, G represents the generator, D represents the discriminator; x is the input real skin original image, y is the skin sensitive effect image processed by the traditional algorithm;

[0109] L L1 (G) is the L1 loss, which represents the loss of the image generated after the G network in the conditional generative adversarial network model and y, using L L1 (G) is used to evaluate the distance between the generated image and the real skin sensitivity effect image; yG(x) represents the distance between the real effect image and the generated image;

[0110] L is the total loss of the conditional generative adversarial network model, λ is a constant, argminmaxL cGAN (G, D) means maximizing the loss of the discriminator and minimizing the loss of the generator.

[0111] Preferably, see Figures 1 to 8The skin sensitivity recognition system based on conditional generative adversarial network provided in this embodiment, the model application module includes an acquisition unit, a determination unit, a conversion unit and a rating unit, wherein the acquisition unit is used to input the skin image to be detected into the image migration model to obtain a face sensitivity effect map and a scalp sensitivity effect map; the determination unit is used to perform HSV color space conversion on the face sensitivity effect map and the scalp sensitivity effect map; the red area is determined according to the hue, saturation and brightness range of red; the conversion unit is used to separate the R channel of the image converted in the HSV color space, count the distribution of the R value of each pixel point in the red area in the R channel, classify the area with an R value less than 140 as a dark sensitive area, classify the area with an R value greater than or equal to 140 as a light sensitive area, and count the area proportion of the dark sensitive area and the area proportion of the light sensitive area; the rating unit is used to scale the sum of the 0.3 power of the area proportion of the dark sensitive area and the area proportion of the light sensitive area to obtain the scaled sensitivity score, and then by comparing the scaled sensitivity score with the threshold score of each rating, it can be mapped to the corresponding skin health rating.

[0112] Further, see Figures 1 to 8 In the skin sensitivity recognition system based on conditional generative adversarial network provided in this embodiment, the calculation method of face and scalp sensitivity in the rating unit is as follows:

[0113] value 敏感 = light color sensitive area ratio + dark color sensitive area ratio 0.3

[0114] Among them, value 敏感 A score representing skin sensitivity.

[0115] Preferably, see Figures 1 to 8 In the skin sensitivity recognition system based on conditional generative adversarial network provided in this embodiment, in the rating unit, the scaled skin sensitivity score distribution is evenly divided into 8 equal parts, mapped to a skin sensitivity health rating of 0 to 8, where 0 to 1 is moderate, 1 to 3 is mild, 3 to 5 is moderate, 5 to 7 is severe, and 7 to 8 is emergency; the mild threshold score is the score where the rating is 1, the moderate threshold score is the score where the rating is 3, the severe threshold score is the score where the rating is 5, and the emergency threshold score is the score where the rating is 7.

[0116] The present embodiment provides a skin sensitivity recognition method and system based on a conditional generative adversarial network. According to a preset image mapping relationship, a pixel-to-pixel mapping relationship data set is constructed; the collected real skin original image is input into the generative network structure Unet to generate a fake image with the same size as the skin original image; the generated fake image, the real skin original image and the real skin effect image form an image pair, and the image pair is input into the discriminator network PatchGAN to obtain the probability of true and false images; according to the output of the generator and the discriminator, the loss Loss is calculated, and the network parameters of the generator and the discriminator are updated by the gradient descent method to obtain an image migration model; the skin image to be detected is input into the image migration model to obtain a skin sensitivity effect map, and the sensitive area of ​​the skin sensitivity effect map is calculated; according to the sensitive area, a skin sensitivity score mapping relationship is designed to evaluate the skin health status. The skin sensitivity recognition method and system based on conditional generative adversarial network provided in this embodiment can stably generate one-to-one corresponding high-quality renderings by constructing a large number of pixel-to-pixel mapping data sets and combining with generative adversarial networks; the distinction between sensitive areas and non-sensitive areas in this embodiment is more accurate than traditional algorithms; the network structure of the present invention is relatively simple, and the calculation amount and complexity of the model are reduced by adjusting the image input size and the number of convolution kernels in the generative network; the Instance Normalization layer with higher complexity is abandoned, and the Batch Normalization layer with lower complexity is used, so that the model can be deployed in real time on various terminal devices.

[0117] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A skin sensitivity recognition method based on conditional generative adversarial network, characterized in that: The following steps are involved: According to the preset image mapping relationship, a pixel-to-pixel mapping relationship dataset is constructed; Input the collected real skin original image into the generation network structure Unet to generate a fake image with the same size as the original skin image; The generated fake image, the real skin original image and the real skin effect image are combined into an image pair, and the image pair is input into the discriminator network PatchGAN to obtain the probability of true and false images; According to the output of the generator and the discriminator, the loss Loss is calculated, and the network parameters of the generator and the discriminator are updated using the gradient descent method to obtain the image migration model; Inputting the skin image to be detected into the image migration model to obtain a skin sensitivity effect map, and calculating the sensitive region area of ​​the skin sensitivity effect map; According to the area of ​​sensitive areas, a skin sensitivity score mapping relationship is designed to evaluate the skin health status.

2. The skin sensitivity recognition method based on conditional generative adversarial network according to claim 1, characterized in that: In the step of calculating the loss Loss according to the output of the generator and the discriminator, and updating the network parameters of the generator and the discriminator by the gradient descent method to obtain the image migration model, the loss function uses the conditional CGAN as the loss function, and the calculation formula is as follows: L cGAN (G,D)=E x,y [logD(x,y)]+E x [log(1-D(x,G(x))] L L1 (G)=E x,y [||y-G(x)||1] L=arg min maxL cGAN (G,D)+λL L1 (G) Among them, L cGAN (G, D) is the adversarial loss, which represents the loss of the generator and discriminator in the conditional generative adversarial network model; E x,y [logD(x,y)] represents the expected value of the discriminator’s probability of distinguishing the input original image from the real effect image, E x [log(1-D(x,G(x))] represents the expected value of the discriminator's discrimination probability between the input original image and the fake image generated by the generator, D(x,y) represents the discriminator's discrimination probability between the input real skin original image and the skin sensitive effect image processed by the traditional algorithm, D(x,G(x)) represents the discriminator's discrimination probability between the input real skin original image and the fake image generated by the generator, G represents the generator, D represents the discriminator; x is the input real skin original image, y is the skin sensitive effect image processed by the traditional algorithm; L L1 (G) is the L1 loss, which represents the loss of the image generated after the G network in the conditional generative adversarial network model and y, using L L1 (G) is used to evaluate the distance between the generated image and the real skin sensitivity effect image; yG(x) represents the distance between the real effect image and the generated image; L is the total loss of the conditional generative adversarial network model, λ is a constant, argminmaxL cGAN (G, D) means maximizing the loss of the discriminator and minimizing the loss of the generator.

3. The skin sensitivity recognition method based on conditional generative adversarial network according to claim 1, characterized in that: The step of inputting the skin image to be detected into the image migration model to obtain a skin sensitivity effect map, and calculating the sensitive region area of ​​the skin sensitivity effect map comprises: Inputting the skin image to be detected into the image migration model to obtain a face sensitivity effect map and a scalp sensitivity effect map; The face sensitivity effect map and the scalp sensitivity effect map are converted into HSV color space; and a red area is determined according to the hue, saturation and brightness range of red; Separate the R channel of the image converted into the HSV color space, count the distribution of the R value of each pixel in the red area in the R channel, classify the area with an R value less than 140 as the dark-sensitive area, and classify the area with an R value greater than or equal to 140 as the light-sensitive area, and count the area proportion of the dark-sensitive area and the area proportion of the light-sensitive area; The sum of the area ratio of the dark-sensitive area and the area ratio of the light-sensitive area to the power of 0.3 is scaled to obtain a scaled sensitivity score, and then the scaled sensitivity score is compared with the threshold scores of each rating to map it to the corresponding skin health rating.

4. The skin sensitivity recognition method based on conditional generative adversarial network according to claim 3, characterized in that: The sum of the area ratio of the dark sensitive area and the area ratio of the light sensitive area to the power of 0.3 is scaled to obtain the scaled sensitivity score, and then the scaled sensitivity score is compared with the threshold score of each rating to map it to the corresponding skin health rating. The calculation method of face and scalp sensitivity is as follows: value 敏感 = light color sensitive area ratio + dark color sensitive area ratio 0.3 Among them, value 敏感 A score representing skin sensitivity.

5. The skin sensitivity recognition method based on conditional generative adversarial network according to claim 4, characterized in that: The sum of the 0.3 power of the dark-sensitive area ratio and the light-sensitive area ratio is scaled to obtain a scaled sensitivity score, and then the scaled sensitivity score is compared with the threshold scores of each rating to map it to the corresponding skin health rating. The scaled skin sensitivity score distribution is evenly divided into 8 equal parts and mapped to a skin sensitivity health rating of 0 to 8, where 0 to 1 is moderate, 1 to 3 is mild, 3 to 5 is moderate, 5 to 7 is severe, and 7 to 8 is emergency; the mild threshold score is the score where rating 1 is located, the moderate threshold score is the score where rating 3 is located, the severe threshold score is the score where rating 5 is located, and the emergency threshold score is the score where rating 7 is located.

6. A skin sensitivity recognition system based on conditional generative adversarial network, characterized in that: include: A data construction module is used to construct a pixel-to-pixel mapping relationship data set according to a preset image mapping relationship; An image generation module is used to input the collected real skin original image into the generation network structure Unet to generate a fake image with the same size as the original skin image; An image discrimination module, used to combine the generated fake image, the real skin original image and the real skin effect image into an image pair, and input the image pair into a discriminator network PatchGAN to obtain the probability of a true or false image; The network training module is used to calculate the loss Loss according to the output of the generator and the discriminator, and use the gradient descent method to update the network parameters of the generator and the discriminator to obtain the image migration model; A model application module, used for inputting the skin image to be detected into the image migration model to obtain a skin sensitivity effect map, and calculating the sensitive area of ​​the skin sensitivity effect map; The evaluation module is used to design a skin sensitivity score mapping relationship according to the area of ​​sensitive areas and evaluate the skin health status.

7. The skin sensitivity recognition system based on conditional generative adversarial network according to claim 6, characterized in that: In the network training module, The loss function uses conditional CGAN as the loss function, and the calculation formula is as follows: L cGAN (G,D)=E x,y [logD(x,y)]+E x [log(1-D(x,G(x))] L L1 (G)=E x,y [||y-G(x)||1] L=arg min maxL cGAN (G,D)+λL L1 (G) Among them, L cGAN (G, D) is the adversarial loss, which represents the loss of the generator and discriminator in the conditional generative adversarial network model; E x,y [logD(x,y)] represents the expected value of the discriminator’s probability of distinguishing the input original image from the real effect image, E x [log(1-D(x,G(x))] represents the expected value of the discriminator's discrimination probability between the input original image and the fake image generated by the generator, D(x,y) represents the discriminator's discrimination probability between the input real skin original image and the skin sensitive effect image processed by the traditional algorithm, D(x,G(x)) represents the discriminator's discrimination probability between the input real skin original image and the fake image generated by the generator, G represents the generator, D represents the discriminator; x is the input real skin original image, y is the skin sensitive effect image processed by the traditional algorithm; L L1 (G) is the L1 loss, which represents the loss of the image generated after the G network in the conditional generative adversarial network model and y, using L L1 (G) is used to evaluate the distance between the generated image and the real skin sensitivity effect image; yG(x) represents the distance between the real effect image and the generated image; L is the total loss of the conditional generative adversarial network model, λ is a constant, argminmaxL cGAN (G, D) means maximizing the loss of the discriminator and minimizing the loss of the generator.

8. The skin sensitivity recognition system based on conditional generative adversarial network according to claim 6, characterized in that: The model application module includes: An acquisition unit, used for inputting the skin image to be detected into the image migration model to obtain a face sensitivity effect map and a scalp sensitivity effect map; A determination unit, configured to perform HSV color space conversion on the face sensitivity effect map and the scalp sensitivity effect map; and determine a red area according to the hue, saturation and brightness range of red; A conversion unit is used to separate the R channel of the image converted into the HSV color space, count the distribution of the R value of each pixel in the red area in the R channel, classify the area with an R value less than 140 as a dark-sensitive area, classify the area with an R value greater than or equal to 140 as a light-sensitive area, and count the area ratio of the dark-sensitive area and the area ratio of the light-sensitive area; The rating unit is used to scale the sum of the 0.3 power of the dark-sensitive area ratio and the light-sensitive area ratio to obtain a scaled sensitivity score, and then map the scaled sensitivity score to the corresponding skin health rating by comparing it with the threshold score of each rating.

9. The skin sensitivity recognition system based on conditional generative adversarial network according to claim 8, characterized in that: In the rating unit, the calculation method of face and scalp sensitivity is as follows: value 敏感 = light color sensitive area ratio + dark color sensitive area ratio 0.3 Among them, value 敏感 A score representing skin sensitivity.

10. The skin sensitivity recognition system based on conditional generative adversarial network according to claim 9, characterized in that: In the rating unit, the scaled skin sensitivity score distribution is evenly divided into 8 equal parts and mapped to a skin sensitivity health rating of 0 to 8, where 0 to 1 is moderate, 1 to 3 is mild, 3 to 5 is moderate, 5 to 7 is severe, and 7 to 8 is emergency; the mild threshold score is the score where the rating is 1, the moderate threshold score is the score where the rating is 3, the severe threshold score is the score where the rating is 5, and the emergency threshold score is the score where the rating is 7.

Images