Skin color adjustment image processing method, device, equipment and storage medium

Abstract

The application provides an image processing method and device for skin color adjustment, equipment and a storage medium. The method comprises the following steps: obtaining a face skin image and a skin color feature map corresponding to each original face image respectively; a skin color transformation model is built, the skin color transformation model comprising a channel fusion layer for fusing the face skin image and the skin color feature map; and the skin color transformation model is trained according to the face skin image and the skin color feature map corresponding to each original face image. The face skin image and the skin color feature map of the original face image are automatically obtained to train the skin color transformation model, so that the result of the skin color transformation model is more natural and closer to the real face skin color. The trained skin color adjustment model can complete skin replacement only through one forward propagation, reduces the time consumption of coloring, and can meet the rapid processing requirement of a mobile terminal. The skin color of the face image can be adjusted to a given skin color value or the skin color of a target face image, and the skin color transformation is more flexible.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an image processing method and device for skin color adjustment, equipment and a storage medium. BACKGROUND

[0002] Images with human beings as the main body often need to process skin color in the image processing process to achieve the beauty effect of skin color adjustment. For example, skin color adjustment can be whitening of the face region in a human face image.

[0003] At present, a skin color adjustment method is provided in the related technology. A large number of human face images are obtained, and a result image after skin color adjustment corresponding to each human face image is obtained through artificial retouching. A network model for skin color adjustment is trained by using the human face images and the result images obtained through artificial retouching. A human face image to be adjusted and a target human face image are input into the trained network model, and the skin color of the human face image to be adjusted is adjusted to the skin color in the target human face image by the network model.

[0004] However, the above-mentioned related technology trains the model by using the result images obtained through artificial retouching. Artificial retouching costs a large amount of manpower and time cost, and the result images obtained through artificial retouching have poor effect of skin color adjustment, and are prone to uneven and unnatural skin color. SUMMARY

[0005] The present application provides an image processing method and device for skin color adjustment, equipment and a storage medium. The face skin images and skin color feature maps of original human face images are automatically obtained to train a skin color transformation model, so that the result of skin color adjustment of the skin color transformation model is more natural and closer to the real human face skin color. The whole process does not need human participation, has high automation degree, and the skin color transformation is more natural and efficient.

[0006] The first aspect embodiment of the present application provides an image processing method for skin color adjustment, comprising:

[0007] Respectively obtaining a face skin image and a skin color feature map corresponding to each original human face image;

[0008] Building a skin color transformation model, wherein the skin color transformation model comprises a channel fusion layer for fusing the face skin image and the skin color feature map;

[0009] Training the skin color transformation model according to the face skin image and the skin color feature map corresponding to each original human face image.

[0010] In some embodiments of the present application, the method for respectively obtaining a face skin image corresponding to each original human face image comprises:

[0011] remove pixel parts of the five organs and the hair from a first original face image, the first original face image being any one of the original face images;

[0012] set each pixel point belonging to the skin in the first original face image after the removing operation as a preset pixel value to obtain a face skin image corresponding to the first original face image.

[0013] In some embodiments of the present application, a skin color feature map corresponding to each original face image is obtained respectively, comprising:

[0014] select a preset number of skin image blocks without blank pixel parts from the first original face image after the removing operation;

[0015] fuse the preset number of skin image blocks into a skin color feature map corresponding to the first original face image.

[0016] In some embodiments of the present application, the fusing of the preset number of skin image blocks into the skin color feature map corresponding to the first original face image comprises:

[0017] select two skin image blocks from the preset number of skin image blocks;

[0018] construct a covariance matrix corresponding to the two skin image blocks;

[0019] calculate weights corresponding to the two skin image blocks respectively according to the covariance matrix;

[0020] fuse the two skin image blocks into one fused image block according to the weights corresponding to the two skin image blocks;

[0021] select a new skin image block from the preset number of skin image blocks, and according to the fused image block and the new skin image block, the step of constructing the covariance matrix is executed cyclically until the preset number of skin image blocks are fused into one image block to obtain the skin color feature map corresponding to the first original face image.

[0022] In some embodiments of the present application, the constructing of the skin color transformation model comprises:

[0023] connect the up-sampling layer and the channel fusion layer in sequence;

[0024] replace each de-convolution layer included in the generator in the preset image translation model with a structure in which a nearest size adjustment layer is connected with a first convolution layer to obtain a new generator;

[0025] connect the channel fusion layer, the new generator and a discriminator included in the preset image translation model in sequence;

[0026] The discriminator is connected with a loss layer, and the loss layer adopts a style loss function.

[0027] In some embodiments of the present application, the training of the skin color transformation model according to the facial skin image and the skin color feature map corresponding to each original face image comprises:

[0028] In the current training cycle, the skin color feature map corresponding to the original face image is scaled to the same size as the facial skin image corresponding to the original face image through the up-sampling layer;

[0029] The skin color feature map and the corresponding facial skin image with the same size are aligned in the channel through the channel fusion layer, and a channel fusion feature map corresponding to the original face image is obtained;

[0030] The channel fusion feature map is input into the new generator to generate a skin-changing result image corresponding to the original face image;

[0031] The skin-changing result image and the channel fusion feature map are input into the discriminator for discrimination to obtain a discrimination probability corresponding to the original face image;

[0032] According to the original face image, the facial skin image, the skin color feature map, the skin-changing result image and the discrimination probability, a loss value corresponding to the current training cycle is calculated through the loss layer.

[0033] In some embodiments of the present application, the calculation of the loss value corresponding to the current training cycle according to the original face image, the facial skin image, the skin color feature map, the skin-changing result image and the discrimination probability through the loss layer comprises:

[0034] According to the original face image, the facial skin image, the skin color feature map, the skin-changing result image and the discrimination probability, an image conversion loss value is calculated through an image conversion loss function included in the loss layer;

[0035] According to the original face image, the facial skin image, the skin color feature map and the skin-changing result image, a minimum distance loss value is calculated through a minimum distance loss function included in the loss layer;

[0036] According to the original face image and the skin-changing result image, a style loss value is calculated through the style loss function included in the loss layer;

[0037] According to the image conversion loss value, the minimum distance loss value and the style loss value, the loss value corresponding to the current training cycle is calculated.

[0038] In some embodiments of the present application, the method further comprises:

[0039] If the current number of trained cycles is greater than the preset number of times, a training cycle with the minimum loss value is determined, and the model parameters corresponding to the training cycle with the minimum loss value and the structure of the skin color transformation model are determined as the trained skin color transformation model.

[0040] If the current number of trained cycles is less than or equal to the preset number of times, the model parameters are adjusted according to the loss value corresponding to the current training cycle, and the next cycle of training is performed on the skin color transformation model according to the adjusted model parameters.

[0041] In some embodiments of the present application, before the face skin image and the skin color feature map corresponding to each original face image are obtained respectively, the method further comprises:

[0042] randomly selecting one original face image from each original face image as a standard face image;

[0043] identifying the coordinates of the pixel points of a preset part in the standard face image;

[0044] aligning the coordinates of the pixel points of the preset part in the second original face image with the coordinates of the pixel points of the preset part in the standard face image, the second original face image being any original face image other than the standard face image among the original face images;

[0045] adjusting the size of each original face image after the alignment operation to a preset size supported by the skin color transformation model.

[0046] In some embodiments of the present application, after the skin color transformation model is trained according to the face skin image and the skin color feature map corresponding to each original face image, the method further comprises:

[0047] receiving a skin color transformation request, the skin color transformation request including a face image to be transformed and a specified skin color value;

[0048] generating a skin color feature map to be transformed according to the specified skin color value;

[0049] obtaining a face skin image corresponding to the face image to be transformed;

[0050] adjusting the skin color of the face image to be transformed to the specified skin color value through the trained skin color transformation model according to the skin color feature map to be transformed and the face skin image.

[0051] In some embodiments of the present application, after the skin color transformation model is trained according to the face skin image and the skin color feature map corresponding to each original face image, the method further includes:

[0052] receiving a skin color transformation request, the skin color transformation request including a face image to be transformed and a target face image;

[0053] obtaining a skin color feature map corresponding to the target face image;

[0054] obtaining a face skin image corresponding to the face image to be transformed;

[0055] adjusting the skin color of the face image to be transformed to the skin color corresponding to the target face image through the trained skin color transformation model according to the skin color feature map corresponding to the target face image and the face skin image corresponding to the face image to be transformed.

[0056] Embodiments of the second aspect of the present application provide an image processing device for skin color adjustment, comprising:

[0057] a data acquisition module configured to acquire a face skin image and a skin color feature map corresponding to each original face image, respectively;

[0058] a model building module configured to build a skin color transformation model, the skin color transformation model including a channel fusion layer for fusing the face skin image and the skin color feature map;

[0059] a model training module configured to train the skin color transformation model according to the face skin image and the skin color feature map corresponding to each original face image.

[0060] Embodiments of the third aspect of the present application provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of the first aspect.

[0061] Embodiments of the fourth aspect of the present application provide a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method of the first aspect.

[0062] The technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0063] In the embodiments of the present application, the original face image does not need to be artificially retouched to obtain a result image of skin color adjustment, but a face skin image and a skin color feature map of the original face image are automatically obtained based on the original face image, the face skin image and the skin color feature map of the original face image are used to train a skin color transformation model, so that the skin color transformation model learns based on the original face image, so that the result of skin color adjustment by the skin color transformation model is more natural and closer to the real face skin color.

[0064] Further, when the trained skin color adjustment model is used for skin color adjustment, only one forward propagation is needed to complete the skin replacement, which reduces the time consumption of coloring and meets the rapid processing demand of the mobile terminal. Moreover, the skin color of the face image to be transformed can be adjusted to a color corresponding to a given skin color value, or the skin color of the face image to be transformed can be adjusted to the skin color in the target face image, so that the skin color transformation is more flexible. From the training of the skin color transformation model to the application, no manual intervention is needed, the degree of automation is high, and the skin color transformation is more natural and efficient.

[0065] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0066] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the present application. Moreover, the same reference numerals are used throughout the same figures to designate similar or equivalent components.

[0067] In the drawings:

[0068] Figure 1 A flowchart of a skin color adjustment image processing method provided by an embodiment of the present application is shown;

[0069] Figure 2 A structural schematic diagram of a skin color transformation model provided by an embodiment of the present application is shown;

[0070] Figure 3 Another flowchart of a skin color adjustment image processing method provided by an embodiment of the present application is shown;

[0071] Figure 4 A structural schematic diagram of a skin color adjustment image processing device provided by an embodiment of the present application is shown;

[0072] Figure 5 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown;

[0073] Figure 6A schematic diagram of a storage medium provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0074] Exemplary embodiments of the present application will be described in detail with reference to the drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0075] It should be noted that, unless otherwise specified, technical terms or scientific terms used in the present application should be understood as their common meanings to those skilled in the art to which the present application pertains.

[0076] A skin color adjustment image processing method, device, equipment and storage medium are described below according to an embodiment of the present application.

[0077] Some skin color adjustment image processing methods are proposed in the related art. An automatic sample image color transfer algorithm establishes a correspondence between sample blocks by calculating the texture features and brightness statistics of each sub-block. This algorithm is prone to cause false matching between non-skin color and skin color regions. The processing object of an image coloring algorithm based on color transfer and expansion is a grayscale image, which cannot be applied to daily skin coloring scenarios. The related art also proposes a scheme for adjusting skin color using a network model. This scheme obtains a result image after skin color adjustment corresponding to a face image through artificial retouching, and trains a network model using these face images and the result image obtained through artificial retouching. A face image to be adjusted and a target face image are input into the trained network model, and the skin color of the face image to be adjusted is adjusted to the skin color in the target face image through the network model. However, artificial retouching costs a large amount of manpower and time, and the trained model using the result image obtained through artificial retouching has poor skin color adjustment effect and is prone to cause uneven and unnatural skin color. In addition, the above various related technologies are based on skin color transformation of the target face to the original face, and cannot randomly select colors for transformation according to user needs, and the brightness and color distribution of the transformation result are greatly affected by the brightness and color distribution of the target face.

[0078] Based on the above problems existing in the related art, the embodiment of the present application provides an image processing method for skin color adjustment. The method does not need to manually retouch the original face image to obtain a result image of skin color adjustment, but automatically obtains a face skin image and a skin color feature map of the original face image based on the original face image, trains a skin color transformation model using the face skin image and the skin color feature map of the original face image, and makes the skin color transformation model learn based on the original face image, so that the result of skin color adjustment by the skin color transformation model is more natural and closer to the real face skin color. Moreover, when the trained skin color adjustment model is used for skin color adjustment, only one forward propagation is needed to complete the skin replacement, which reduces the time consumption of coloring and can meet the rapid processing demand of mobile terminals. From the training of the skin color transformation model to the application, no manual intervention is needed, the automation degree is high, and the skin color transformation is more natural and efficient.

[0079] Referring to Figure 1 The method specifically includes the following steps:

[0080] Step 101: Obtain a face skin image and a skin color feature map corresponding to each original face image, respectively.

[0081] First, a large number of original face images are obtained. The original face image is an image of a front face, and the image is clear and without watermark. The face is free of hair, eyes or masks and other obstructions. A shooting device can be used to shoot different faces to obtain the original face image. The shooting process cannot be shaken, and a shooting device with high resolution can be used to shoot to obtain a clear original face image. For example, the resolution of the shooting device can be more than 1 million pixels. The embodiment of the present application can also obtain a large number of original face images from the Internet.

[0082] After a large number of original face images are obtained, the obtained original face images need to be standardized. Specifically, a standard face image is randomly selected from each original face image. The coordinates of the pixel points of the preset parts in the standard face image are identified. The preset parts can include parts such as eyes, nose and mouth. The coordinates of the pixel points of the preset parts can include the coordinates of the pixel points at positions such as left and right eyes, nose tip and left and right corners of the mouth. For each original face image other than the standard face image among the obtained large number of original face images, any original face image other than the standard face image is referred to as a second original face image for ease of description. The coordinates of the pixel points of the above-mentioned preset parts in the second original face image are identified, and then the coordinates of the pixel points of the preset parts of the second original face image are aligned with the coordinates of the pixel points of the preset parts in the standard face image.

[0083] In a specific implementation, a program can be written to call the dlib library to identify the coordinates of the pixel points of the preset parts in the standard face image and each second original face image. The standard face image is kept unchanged, and then the second original face image is deformed, stretched, or compressed so that the coordinates of the pixel points of the preset parts in the second original face image are aligned with the coordinates of the pixel points of the preset parts in the standard face image. That is, the coordinates of the pixel points of the left and right eyes in the second original face image are moved to the coordinates of the pixel points of the left and right eyes in the standard face image, the coordinates of the pixel points of the tip of the nose in the second original face image are moved to the coordinates of the pixel points of the tip of the nose in the standard face image, the coordinates of the pixel points of the left and right corners of the mouth in the second original face image are moved to the coordinates of the pixel points of the left and right corners of the mouth in the standard face image, and so on.

[0084] Through the alignment operation, the face in each original face image is located at the same position in the image, and the model training error caused by the different positions of the face in the image is reduced, and the accuracy of subsequent training of the skin color transformation model is improved.

[0085] Since the skin color transformation model in the embodiment of the present application has its supported image size, the size of each original face image after the alignment operation is also adjusted to the preset size supported by the skin color transformation model. The preset size can be 572*572, 620*620, etc. The embodiment of the present application does not limit the specific value of the preset size, and in actual application, the preset size can be set according to the actual situation of the image size supported by the skin color transformation model.

[0086] After each original face image obtained in the above manner is aligned and scaled to the preset size, the face skin image and the skin color feature map of each original face image are obtained, respectively. The face skin image only includes the skin part in the original face image and does not include the facial features and hair part. The skin color feature map is an image block, and the pixel value of the pixel point in the image block can represent the skin color of the face in the original face image. Since the face skin image and the skin color feature map of each original face image are obtained in the same way, the embodiment of the present application only takes the first original face image as an example for detailed description, and the first original face image is any original face image in each original face image after the standardization operation.

[0087] For the facial skin image, first remove the pixel part of the five organs and the hair from the first original face image. Specifically, the first original face image can be segmented using a semantic segmentation network (such as BiSeNet) to remove the pixel points belonging to the five organs and the hair in the first original face image. After the removal operation, the first original face image has blank pixels at the five organs and the hair. Then, each pixel point belonging to the skin in the first original face image after the removal operation is set to a preset pixel value to obtain the facial skin image corresponding to the first original face image. That is, the pixel value of each pixel point after the removal operation except the blank pixels is set to a preset pixel value, and the preset pixel value is a pixel value corresponding to any color except white, such as a pixel value (0, 255, 0) corresponding to green or a pixel value (255, 0, 0) corresponding to red.

[0088] For each original face image, the facial skin image corresponding to each original face image is obtained in the above manner.

[0089] By removing the pixels not belonging to the skin in the original face image and then setting the pixel values of all pixel points belonging to the skin to the same value, the facial skin image obtained in this way can clearly and completely present the entire pixel area occupied by the facial skin in the original face image, making the area requiring skin color transformation more accurate, avoiding confusion between non-skin areas and skin areas, improving the accuracy of skin replacement, and making the skin color transformation more natural.

[0090] For the skin color feature map corresponding to the original face image, the first original face image obtained by removing the pixel part of the five organs and the hair is selected to obtain a preset number of skin image blocks not containing blank pixels. Specifically, a preset number of skin image blocks of equal size can be randomly selected from the first original face image after removing the five organs and the hair. If the selected skin image block includes blank pixels formed by removing the five organs or the hair, it is reselected until a preset number of skin image blocks of equal size and not containing blank pixels are selected. The size of the selected skin image block is a pre-set size, such as 28*28 or 30*30, etc. The preset number can be 8 or 9, etc.

[0091] The embodiments of the present application do not limit the specific values of the size and the preset number of the above-mentioned skin image block, which can be set according to the needs in actual application.

[0092] After the skin image blocks are selected in the above-mentioned manner, the preset number of selected skin image blocks are fused into a skin color feature map corresponding to the first original face image. In the embodiments of the present application, morphological Top-hat transformation algorithm, mean fusion algorithm, logarithmic mean fusion algorithm, etc. image fusion technology can be used to fuse the preset number of skin image blocks into one image block.

[0093] The embodiment of the present application only details the process of fusing images by using morphological Top-hat transform algorithm, which fuses images by adaptively calculating the weight in PCA (Principal Component Analysis) fusion and fusing images by using the weight. Specifically, two skin image blocks are selected from a preset number of skin image blocks. A covariance matrix corresponding to the two skin image blocks is constructed. First, an image matrix composed of the two skin image blocks is constructed by formula (1). Then, the covariance matrix corresponding to the image matrix is calculated by formula (2).

[0094] V = [sub_block_i, sub_block_j] … (1)

[0095]

[0096] In formula (1) and (2), sub_block_i is a skin image block i, sub_block_j is a skin image block j, V is an image matrix composed of the skin image block i and the skin image block j. C is a covariance matrix corresponding to the image matrix V, is a covariance of the skin image block i and itself, is a covariance of the skin image block i and the skin image block j, is a covariance of the skin image block j and the skin image block i, is a covariance of the skin image block j and itself.

[0097] The covariance elements in the above covariance matrix can be calculated by formula (3) as follows,

[0098]

[0099] In formula (3), is an arithmetic mean of pixel values of the skin image block i, is an arithmetic mean of pixel values of the skin image block j, x il is a pixel value of an lth pixel point in the skin image block i, x jl is a pixel value of an lth pixel point in the skin image block j.

[0100] After the covariance matrix of the two skin image blocks is calculated by the above method, the weights corresponding to the two skin image blocks are calculated according to the covariance matrix. First, the eigenvalues and the eigenvectors corresponding to each eigenvalue of the covariance matrix are calculated by the eigenvalue formula shown in formula (4).

[0101] |λI-C| = 0 … (4)

[0102] In formula (4), C is a covariance matrix, I is an eigenvector, and λ is an eigenvalue.

[0103] The plurality of eigenvalues corresponding to the covariance matrix and the eigenvector corresponding to each eigenvalue are calculated by formula (4). The maximum eigenvalue is selected from the plurality of calculated eigenvalues, and the eigenvector corresponding to the maximum eigenvalue is selected. The weights corresponding to the two skin image blocks are calculated by formula (5) according to the eigenvector corresponding to the maximum eigenvalue.

[0104]

[0105] In formula (5), ω1 and ω2 are the weights corresponding to the two skin image blocks selected above, respectively, and (ξ i1 , ξ i2 ) is the eigenvector corresponding to the maximum eigenvalue.

[0106] After the weights corresponding to the two skin image blocks are calculated in the above manner, the two skin image blocks are fused into a fused image block by formula (6) according to the weights corresponding to the two skin image blocks.

[0107] VF = ω1 * sub_block_i + ω2 * sub_block_j… (6)

[0108] In formula (6), VF is the fused image block obtained by fusion, sub_block_i is the skin image block i, sub_block_j is the skin image block j, ω1 is the weight of the skin image block i, and ω2 is the weight of the skin image block j.

[0109] After the two skin image blocks are fused into an image block in the above manner, a new skin image block is selected from the other skin image blocks in the preset number of skin image blocks, excluding the fused skin image blocks. According to the fused image block and the selected new skin image block, the step of constructing the covariance matrix is executed in a loop until the preset number of skin image blocks are fused into an image block, and the skin color feature map corresponding to the first original face image is obtained.

[0110] That is, the preset number of skin image blocks are fused one by one in the above manner, and finally an image block is obtained, which is taken as the skin color feature map corresponding to the first original face image. For each original face image, the skin color feature map corresponding to each original face image is also generated in this way.

[0111] The morphological Top-hat transformation algorithm is used in the above process of generating the skin color feature map to fuse the images. The algorithm performs image fusion based on the correlation between the selected component images, so that the data of the shared features in the two skin image blocks to be fused is compressed, and the data of the unique features is enhanced, so that the skin color feature map obtained finally retains the skin color features in the original face image to the greatest extent, thereby making the skin color transformation effect of the skin color transformation model trained and learned based on the skin color feature map closer to the real face skin color.

[0112] Step 102: Building a skin color transformation model, which includes a channel fusion layer for fusing a face skin image and a skin color feature map.

[0113] As shown in Figure 2 The upsampling layer and the channel fusion layer are connected in sequence. The upsampling layer includes a nearest-neighbor resizing layer and a second convolutional layer connected in sequence. Each deconvolutional layer included in the generator of the preset image translation model is replaced by a structure in which a nearest-neighbor resizing layer is connected to a first convolutional layer, to obtain a new generator. The preset image translation model in the embodiment of the present application can adopt a network structure such as Pix2Pix or Pix2PixHD. The network structure such as Pix2Pix or Pix2PixHD includes a generator and a discriminator. The generator usually includes one or more deconvolutional layers. Using a deconvolutional layer to generate an image will cause a checkerboard effect in which color blocks are unevenly overlapped in the resulting image, so that the color of some parts in the resulting image is darker than that of other parts, resulting in an unnatural skin color transformation effect. Therefore, in the embodiment of the present application, each deconvolutional layer in the generator of the network structure such as Pix2Pix or Pix2PixHD is replaced by a structure in which a nearest-neighbor resizing layer is connected to a first convolutional layer. The structure in which the nearest-neighbor resizing layer is connected to the first convolutional layer can effectively solve the problem of the checkerboard effect in the generated image.

[0114] After the above replacement operation, the channel fusion layer, the new generator, and the discriminator included in the preset image translation model are connected in sequence. Finally, the discriminator is connected to a loss layer, and the loss layer adopts a style loss function.

[0115] In the embodiment of the present application, the evolution process of the loss function adopted by the loss layer is as follows:

[0116] (1) The loss function adopted in the preset image translation model (such as Pix2Pix) is formula (7),

[0117]

[0118] In formula (7), x is the facial skin image, y is the original human face image, z is the skin color feature map, G(x, z) is the skin-changing result image corresponding to the original human face image generated by the generator of the preset translation model, and D is the probability determined by the discriminator of the preset translation model.

[0119] As can be seen from formula (7), the objective of the discriminator D is to make the value of formula (7) as large as possible, and the optimization objective of the generator G is to make log(1-D(x, G(x, z)) in formula (7) as small as possible. In order to avoid the discriminator D being too strong, resulting in the generator G being unable to be trained, the embodiment of the present application adopts a deformation of formula (7), that is, an image conversion loss function shown in formula (8),

[0120]

[0121] In formula (8), the optimization objective of the generator G is changed to make log(D(x, G(x, z)) as large as possible.

[0122] (2) In order to constrain whether the generated image and the real image are close, that is, to minimize the distance between the two, the preset image translation model (such as Pix2Pix) selects L1 loss, that is, a minimum distance loss function shown in formula (9),

[0123]

[0124] (3) In order to make the skin-changing result image consistent with the style of the original human face image, the embodiment of the present application introduces a style loss function. Generally, the shallow network extracts local detail texture features, and the deep network extracts more abstract contour, size and other information. The feeling shown by the combination of these features is the style of the image. The Gram matrix calculated from these feature vectors can extract the hidden relationship between image features, that is, the correlation between features. If the difference between the Gram matrices of the feature vectors of two images is small, it can be determined that the styles of the two images are similar.

[0125] First, the Gram matrix of the original human face image is calculated by formula (10) as follows, and the Gram matrix of the skin-changing result image corresponding to the original human face image is calculated by formula (11).

[0126]

[0127]

[0128] wherein, is the output of the original face image y at the (i, j, k) position of the lth layer of the discriminator. Where i, j, k correspond to height, width and channel respectively, and k' is another channel different from k. l specifies the second-to-last convolutional layer of the discriminator.

[0129] According to the Gram matrix of the original face image and the Gram matrix of the skin changing result image, the final style loss function can be defined as formula (12):

[0130]

[0131] (4), combining formula (8), (9) and (12), the final loss function used in the loss layer of the skin color transformation model in the embodiment of the application is formula (13):

[0132]

[0133] After the network structure of the skin color transformation model is built and the loss function used in the loss layer is determined through the operation of the present step, the model training is performed through the operation of the following step 103.

[0134] Step 103: training the skin color transformation model according to the face skin image and the skin color feature map corresponding to each original face image.

[0135] The skin color transformation model can process multiple face skin images and skin color feature maps corresponding to multiple original face images at the same time, and the number of images that can be processed at the same time can be referred to as batch size. In the current training period, the face skin images and the skin color feature maps corresponding to the batch size of the original face images of the skin color transformation model are obtained from the face skin images and the skin color feature maps corresponding to a large number of original face images obtained from step 101. Each face skin image and the corresponding skin color feature map obtained are input into the up-sampling layer in the skin color transformation model, and the skin color feature map corresponding to the original face image is scaled to the same size as the face skin image corresponding to the original face image through the up-sampling layer. For example, the size of the face skin image corresponding to the original face image is 572*572*3, where 3 represents the RGB three channels. And the size of the skin color feature map corresponding to the original face image is 28*28*3. Then the up-sampling layer scales the size of the skin color feature map to 572*572*3.

[0136] The face skin image and the skin color feature map of the same size processed by the up-sampling layer are input into a channel fusion layer, the channel fusion layer aligns the channels of the skin color feature map and the corresponding face skin image of the same size, and a channel fusion feature map corresponding to the original face image is obtained. For example, for the face skin image and the skin color feature map both of which have a size of 572*572*3, the channel fusion layer splices the two to obtain a channel fusion feature map with a size of 572*572*6.

[0137] In the embodiment of the present application, the up-sampling layer and the channel fusion layer are added before the new generator in the skin color transformation model, so that the skin color transformation model can learn the information of the area occupied by the face skin and the skin color at the same time. The up-sampling layer includes a nearest resizing layer and a second convolutional layer connected in sequence, which well solves the problem of size mismatch between the skin color feature map and the face skin image. The size of the skin color feature map is adjusted by the nearest resizing layer first, and then the second convolutional layer is used for processing, so that the skin color transformation model can learn how to better enlarge the skin color feature map during the training process.

[0138] After the channel alignment by the channel fusion layer, the obtained channel fusion feature map is input into the new generator in which the deconvolutional layer is replaced, and a skin-changing result image corresponding to the original face image is generated by the new generator. The new generator transforms the pixel values of the pixel points in the face skin image included in the channel fusion feature map into the pixel values of the pixel points in the skin color feature map. Since all the pixel values of the pixel points in the face skin image are the same, the pixel values of the pixel points in the skin color feature map represent the real skin color of the original face image. Therefore, the new generator generates the skin-changing result image, which is equivalent to adjusting the color of the pixel area occupied by the skin in the original face image to the skin color of the original face image itself.

[0139] After the new generator generates the skin-changing result image, the skin-changing result image and the channel fusion feature map are input into the discriminator for discrimination, and a discrimination probability corresponding to the original face image is obtained. Finally, according to the original face image, the face skin image, the skin color feature map, the skin-changing result image and the discrimination probability, a loss value corresponding to the current training period is calculated by the loss layer.

[0140] Specifically, according to the original face image, the facial skin image, the skin color feature map, the skin changing result image and the discrimination probability, the image conversion loss value is calculated by the image conversion loss function included in the loss layer. That is, the image conversion loss value is calculated by the above formula (8). Then, according to the original face image, the facial skin image, the skin color feature map and the skin changing result image, the minimum distance loss value is calculated by the minimum distance loss function included in the loss layer. That is, the minimum distance loss value is calculated by the above formula (9). According to the original face image and the skin changing result image, the style loss value is calculated by the style loss function included in the loss layer. That is, the style loss value is calculated by the above formula (12). Finally, according to the image conversion loss value, the minimum distance loss value and the style loss value, the loss value corresponding to the current training period is calculated by the above formula (13).

[0141] After the training of the current training period is completed in the above manner, the current trained period number is increased by one. It is judged whether the trained period number after being increased by one is greater than the preset number. If yes, the training period with the minimum loss value is determined from the trained periods, and the model parameters corresponding to the training period with the minimum loss value and the structure of the skin color transformation model are determined as the trained skin color transformation model. If the current trained period number is less than or equal to the preset number, the model parameters are adjusted according to the loss value corresponding to the current training period. Then, according to the adjusted model parameters, the next period of training of the skin color transformation model is performed according to the above training process.

[0142] The preset number can be 300 or 500, etc. The embodiments of the present application do not limit the specific value of the preset number, which can be set according to the requirements in actual application.

[0143] After the trained skin color transformation model is obtained in the above manner, the skin color transformation model can be used to adjust the skin color of a user's face image. In some embodiments of the present application, a skin color value can be given, and the skin color of a face image to be transformed is adjusted to the given skin color value by the skin color transformation model. Specifically, the embodiments of the present application can provide a virtual color card for a user, which includes a plurality of colors. When the user needs to adjust the skin color of a face image, a color can be selected from the color card as a target color to be adjusted, and a skin color transformation request is sent to the server. The server receives the skin color transformation request, which includes a face image to be transformed and a specified skin color value corresponding to the target color selected by the user. According to the specified skin color value, a skin color feature map to be transformed is generated. Specifically, a blank image block is created, which has the same size as the skin color feature map in step 101. Then the pixel value of each pixel point in the blank image block is set to the specified skin color value included in the skin color transformation request. Then the face skin image corresponding to the face image to be transformed is obtained, and the obtaining process is the same as the obtaining method of the face skin image in step 101, which will not be described here. Then, according to the skin color feature map to be transformed and the face skin image, the skin color of the face image to be transformed is adjusted to the specified skin color value by the trained skin color transformation model.

[0144] Specifically, the skin color feature map corresponding to the specified skin color value and the face skin image corresponding to the face image to be transformed are input into the up-sampling layer of the trained skin color transformation model, the skin color feature map corresponding to the specified skin color value is scaled to the same size as the face skin image corresponding to the face image to be transformed by the up-sampling layer, and then the skin color feature map and the face skin image with the same size are aligned in the channel by the channel fusion layer, to obtain a corresponding channel fusion feature map. Then, the new generator included in the skin color transformation model generates a skin color adjusted skin transformation result image according to the channel fusion feature map.

[0145] In some other embodiments of the present application, a target face image can also be given, and the skin color of a face image to be transformed is adjusted to the skin color of the target face image by the trained skin color transformation model. Specifically, a skin color transformation request is received, which includes a face image to be transformed and a target face image. The skin color feature map corresponding to the target face image is obtained in the manner of obtaining the skin color feature map in step 101. And the face skin image corresponding to the face image to be transformed is obtained in the manner of obtaining the face skin image in step 101. According to the skin color feature map corresponding to the target face image and the face skin image corresponding to the face image to be transformed, the skin color of the face image to be transformed is adjusted to the skin color corresponding to the target face image by the trained skin color transformation model.

[0146] Specifically, the skin color feature map corresponding to the target face image and the face skin image corresponding to the face image to be transformed are input into the up-sampling layer of the trained skin color transformation model, the skin color feature map corresponding to the target face image is scaled to the same size as the face skin image corresponding to the face image to be transformed through the up-sampling layer, and then the skin color feature map and the face skin image of the same size are aligned in the channel through the channel fusion layer to obtain a corresponding channel fusion feature map. Then, the new generator included in the skin color transformation model generates a skin color adjusted skin changing result image according to the channel fusion feature map.

[0147] Since the skin color transformation model is trained based on the original face image, the skin color adjustment of the face image through the skin color transformation model can make the skin changing result image more natural. Moreover, only one forward propagation of the skin color transformation model is needed during the skin color conversion, which greatly reduces the time consumption of the skin color transformation and is suitable for rapid processing on mobile terminals. Moreover, only one skin color value needs to be given, and the skin color of the face image to be transformed can be adjusted to the color corresponding to the given skin color value. The skin color of the face image to be transformed can also be adjusted to the skin color in the target face image, making the skin color transformation more flexible.

[0148] In order to facilitate understanding of the skin color adjustment scheme of the embodiments of the present application, the following will be described in conjunction with the drawings. As shown in the drawings, the face skin image and the skin color feature map are obtained from the original face image, and then the skin color transformation model is trained using the obtained face skin image and skin color feature map. In application, the face image to be transformed and the specified skin color value or the target face image are input into the trained skin color transformation model, and the skin changing result image can be obtained. Figure 3

[0149] In the embodiments of the present application, the original face image does not need to be artificially retouched to obtain the result image of skin color adjustment, but the face skin image and the skin color feature map of the original face image are automatically obtained based on the original face image, and the face skin image and the skin color feature map of the original face image are used to train the skin color transformation model, so that the skin color transformation model learns based on the original face image, thereby making the result of the skin color adjustment of the skin color transformation model more natural and closer to the real face skin color. Moreover, only one forward propagation is needed to complete the skin changing when the trained skin color adjustment model is used for skin color adjustment, which reduces the time consumption of coloring and can meet the rapid processing requirements of mobile terminals. Moreover, the skin color of the face image to be transformed can be adjusted to the color corresponding to the given skin color value. The skin color of the face image to be transformed can also be adjusted to the skin color in the target face image, making the skin color transformation more flexible. From the training of the skin color transformation model to the application, no manual intervention is needed, the degree of automation is high, and the skin color transformation is more natural and efficient.

[0150] ​The embodiment of the present application further provides an image processing device for skin color adjustment, which is used for executing the image processing method for skin color adjustment provided by any of the above embodiments. Referring to Figure 4 The device comprises:

[0151] The data acquisition module 401 is configured to acquire the facial skin image and the skin color feature map corresponding to each original face image, respectively.

[0152] The model building module 402 is configured to build a skin color transformation model, wherein the skin color transformation model comprises a channel fusion layer used for fusing the facial skin image and the skin color feature map.

[0153] The model training module 403 is configured to train the skin color transformation model according to the facial skin image and the skin color feature map corresponding to each original face image.

[0154] The data acquisition module 401 is configured to remove the pixel part of the facial features and the hair from the first original face image, wherein the first original face image is any of the original face images; and set each pixel point belonging to the skin in the first original face image after the removal operation to a preset pixel value, so as to obtain the facial skin image corresponding to the first original face image.

[0155] The data acquisition module 401 is configured to select a preset number of skin image blocks not containing a blank pixel part from the first original face image after the removal operation; and fuse the preset number of skin image blocks into the skin color feature map corresponding to the first original face image.

[0156] The data acquisition module 401 is configured to select two skin image blocks from the preset number of skin image blocks; construct a covariance matrix corresponding to the two skin image blocks; calculate the weight corresponding to the two skin image blocks according to the covariance matrix, respectively; fuse the two skin image blocks into one fused image block according to the weight corresponding to the two skin image blocks; select a new skin image block from the preset number of skin image blocks, and perform the steps of constructing the covariance matrix again according to the fused image block and the new skin image block, until the preset number of skin image blocks are fused into one image block, so as to obtain the skin color feature map corresponding to the first original face image.

[0157] The model building module 402 is configured to sequentially connect the up-sampling layer and the channel fusion layer; replace each deconvolution layer included in the generator in the preset image translation model with a structure in which the nearest size adjustment layer is connected with the first convolution layer, so as to obtain a new generator; sequentially connect the channel fusion layer, the new generator and the discriminator included in the preset image translation model; and connect the discriminator with the loss layer, wherein the loss layer adopts a style loss function.

[0158] The model training module 403 is configured to, in the current training period, scale the skin color feature map corresponding to the original face image to the same size as the face skin image corresponding to the original face image through an upsampling layer; perform channel alignment on the skin color feature map and the face skin image of the same size through a channel fusion layer to obtain a channel fusion feature map corresponding to the original face image; input the channel fusion feature map into a new generator to generate a skin-changing result image corresponding to the original face image; input the skin-changing result image and the channel fusion feature map into a discriminator for discrimination to obtain a discrimination probability corresponding to the original face image; and calculate a loss value corresponding to the current training period through a loss layer according to the original face image, the face skin image, the skin color feature map, the skin-changing result image and the discrimination probability.

[0159] The model training module 403 is configured to calculate an image conversion loss value through an image conversion loss function included in the loss layer according to the original face image, the face skin image, the skin color feature map, the skin-changing result image and the discrimination probability; calculate a minimum distance loss value through a minimum distance loss function included in the loss layer according to the original face image, the face skin image, the skin color feature map and the skin-changing result image; calculate a style loss value through a style loss function included in the loss layer according to the original face image and the skin-changing result image; and calculate the loss value corresponding to the current training period according to the image conversion loss value, the minimum distance loss value and the style loss value.

[0160] The model training module 403 is configured to, if the current number of trained periods is greater than the preset number of times, determine a training period with the minimum loss value, and determine the model parameters corresponding to the training period with the minimum loss value and the structure of the skin color transformation model as the trained skin color transformation model; if the current number of trained periods is less than or equal to the preset number of times, adjust the model parameters according to the loss value corresponding to the current training period; and perform the next period of training on the skin color transformation model according to the adjusted model parameters.

[0161] The device further includes an image alignment module configured to randomly select one original face image as a standard face image from each original face image; identify the coordinates of the pixel points of a preset part in the standard face image; align the coordinates of the pixel points of the preset part of a second original face image with the coordinates of the pixel points of the preset part in the standard face image, the second original face image being any original face image other than the standard face image among the original face images; and adjust the size of each original face image after the alignment operation to a preset size supported by the skin color transformation model.

[0162] The device further comprises a skin color adjustment module configured to receive a skin color transformation request, the skin color transformation request comprising a face image to be transformed and a specified skin color value; generate a skin color feature map of the face image to be transformed according to the specified skin color value; obtain a facial skin image corresponding to the face image to be transformed; and adjust the skin color of the face image to be transformed to the specified skin color value by using the trained skin color transformation model according to the skin color feature map of the face image to be transformed and the facial skin image.

[0163] The skin color adjustment module can also be configured to receive a skin color transformation request, the skin color transformation request comprising a face image to be transformed and a target face image; obtain a skin color feature map corresponding to the target face image; obtain a facial skin image corresponding to the face image to be transformed; and adjust the skin color of the face image to be transformed to the skin color corresponding to the target face image by using the trained skin color transformation model according to the skin color feature map corresponding to the target face image and the facial skin image corresponding to the face image to be transformed.

[0164] The image processing device for skin color adjustment provided by the above embodiments of the present application and the image processing method for skin color adjustment provided by the embodiments of the present application have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0165] The embodiments of the present application also provide an electronic device for executing the image processing method for skin color adjustment described above. Please refer to Figure 5 which shows a schematic diagram of an electronic device provided by some embodiments of the present application. As shown in Figure 5 The electronic device 5 comprises a processor 500, a memory 501, a bus 502 and a communication interface 503, the processor 500, the communication interface 503 and the memory 501 are connected through the bus 502; the memory 501 stores a computer program executable on the processor 500, and the processor 500 executes the computer program to perform the image processing method for skin color adjustment provided by any of the preceding embodiments of the present application.

[0166] The memory 501 can include a high-speed random access memory (RAM: Random Access Memory) and can also include a non-volatile memory such as at least one disk memory. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 503 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0167] The bus 502 can be an ISA bus, a PCI bus, an EISA bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. The memory 501 is configured to store programs, and the processor 500 executes the programs after receiving execution instructions. The skin color adjustment image processing method disclosed in any of the embodiments of the present application can be applied to the processor 500 or implemented by the processor 500.

[0168] The processor 500 can be an integrated circuit chip with processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 500 or an instruction in the form of software. The processor 500 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), and the like; or can be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step, and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory 501, and the processor 500 reads the information in the memory 501 and combines the hardware to complete the steps of the above method.

[0169] The electronic device provided by the embodiments of the present application and the skin color adjustment image processing method provided by the embodiments of the present application have the same beneficial effects as the method they adopt, run, or implement.

[0170] The embodiments of the present application also provide a computer readable storage medium corresponding to the skin color adjustment image processing method provided by the preceding embodiments. Please refer to Figure 6 The computer readable storage medium shown is an optical disc 30, and a computer program (i.e., a program product) is stored on the optical disc 30. When the computer program is run by a processor, the skin color adjustment image processing method provided by any of the preceding embodiments is executed.

[0171] It should be noted that examples of the computer-readable storage medium can also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical, magnetic storage mediums, and the like, which are not listed one by one here.

[0172] The computer-readable storage medium provided by the above embodiments of the present application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, with the skin color adjusted image processing method provided by the embodiments of the present application having the same beneficial effects.

[0173] It should be noted that:

[0174] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known structures and techniques are not shown in detail in order not to obscure the understanding of the present specification.

[0175] Similarly, it should be appreciated that the individual features of the application described in the above description of example embodiments of the application are not to be construed as requirements or limitations of the application, but rather as instances of implementing the same. Thus, the application is not limited to the embodiments described above, which are presented as examples only but rather can be modified in various ways within the scope of protection defined by the appended claims.

[0176] Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a transitional phrase thereof. Also, the term "consisting essentially of' is intended to be inclusive of the elements recited in the claim, and is not meant to be exclusive of additional elements that do not materially affect the basic and novel characteristics of the claimed application. The term "consisting of' is intended to be exclusive of any element not recited in the claim.

[0177] The above description is merely illustrative of the application, and is not to be taken in a limiting sense. It is being contemplated that various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the application. Accordingly, the scope of the application should be determined by the scope of the claims and their equivalents.

Claims

1. A method of image processing for skin color adjustment, characterized by, The method comprises the following steps: remove the pixel parts of the five organs and the hair from the first original face image, wherein the first original face image is any one of each original face image; obtain the facial skin image and the skin color feature map corresponding to each original face image respectively; build a skin color transformation model, wherein the skin color transformation model comprises a channel fusion layer for fusing the facial skin image and the skin color feature map; train the skin color transformation model according to the facial skin image and the skin color feature map corresponding to each original face image; wherein, the step of obtaining the skin color feature map corresponding to each original face image respectively comprises: select a preset number of skin image blocks without blank pixel parts from the first original face image after the removal operation; select two skin image blocks from the preset number of skin image blocks; construct a covariance matrix corresponding to the two skin image blocks; calculate the weights corresponding to the two skin image blocks respectively according to the covariance matrix; fuse the two skin image blocks into a fused image block according to the weights corresponding to the two skin image blocks; select a new skin image block from the preset number of skin image blocks, and start the step of constructing the covariance matrix again according to the fused image block and the new skin image block, until the preset number of skin image blocks are fused into an image block, and the skin color feature map corresponding to the first original face image is obtained.

2. The method of claim 1, wherein, The step of obtaining the facial skin image corresponding to each original face image respectively comprises: set each pixel point belonging to the skin in the first original face image after the removal operation to a preset pixel value to obtain the facial skin image corresponding to the first original face image.

3. The method of claim 1, wherein, The step of building a skin color transformation model comprises: connect the up-sampling layer and the channel fusion layer in sequence; replace each deconvolution layer included in the generator of the preset image translation model with the structure of the nearest size adjustment layer connected with the first convolution layer to obtain a new generator; connect the channel fusion layer, the new generator and the discriminator included in the preset image translation model in sequence; connect the discriminator with a loss layer, and the loss layer adopts a style loss function.

4. The method of claim 3, wherein, The step of training the skin color transformation model according to the facial skin image and the skin color feature map corresponding to each original face image comprises: in the current training period, scale the skin color feature map corresponding to the original face image to the same size as the facial skin image corresponding to the original face image through the up-sampling layer; align the channels of the skin color feature map and the corresponding facial skin image with the same size through the channel fusion layer to obtain a channel fusion feature map corresponding to the original face image; input the channel fusion feature map into the new generator to generate a skin-changing result image corresponding to the original face image; input the skin-changing result image and the channel fusion feature map into the discriminator for discrimination to obtain a discrimination probability corresponding to the original face image; According to the original face image, the face skin image, the skin color feature map, the skin changing result map and the discrimination probability, a loss value corresponding to the current training period is calculated through the loss layer.

5. The method of claim 4, wherein, According to the original face image, the face skin image, the skin color feature map, the skin changing result map and the discrimination probability, a loss value corresponding to the current training period is calculated through the loss layer. According to the original face image, the face skin image, the skin color feature map, the skin changing result map and the discrimination probability, an image conversion loss value is calculated through an image conversion loss function included in the loss layer. According to the original face image, the face skin image, the skin color feature map and the skin changing result map, a minimum distance loss value is calculated through a minimum distance loss function included in the loss layer. According to the original face image and the skin changing result map, a style loss value is calculated through the style loss function included in the loss layer. According to the image conversion loss value, the minimum distance loss value and the style loss value, a loss value corresponding to the current training period is calculated.

6. The method of claim 4, wherein, The method further comprises: If the current number of trained periods is greater than a preset number, a training period with a minimum loss value is determined, and a model parameter corresponding to the training period with the minimum loss value and a structure of the skin color transformation model are determined as a trained skin color transformation model; If the current number of trained periods is less than or equal to the preset number, a model parameter is adjusted according to the loss value corresponding to the current training period, and the skin color transformation model is trained in the next period according to the adjusted model parameter.

7. The method according to any one of claims 1 to 6, characterized in that, Before the face skin image and the skin color feature map corresponding to each original face image are respectively obtained, the method further comprises: randomly selecting one original face image from each original face image as a standard face image; recognizing coordinates of pixel points of a preset part in the standard face image; aligning the coordinates of the pixel points of the preset part of the second original face image with the coordinates of the pixel points of the preset part in the standard face image, the second original face image being any original face image except the standard face image among the original face images; adjusting the size of each original face image after the alignment operation to a preset size supported by the skin color transformation model.

8. The method according to any one of claims 1 to 6, characterized in that, After the skin color transformation model is trained according to the face skin image and the skin color feature map corresponding to each original face image, the method further comprises: receiving a skin color transformation request, the skin color transformation request comprising a face image to be transformed and a specified skin color value; generating a skin color feature map to be transformed according to the specified skin color value; obtaining a face skin image corresponding to the face image to be transformed; adjusting the skin color of the face image to be transformed to the specified skin color value through the trained skin color transformation model according to the skin color feature map to be transformed and the face skin image.

9. The method according to any one of claims 1 to 6, characterized in that, After the skin color transformation model is trained according to the face skin image and the skin color feature map corresponding to each original face image, the method further comprises: Receiving a skin color transformation request, the skin color transformation request comprising a face image to be transformed and a target face image; Obtaining a skin color feature map corresponding to the target face image; Obtaining a facial skin image corresponding to the face image to be transformed; Adjusting a skin color of the face image to be transformed to a skin color corresponding to the target face image through the trained skin color transformation model according to the skin color feature map corresponding to the target face image and the facial skin image corresponding to the face image to be transformed.

10. An image processing apparatus of skin color adjustment, characterized by, Comprise: A data acquisition module configured to remove pixel parts of facial features and hair from a first original face image, the first original face image being any one of each original face image; and obtain a facial skin image and a skin color feature map corresponding to each original face image, respectively; A model building module configured to build a skin color transformation model, the skin color transformation model comprising a channel fusion layer configured to fuse the facial skin image and the skin color feature map; A model training module configured to train the skin color transformation model according to the facial skin image and the skin color feature map corresponding to each original face image; Wherein, obtaining the skin color feature map corresponding to each original face image, respectively, comprises: Selecting a preset number of skin image blocks without blank pixel parts from the first original face image after the removing operation; Selecting two skin image blocks from the preset number of skin image blocks; Constructing a covariance matrix corresponding to the two skin image blocks; According to the covariance matrix, the weights corresponding to the two skin image blocks are calculated, respectively; According to the weights corresponding to the two skin image blocks, the two skin image blocks are fused into one fused image block; From the preset number of skin image blocks, a new skin image block is selected, and according to the fused image block and the new skin image block, the step of constructing the covariance matrix is executed in a loop until the preset number of skin image blocks are fused into one image block, obtaining the skin color feature map corresponding to the first original face image.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor runs the computer program to realize the method of any one of claims 1-9.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the method of any one of claims 1-9.

Images