Model generation, 3D hairstyle generation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a model generation, 3D hairstyle generation method, device, electronic device and storage medium. The model generation method includes: obtaining a 2D sample head image and a 3D hairstyle image corresponding to the 2D sample head image; inputting the 2D sample head image as an input image into a backbone model to obtain a first image; inputting the first image into a generator of a generative adversarial network to obtain a second image generated by the generator, comparing the second image with the input image, training the backbone model and the generative adversarial network until the second image generated by the generator meets a preset condition, and then terminating the training; determining the backbone model at the end of the training as a 3D hair reconstruction model. Through this method, the output result of the backbone model is "refined" through the adversarial generative network during the model training process, so that the trained 3D hair reconstruction model can output a 3D hairstyle with good effect, closer to the input two-dimensional plane image, and more suitable.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a model generation, a 3D hairstyle generation method, a device, an electronic device and a storage medium. Background Art

[0002] 3D hair reconstruction refers to the reconstruction of a three-dimensional human head hairstyle based on a flat head image. Due to the diversity of hair shapes and textures, 3D hair reconstruction is a very complex and difficult task. In traditional 3D hair reconstruction methods, it is impossible to take into account the learning of many hair points. There is still a big difference between the 3D hairstyle reconstructed by the traditional 3D hair reconstruction model and the flat head image. There is a common situation where the hair texture and color do not match or are not suitable, resulting in poor accuracy of the 3D hair reconstruction model and poor effect of the reconstructed 3D hairstyle. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a model generation, 3D hairstyle generation method, device, electronic device and storage medium, which can refine the final result of the 3D hair reconstruction model during the training process to improve the accuracy of 3D hair reconstruction. The specific technical solution is as follows:

[0004] In a first aspect of the present invention, a model generation method is first provided, the method comprising:

[0005] Acquire a 2D sample head image and a 3D hairstyle image corresponding to the 2D sample head image;

[0006] The 2D sample head image is input as an input image to the backbone model to obtain a first image; the first image is a 3D hairstyle image, and the backbone model is used to generate a 3D hairstyle image according to the 2D head image, and the 3D hairstyle image corresponding to the 2D sample head image is a label of the backbone model during the training process;

[0007] Inputting the first image to a generator of a generative adversarial network to obtain a second image generated by the generator, comparing the second image with the input image, training the backbone model and the generative adversarial network until the second image meets a preset condition, and then terminating the training; the preset condition is that the pixel value error between the second image and the input image is less than a preset threshold;

[0008] The backbone model at the end of training is determined as the 3D hair reconstruction model.

[0009] Optionally, during the training process of the backbone model and the generative adversarial network, a first loss function value corresponding to the first image output by the backbone model is determined by a first loss function;

[0010] Determine a second loss function value corresponding to the second image output by the generator through a second loss function;

[0011] A total loss function value is determined according to the first loss function value and the second loss function value, and model parameters of the backbone model and the generative adversarial network are updated according to the total loss function value.

[0012] Optionally, the second loss function at least includes: an adversarial loss function and a reconstruction loss function, and the first loss function, the adversarial loss function and the reconstruction loss function are all L1 loss functions; and determining the second loss function value corresponding to the second image output by the generator by the second loss function includes:

[0013] Determine an adversarial loss function value corresponding to the second image by using the adversarial loss function;

[0014] Determine a reconstruction loss function value corresponding to the second image by using the reconstruction loss function;

[0015] The second loss function value is determined according to the adversarial loss function value and the weight corresponding to the adversarial loss function, as well as the reconstruction loss function value and the weight corresponding to the reconstruction loss function.

[0016] Optionally, in the initial stage of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function is 0; in the middle and late stages of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function increases from 0 to 1.

[0017] Optionally, the acquiring of the 2D sample head image and the 3D hairstyle map corresponding to the 2D sample head image includes: acquiring the 3D hairstyle map, and mapping the 3D hairstyle map into the 2D sample head image;

[0018] The step of inputting the 2D sample head image as an input image into the backbone model to obtain a first image includes:

[0019] Preprocessing the 2D sample head image to obtain first preprocessing data corresponding to the 2D sample head image, wherein the first preprocessing data at least includes a direction map corresponding to the 2D sample head image; inputting the first preprocessing data into a backbone model to obtain a first image;

[0020] Among them, the input image of the backbone model is the directional image corresponding to the 2D sample head image, and the structure of the backbone model is: the backbone model includes an encoder and a decoder, the encoder and the decoder are both composed of multiple serial unit modules, and multiple parallel 3D convolution modules are connected between the encoder and the decoder; there are direct branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module, and each unit module fuses the feature information input in the unit module connected upstream of the data stream, so that the unit module can continuously fuse the feature information of the first preprocessed data, and the feature information at least includes position information, and the position information includes coordinate information of the hair point.

[0021] In a second aspect of the present invention, a 3D hairstyle generation method is provided, the method comprising:

[0022] Get the 2D head image to be processed;

[0023] The 2D head image to be processed is input into the 3D hair reconstruction model generated by the model generation method according to the first aspect of the present invention, to obtain a 3D hairstyle image output by the 3D hair reconstruction model.

[0024] Optionally, the step of inputting the 2D human head image to be processed into a 3D hair reconstruction model generated by the model generation method according to the first aspect of the present invention to obtain a 3D hairstyle image output by the 3D hair reconstruction model comprises:

[0025] Preprocessing the 2D human head image to be processed to obtain second preprocessing data corresponding to the 2D human head image to be processed;

[0026] The second preprocessed data is input into the 3D hair reconstruction model generated by the model generation method according to the first aspect of the present invention to obtain a 3D hairstyle image output by the 3D hair reconstruction model.

[0027] In a third aspect of the present invention, a model generation device is provided, the device comprising:

[0028] A first image acquisition module, used to acquire a 2D sample head image and a 3D hairstyle image corresponding to the 2D sample head image;

[0029] A first model training module is used to input the 2D sample head image as an input image into the backbone model to obtain a first image; the first image is a 3D hairstyle image, and the backbone model is used to generate the 3D hairstyle image according to the 2D head image, and the 3D hairstyle image corresponding to the 2D sample head image is the label of the backbone model during the training process;

[0030] A second model training module is used to input the first image into a generator of a generative adversarial network to obtain a second image generated by the generator, compare the second image with the input image, and train the backbone model and the generative adversarial network until the second image meets a preset condition, and then terminate the training; the preset condition is that the pixel value error between the second image and the input image is less than a preset threshold;

[0031] The model generation module is used to determine the backbone model at the end of training as a 3D hair reconstruction model.

[0032] Optionally, the device further comprises:

[0033] A first loss function value determination module, used to determine a first loss function value corresponding to a first image output by the backbone model through a first loss function during the training process of the backbone model and the generative adversarial network;

[0034] A second loss function value determination module, used to determine a second loss function value corresponding to a second image output by the generator through a second loss function;

[0035] A model parameter updating module is used to determine a total loss function value according to the first loss function value and the second loss function value, and to update the model parameters of the backbone model and the generative adversarial network according to the total loss function value.

[0036] Optionally, the second loss function at least includes: an adversarial loss function and a reconstruction loss function, and the first loss function, the adversarial loss function and the reconstruction loss function are all L1 loss functions;

[0037] The second loss function value determination module includes:

[0038] An adversarial loss function value determination module, used to determine an adversarial loss function value corresponding to the second image by using the adversarial loss function;

[0039] A reconstruction loss function value determination module, used to determine a reconstruction loss function value corresponding to the second image by using the reconstruction loss function;

[0040] The second loss function value determination submodule is used to determine the second loss function value based on the adversarial loss function value and the weight corresponding to the adversarial loss function, as well as the reconstruction loss function value and the weight corresponding to the reconstruction loss function.

[0041] Optionally, in the initial stage of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function is 0; in the middle and late stages of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function increases from 0 to 1.

[0042] Optionally, the first image acquisition module includes: a first image acquisition submodule, used to acquire a 3D hairstyle image and map the 3D hairstyle image into a 2D sample head image;

[0043] The first model training module includes: a first model training submodule, which is used to preprocess the 2D sample head image to obtain first preprocessed data corresponding to the 2D sample head image, wherein the first preprocessed data at least includes a direction map corresponding to the 2D sample head image; and input the first preprocessed data into the backbone model to obtain a first image;

[0044] Among them, the input image of the backbone model is the directional image corresponding to the 2D sample head image, and the structure of the backbone model is: the backbone model includes an encoder and a decoder, the encoder and the decoder are both composed of multiple serial unit modules, and multiple parallel 3D convolution modules are connected between the encoder and the decoder; there are direct branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module, and each unit module fuses the feature information input in the unit module connected upstream of the data stream, so that the unit module can continuously fuse the feature information of the first preprocessed data, and the feature information at least includes position information, and the position information includes coordinate information of the hair point.

[0045] In a fourth aspect of the present invention, there is also provided a 3D hairstyle generating device, the device comprising:

[0046] The second image acquisition module is used to acquire a 2D head image to be processed;

[0047] The first 3D hairstyle acquisition module is used to input the 2D head image to be processed into the 3D hair reconstruction model generated by the model generation method according to the first aspect of the present invention, and obtain a 3D hairstyle image output by the 3D hair reconstruction model.

[0048] Optionally, the first 3D hairstyle acquisition module includes:

[0049] A first data processing module, used for preprocessing the 2D head image to be processed to obtain second preprocessing data corresponding to the 2D head image to be processed;

[0050] The second 3D hairstyle acquisition module is used to input the second pre-processed data into the 3D hair reconstruction model generated by the model generation method according to the first aspect of the present invention, and obtain a 3D hairstyle image output by the 3D hair reconstruction model.

[0051] In a fifth aspect of the implementation of the present invention, there is also provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the model generation method described in the first aspect of the embodiment of the present invention or the steps of the 3D hairstyle generation method described in the second aspect of the embodiment of the present invention are implemented.

[0052] In a sixth aspect of the implementation of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the model generation method described in the first aspect of the embodiment of the present invention or the steps of the 3D hairstyle generation method described in the second aspect of the embodiment of the present invention are implemented.

[0053] By adopting the model generation method provided by the embodiment of the present invention, the learning process of the 3D hair reconstruction model can be corrected to ensure that the 3D hairstyle image generated by the 3D hair reconstruction model meets expectations and is compatible with the 2D head image input to the 3D hair reconstruction model. This method further processes the first image output by the 3D hair reconstruction model through the generator of the adversarial generative network, and compares the second image generated by the generator with the input image of the 3D hair reconstruction model. When the second image generated by the generator meets the preset conditions (that is, the second image is basically consistent with the input image), the training is terminated, thereby obtaining a 3D hair reconstruction model with better reconstruction effect, so as to achieve "fine-tuning" of the output results of the 3D hair reconstruction model through the adversarial generative network during the training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0055] Figure 1 is a flow chart of a model generation method shown in one embodiment of the present invention;

[0056] Figure 2 is a schematic diagram of a model output effect shown in an embodiment of the present invention;

[0057] Figure 3 is a flow chart of a model updating method shown in one embodiment of the present invention;

[0058] Figure 4 is a flow chart of a model updating method shown in one embodiment of the present invention;

[0059] Figure 5 is a flow chart of a model generation method shown in one embodiment of the present invention;

[0060] Figure 6 is a structural schematic diagram of a backbone model shown in one embodiment of the present invention;

[0061] Figure 7 is a schematic diagram of a model generation method shown in one embodiment of the present invention;

[0062] Figure 8 is a flow chart of a 3D hairstyle generation method shown in one embodiment of the present invention;

[0063] Fig. 9 is a flow chart of a 3D hairstyle generation method shown in another embodiment of the present invention;

[0064] Fig.10 is a structural block diagram of a model generation device provided by an embodiment of the present invention;

[0065] Fig.11 is a structural block diagram of a 3D hairstyle generating device provided by an embodiment of the present invention;

[0066] Fig.12 It is a schematic diagram of an electronic device shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Since there is still a large difference between the 3D hairstyle reconstructed by the traditional 3D hair reconstruction model and the input flat head image, there is a common situation that the texture and color of the hair do not match or are not suitable, resulting in poor accuracy of the 3D hair reconstruction model and poor effect of the reconstructed 3D hairstyle. Based on this, an embodiment of the present invention proposes a model generation method, which can introduce an adversarial generative network to refine the output result of the 3D hair reconstruction model when training the 3D hair reconstruction model, and compare the generated graph generated by processing the output result of the 3D hair reconstruction model by the adversarial generative network with the input graph of the 3D hair reconstruction model. When the generated graph is basically consistent with the input graph (that is, the pixel value error between the two is less than a preset threshold), the training is terminated, so that the trained 3D hair reconstruction model can output a 3D hairstyle with good effect, closer to the input two-dimensional plane image, and more suitable.

[0069] refer to Figure 1 , Figure 1 FIG. 1 is a flow chart of a model generation method according to an embodiment of the present invention. Figure 1 As shown, the model generation method of this embodiment may include the following steps:

[0070] Step S11: obtaining a 2D sample head image and a 3D hairstyle image corresponding to the 2D sample head image.

[0071] In this embodiment, the model needs to be constructed by first obtaining a training sample set, and the training sample set in this embodiment is a plurality of 2D sample head images and the 3D hairstyle images corresponding to the plurality of 2D sample head images. It can be understood that the training sample set in this embodiment is a plurality of groups of sample image pairs, each group of sample image pairs is a 2D sample head image and a 3D hairstyle image corresponding to the 2D sample head image, and during the training process, the label of the 2D sample head image is the 3D hairstyle image corresponding to the 2D sample head image. In addition, it should be noted that the 3D hairstyle image in this embodiment is a 3D hairstyle image containing a human head structure.

[0072] Step S12: input the 2D sample head image as an input image into the backbone model to obtain a first image; the first image is a 3D hairstyle image, and the backbone model is used to generate a 3D hairstyle image based on the 2D head image, and the 3D hairstyle image corresponding to the 2D sample head image is the label of the backbone model during the training process.

[0073] In this embodiment, a backbone model is pre-set, and the backbone model of this embodiment can be any 3D hair reconstruction model, which is used to generate a 3D hairstyle image according to a 2D head image. The input of the backbone model is a 2D sample head image in a training sample set, and the 2D sample head image is input into the backbone model as an input image. The backbone model processes the 2D sample head image to obtain a first image, which is the 3D hairstyle image generated by the backbone model, that is, the output of the backbone model.

[0074] Step S13: Input the first image to the generator of the generative adversarial network to obtain a second image generated by the generator, compare the second image with the input image, train the backbone model and the generative adversarial network until the second image meets a preset condition, and then terminate the training; the preset condition is that the pixel value error between the second image and the input image is less than a preset threshold.

[0075] In this embodiment, a generative adversarial network branch is introduced into the final output part of the backbone model for auxiliary learning to "fine-tune" the output of the backbone model. Among them, the generative adversarial network is a deep learning model that produces effective output through mutual game learning between the generator and the discriminator, and it has a strong learning ability. In the training process of the generative adversarial network, the generator and the discriminator compete with each other to gradually adjust the model parameters of the generator and the discriminator. It can be understood that the training process of the generative adversarial network is: the training (update) of the generator and the training (update) of the discriminator are performed alternately.

[0076] In the model training process of this embodiment, the generator part of the generative adversarial network is introduced into the final output part of the backbone model, and the first image output by the backbone model is input into the generator of the generative adversarial network, so that the generator processes the first image to obtain the second image, and the second image is a flat human head image. Then the second image (i.e., the generated image of the generator) is compared with the input image of the backbone model, and the backbone model and the generative adversarial network are trained until the generator generates a generated image that is basically consistent with the input image, and the training is terminated. At this point, this embodiment corrects the learning process of the backbone model through the generative adversarial network to ensure that the learning process of the backbone model is carried out in a relatively correct direction (i.e., the 3D hairstyle image generated by the backbone model meets expectations and is compatible with the input image of the backbone model).

[0077] The generated image is basically consistent with the input image, which can be understood as follows: the second image generated by the generator meets the preset condition, and the preset condition is that the pixel value error between the second image and the input image is less than the preset threshold; the pixel value error can be obtained by L1 loss (L1 loss function), and the preset threshold is the maximum pixel value error between the second image and the input image when the generated image is basically consistent with the input image, which is set in advance based on experience. The preset threshold can be adjusted accordingly based on training effects and manual experience, and this embodiment does not impose any restrictions on the specific value of the preset threshold.

[0078] The discriminator of the generative adversarial network is trained using the input image and the second image generated by the generator, so that the discriminator can distinguish which are the input images and which are the generated second images (this can tell the generator: the second image you generated is inconsistent with the input image and needs to be optimized). In other words, the role of the discriminator is to make the second image closer to the input image.

[0079] That is to say, the discriminator of the generative adversarial network can determine whether the two are consistent based on the second image output by the generator and the input image of the backbone model. For example, the discriminator can determine whether the pixel value error between the second image and the input image is less than a preset threshold, and decide whether the generator corresponding to the discriminator needs to continue training based on the judgment result (for example, whether the second image is consistent with the input image, or whether the pixel value error between the second image and the input image is less than a preset threshold).

[0080] Step S14: Determine the trunk model at the end of training as the 3D hair reconstruction model.

[0081] In this embodiment, during the training of the backbone model and the generative adversarial network, when the generator is trained to generate a second image that meets the above preset conditions, the backbone model at the end of the training is determined as a 3D hair reconstruction model, and the 3D hair reconstruction model can output a 3D hairstyle image that is better, closer to the two-dimensional plane image input by the model, and more suitable. Figure 2 As shown, Figure 2 is a schematic diagram of a model output effect shown in an embodiment of the present invention. Figure 2 The left picture in the figure is the input 2D head image, and the right picture is the 3D hairstyle image output by the 3D hair reconstruction model.

[0082] Through the model generation method of this embodiment, when training the 3D hair reconstruction model, an adversarial generative network is introduced into the final output part of the 3D hair reconstruction model to refine the output result of the 3D hair reconstruction model to improve the accuracy of 3D hair reconstruction. The first image output by the 3D hair reconstruction model is processed by the generator of the adversarial generative network to obtain a second image, and the second image is compared with the input image of the 3D hair reconstruction model. When the second image generated by the generator meets the preset conditions, the training is terminated to obtain a 3D hair reconstruction model with better effect. Through this method, the learning process of the 3D hair reconstruction model can be corrected to ensure that the 3D hairstyle image generated by the 3D hair reconstruction model meets expectations and is compatible with the 2D human head image input to the 3D hair reconstruction model.

[0083] In combination with the above embodiments, in one implementation, the generator of the generative adversarial network is a decoder structure, that is, a decoder part for improving resolution, which is composed of multiple residual modules, and each residual module includes a jump connection inside, and the function of the jump connection is to perform feature fusion of the input and output of the residual module. The generator further upsamples the output of the backbone model (such as the final 32*32 feature map output), such as outputting a 256*256*3 feature map, and the ground truth (correct label) corresponding to the feature map is the real input map of the backbone network, so it is necessary to make full use of the existing input and refine the final result of the model to ensure that the backbone model learns in a relatively correct direction.

[0084] In combination with the above embodiments, in one implementation, the present invention also provides a model updating method, such as Figure 3 As shown, Figure 3 is a flow chart of a model updating method shown in an embodiment of the present invention. Specifically, the method includes steps S31 to S33:

[0085] Step S31: During the training process of the backbone model and the generative adversarial network, a first loss function value corresponding to a first image output by the backbone model is determined by a first loss function.

[0086] In this embodiment, during the training process of the backbone model and the generative adversarial network, the model parameters of the backbone model and the generative adversarial network can be iteratively updated through the loss function to optimize the backbone model so that it can output a 3D hairstyle image with better effect. Among them, the first loss function in this embodiment is the loss function of the backbone model, and the first loss function value corresponding to the first image output by the backbone model can be determined through the first loss function, that is, the first loss function value is calculated through the first loss function according to the first image output by the backbone model and the label of the backbone model (the 3D hairstyle image corresponding to the 2D sample head image).

[0087] Step S32: Determine a second loss function value corresponding to the second image output by the generator through a second loss function.

[0088] The second loss function in this embodiment is the loss function of the generative adversarial network, and the second loss function value corresponding to the second image output by the generator can be determined by the second loss function, that is, the second loss function value is calculated by the second loss function according to the second image output by the generator and the input image of the backbone model (2D sample head image). It should be noted that there is no restriction on the order of step S32 and step S31 in this embodiment, and both can be performed simultaneously, step S31 can be performed first and then step S32, or step S32 can be performed first and then step S31.

[0089] Step S33: Determine a total loss function value according to the first loss function value and the second loss function value, and update model parameters of the backbone model and the generative adversarial network according to the total loss function value.

[0090] In this embodiment, the first loss function value and the second loss function value are obtained, and the total loss function value can be determined according to the first loss function value and the second loss function value, and then the model parameters of the backbone model and the model parameters of the generative adversarial network are updated according to the total loss function value. Optionally, the method for determining the total loss function value in this embodiment can be determined according to the actual training effect, such as the total loss function value can be the sum of the first loss function value and the second loss function value, or the total loss function value can be obtained by weighted summing the first loss function value and the second loss function value, etc. This embodiment does not impose any restrictions on this.

[0091] In this embodiment, the model parameters of the backbone model and the generative adversarial network in the training process are updated through the first loss function and the second loss function, so that the trained backbone model can output a 3D hairstyle image that is closer to and more matched with the 2D sample head image, thereby further improving the accuracy of 3D hair reconstruction.

[0092] In combination with the above embodiments, in one implementation, the present invention further provides a model updating method, in which the second loss function at least includes: an adversarial loss function and a reconstruction loss function, and the first loss function, the adversarial loss function and the reconstruction loss function are all L1 loss functions. Figure 4 As shown, Figure 4 is a flow chart of a model updating method shown in an embodiment of the present invention. Specifically, in the method, step S32 specifically includes steps S41 to S43:

[0093] Step S41: Determine the adversarial loss function value corresponding to the second image through the adversarial loss function.

[0094] In this embodiment, the loss function of the generative adversarial network includes at least: an adversarial loss function and a reconstruction loss function. Among them, the adversarial loss function is used to use the discriminator to force the second image to be closer to the input image of the backbone model. Specifically, the adversarial loss function uses the discriminator to supervise the second image generated by the generator, forcing the second image generated by the generator to continuously approach the input image of the backbone model until the second image is "basically consistent" with the input image.

[0095] In this embodiment, the first loss function, adversarial loss function and reconstruction loss function can all be L1 loss functions, and the L1 loss function is the mean absolute error loss function. In this way, the first loss function of the backbone model calculates the absolute value error of the hair coordinate points between the first image and the label of the backbone model (the 3D hairstyle image corresponding to the 2D sample head image), and the adversarial loss function and reconstruction loss function of the generative adversarial network calculate the absolute value error of the pixel values ​​between the second image and the input image of the backbone model (2D sample head image). In step S41, this embodiment can determine the adversarial loss function value corresponding to the second image through the adversarial loss function, that is, the adversarial loss function value is calculated through the adversarial loss function according to the second image output by the generator and the input image of the backbone model (2D sample head image).

[0096] Step S42: Determine a reconstruction loss function value corresponding to the second image through the reconstruction loss function.

[0097] In this embodiment, the reconstruction loss function value corresponding to the second image can be determined by the reconstruction loss function, that is, the reconstruction loss function value is calculated by the reconstruction loss function based on the second image output by the generator and the input image of the backbone model (2D sample head image).

[0098] Step S43: Determine the second loss function value according to the adversarial loss function value and the weight corresponding to the adversarial loss function, and the reconstruction loss function value and the weight corresponding to the reconstruction loss function.

[0099] In this embodiment, the weights corresponding to the adversarial loss function and the reconstruction loss function are set in advance. After determining the adversarial loss function value and the adversarial loss function value, the second loss function value can be determined according to the adversarial loss function value and the weight corresponding to the adversarial loss function, and the reconstruction loss function value and the weight corresponding to the reconstruction loss function. For example, the method for determining the second loss function value in this embodiment can be: second loss function value = adversarial loss function value * weight corresponding to the adversarial loss function + reconstruction loss function value * weight corresponding to the reconstruction loss function.

[0100] In this embodiment, the second loss function is set to include at least an adversarial loss function and a reconstruction loss function, and the L1 loss function is used as the loss function of the first loss function, the adversarial loss function and the reconstruction loss function. The strong constraint of the L1 loss function is used to better train the backbone model and the generative adversarial network, so as to better refine the learning results of the backbone model.

[0101] In combination with the above embodiments, in one implementation, considering the strong constraint of the L1 loss function, it is inevitable that it will interfere too much with the learning of the backbone model. Therefore, in this embodiment, in the initial stage of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function is set to 0; in the middle and late stages of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function is gradually increased, and the weight of the reconstruction loss function is increased from 0 to 1.

[0102] For example, if the training of the entire model is divided into 300 epochs, where one epoch is the process of training all training samples once, then the initial stage of the training process in this embodiment is the 0-100 epoch stage, and the middle and late stages of the training process are the 100 epoch-300 epoch stages. This embodiment can be used in the model training process to gradually increase the reconstruction loss function through the training stages and training effects to refine the learning results of the entire network.

[0103] In this embodiment, the weight value of the reconstruction loss function is set in stages. The weight is set to 0 at the beginning of training, and the weight value of the reconstruction loss function is gradually increased in the middle and late stages. Experiments have shown that the training method of gradually increasing weights in this embodiment has better results than setting the weights at the beginning of the model network, thereby further improving the accuracy of the model's 3D hair reconstruction.

[0104] In combination with the above embodiments, in one implementation, considering that in the current traditional 3D hair reconstruction model through regression, as the model network structure deepens, the hair position information will inevitably be continuously lost, which ultimately leads to poor 3D hairstyle effects generated by the model, and a large gap with the flat head image. Therefore, in this embodiment, not only the output results of the backbone model are refined through the generative adversarial network to improve the effect of the 3D hair reconstruction model, but also the structure of the backbone model is further improved so that it can be trained into a 3D hair reconstruction model that is well suitable for regression tasks.

[0105] like Figure 5 As shown, Figure 5 is a flow chart of a model generation method shown in an embodiment of the present invention. Specifically, in the method, the above step S11 specifically includes step S51, and the above step S12 specifically includes step S52:

[0106] Step S51: Acquire a 3D hairstyle image, and map the 3D hairstyle image into a 2D sample head image.

[0107] In this embodiment, a plurality of 3D hairstyle images are first obtained, and then the 3D hairstyle images are mapped to obtain 2D sample head images corresponding to the 3D hairstyle images. In one implementation, the 3D hairstyle images may be orthogonally projected, and the Z axis in the 3D hairstyle images may be removed to convert the images into plane images. It is understood that there are many ways to map the 3D hairstyle images into 2D sample head images, and this embodiment does not limit this.

[0108] Step S52: preprocess the 2D sample head image to obtain first preprocessed data corresponding to the 2D sample head image, wherein the first preprocessed data at least includes a direction map corresponding to the 2D sample head image; input the first preprocessed data into the backbone model to obtain a first image.

[0109] In this embodiment, considering that the 2D sample head image is obtained by mapping the 3D hairstyle image, there is a difference between the 2D head image (i.e., the real head plane image) input when the 3D hair reconstruction model is applied. Therefore, in order to make the difference between the 2D head image used in the model training and the model application as little as possible to affect the 3D hairstyle image generated when the model is applied, and not affect the training effect of the model, after obtaining the 2D sample head image, the 2D sample head image is preprocessed to obtain the first preprocessed data corresponding to the 2D sample head image. Among them, the preprocessing in this embodiment can be a processing method for extracting feature information with a small difference from the real human hair hairstyle plane image from the 2D sample head image. The first preprocessed data in this embodiment is: data in the 2D sample head image that is useful for model training and has a small difference from the real head image; the first preprocessed data at least includes the direction map corresponding to the 2D sample head image. In this embodiment, the first preprocessed data is used for model training, which is more conducive to training a better 3D hair reconstruction model.

[0110] In this embodiment, after obtaining the first preprocessed data corresponding to the 2D sample head image, the first preprocessed data is used as the input of the backbone model, and the 3D hairstyle image corresponding to the 2D sample head image is used as the output target, and the backbone model is trained to obtain the first image output by the backbone model.

[0111] Among them, the structural design of the backbone model in this embodiment is as follows: the backbone model includes an encoder and a decoder, and the encoder and the decoder are composed of multiple serial unit modules, and each unit module has a direct branch between the input and output ends, and multiple parallel 3D convolution modules are connected between the encoder and the decoder, and each 3D convolution module also has a direct branch between the input and output ends. The direct branch can be a jump connection, which is used to add the input of the module to the output after convolution through the direct edge, so as to fuse the features of the module input and output. Figure 6 As shown, Figure 6 is a schematic diagram of a structure of a backbone model shown in an embodiment of the present invention. It should be noted that: Figure 6 The direct branches shown in the figure are only to better illustrate the direction of data flow. In fact, the direct branches are all inside each module (each unit module and each 3D convolution module) to perform feature fusion of module input and output.

[0112] like Figure 7 As shown, Figure 7 FIG. 1 is a schematic diagram of a model generation method according to an embodiment of the present invention. Figure 7In the figure, Input is the input of the backbone model (the first preprocessed data), Down and Up represent the backbone model, where Down represents the encoder and Up represents the decoder. The encoder-decoder part of the backbone model outputs the first image, which is the 3D hairstyle image. Figure 7 Gan in the figure represents the generator part of the generative adversarial network, and the output result of the backbone model is "refined" by the generator of the generative adversarial network, so that the generator generates a generated image (second image) that is "basically consistent" with the input image of the backbone model. In this embodiment, the generated image of the backbone model is a directional image corresponding to the 2D sample head image, and the discriminator of the generative adversarial network is used to make the second image closer to the directional image corresponding to the 2D sample head image.

[0113] In this embodiment, through the entire backbone model structure and the directly connected branches in the module, the unit module can perform feature fusion on its own input and output, and its own input is the final output of the unit module connected upstream of its data flow, and the final output of the unit module connected upstream of the unit module data flow is the result of the feature fusion of the input and output of the unit module connected upstream of the unit module data flow. Therefore, each unit module in the backbone model can fuse the feature information input in the unit module connected upstream of its data flow, wherein the feature information at least includes: hair position information (such as coordinate information of hair points) and hair semantic information (such as hair color, texture and other information), so that during the model training process, the unit module of the backbone model can continuously fuse the position information of the first preprocessed data, and make full use of the position information of the hair in the 2D sample head image for training, so that the 3D hairstyle image finally output by the backbone model can be closer to and more matched with the input 2D sample head image.

[0114] Through the model generation method proposed in this embodiment, each serial unit module can fuse the feature information input from the unit module connected upstream of its data stream, and the decoder can continuously fuse the information of the encoder through multiple parallel 3D convolution modules, so that each unit module in the 3D hair reconstruction model can fuse the feature information input from the unit module connected upstream of its data stream, which means that the unit modules in the model can continuously fuse the position information of the lower layer of the model, ensuring that the model can reduce the position information of the hair as much as possible as the network structure deepens, so that the trained 3D hair reconstruction model can make full use of the position information of the hair in the image for processing. At the same time, the output result of the backbone model is "refined" by introducing an adversarial generation network, so that the 3D hairstyle map finally output by the 3D hair reconstruction model is closer to and more matched with the input 2D human head image, thereby generating a 3D hair reconstruction model that is well suitable for regression tasks and has better model output effect.

[0115] In combination with the above embodiments, in one implementation, Figure 6 As shown, the encoder and decoder of the backbone model are respectively composed of multiple unit modules connected in series, and the last unit module in the encoder is connected in series with the first unit module in the decoder; wherein, the unit modules in the encoder correspond to the unit modules in the decoder one by one; a 3D convolution module is connected between each unit module in the encoder and the corresponding unit module in the decoder.

[0116] The encoder in this embodiment is composed of a plurality of identical unit modules connected in series, and the decoder is also composed of a plurality of identical unit modules connected in series, and the last unit module in the encoder is connected in series with the first unit module in the decoder, thereby forming a codec in the backbone model. Among them, the unit modules in the encoder correspond to the unit modules in the decoder one by one, that is, each unit module in the encoder has a corresponding unit module in the decoder, and is symmetrical based on the connection between the codec and the decoder. For example, if the encoder and the decoder are respectively composed of three unit modules connected in series, the first unit module in the encoder corresponds to the third unit module in the decoder, the second unit module in the encoder corresponds to the second unit module in the decoder, and the third unit module in the encoder corresponds to the first unit module in the decoder.

[0117] In this embodiment, a 3D convolution module is connected between each unit module in the encoder and the corresponding unit module in the decoder. The 3D convolution module in this embodiment is essentially a 3D convolution operation, which is used to fuse the information of the encoder and the decoder. Continuing with the above example: if the encoder and the decoder are respectively composed of three unit modules connected in series, a 3D convolution module is connected between the first unit module in the encoder and the third unit module in the corresponding decoder; a 3D convolution module is connected between the second unit module in the encoder and the second unit module in the corresponding decoder; a 3D convolution module is connected between the third unit module in the encoder and the first unit module in the corresponding decoder.

[0118] In this embodiment, the output of each module in the backbone model is:

[0119] For the unit module: the output obtained after the unit module processes the input is not directly given to the next unit module, or the 3D convolution module connected to it. Instead, the input and output of the unit module are fused through the direct connection branch in the unit module (such as the jump connection, which is used to add the input of the module to the output after convolution through the direct connection edge) to obtain the final output of the unit module, and then the final output is given to the next unit module, or the 3D convolution module connected to it.

[0120] For the 3D convolution module: the output obtained by the 3D convolution module after processing the input is not directly given to the unit module in the decoder connected to it. Instead, the input and output of the 3D convolution module are feature fused through the direct connection branch (such as skip connection) in the 3D convolution module to obtain the final output of the 3D convolution module, and then the final output is given to the unit module in the decoder connected to it.

[0121] Among them, the input of each module in this embodiment is: the input of the first unit module in the encoder is: the first preprocessed data; the input of other unit modules in the encoder except the first unit module is: the final output of the previous unit module; the input of the 3D convolution module is: the final output of the unit module in the encoder connected to it; the input of the unit module of the decoder is: the final output of the previous unit module, and the final output of the 3D convolution module connected to the unit module of the decoder.

[0122] Through the structure of the backbone model proposed in this embodiment, the unit modules in the encoder and the decoder correspond one-to-one, and the corresponding unit modules in the codec are connected with a 3D convolution module, so that the encoder and the decoder can perform information fusion through multiple parallel 3D convolution modules, thereby further continuously fusing the low-level position information in the model and reducing the position information in the model as much as possible during the model training process. In the decoder of the present embodiment, each unit module in the decoder can continuously fuse the input information of the previous unit module (i.e., the unit module connected upstream of the data stream), and each unit module in the decoder can not only fuse the input information of the previous unit module, but also fuse the input information of the corresponding module in the encoder through the 3D convolution module, so that during the model training process, the unit module can continuously fuse the position information of the low layer of the model (the position information of the first preprocessed data) to ensure that as the model network structure deepens, the semantic information in the unit module is increased as much as possible while the position information is reduced as little as possible, so as to improve the training effect of the model. At the same time, the output result of the backbone model is "refined" by introducing an adversarial generation network, so that the 3D hairstyle map finally output by the 3D hair reconstruction model is closer to and more matched with the input 2D human head image, thereby generating a 3D hair reconstruction model that is well suitable for regression tasks and has better model output effect.

[0123] In an optional implementation manner, the first preprocessed data includes a directional map of a 2D sample head image, a mask corresponding to a hair region of the 2D sample head image, and head contour information of the 2D sample head image.

[0124] Among them, the 2D sample head image may be processed by a filter to obtain a directional map of the 2D sample head image. Among them, this embodiment may be processed by a Gabor filter to obtain a directional map of the 2D sample head image. This embodiment may be image segmentation of the 2D sample head image by an image segmentation algorithm or a special image segmentation model to obtain a mask of the 2D sample head image, and then determine the mask corresponding to the hair area in the 2D sample head image from the mask of the 2D sample head image. This embodiment may be processed by a human body contour recognition model to obtain head contour information in the 2D sample head image.

[0125] In one embodiment, the backbone model further includes an initial convolution module, which is disposed upstream of the data stream outside the encoder. The initial convolution module is used to perform dimensionality upscaling on the first preprocessed data to obtain a preliminary feature map as an input of the first unit module in the encoder.

[0126] In this embodiment, the steps performed by each unit module may include:

[0127] First, information is extracted from the input of the unit module to obtain a first feature map with reduced resolution.

[0128] In this embodiment, each unit module will first extract information from the input of the unit module, such as completing the input information extraction through a convolution with stride=2, so as to obtain a first feature map with reduced resolution.

[0129] Secondly, the resolution of the first feature map is restored to the input resolution through bilinear interpolation to obtain the intermediate feature map.

[0130] After obtaining the first feature map, the unit module of this embodiment will then restore the resolution of the first feature map to the resolution of the unit module input by a bilinear interpolation method, thereby obtaining an intermediate feature map. When the unit module of this embodiment extracts information, the resolution will decrease. Since the resolution continues to decrease, a certain amount of information will be lost. Therefore, the unit module restores the resolution of the feature map by a bilinear interpolation method, so that the unit module can maintain a constant resolution of the feature map and avoid information loss as much as possible.

[0131] Then, the number of channels of the intermediate feature map is increased to obtain the final feature map.

[0132] After obtaining the intermediate feature map in this embodiment, the number of channels of the intermediate feature map can be increased by a 1*1 convolution, such as doubling the number of channels when the unit module is input, thereby obtaining the final feature map. Since convolution will reduce the input feature information to a certain extent, the unit module of this embodiment compensates for the lost information by increasing the number of channels.

[0133] Finally, the final feature map and the input are fused through the direct connection branch to obtain the final output of the unit module.

[0134] In this embodiment, there is a direct connection branch between the input and output ends of the unit module, and the unit module can fuse the final feature map and the input of the unit module through the direct connection branch to obtain the final output of the unit module. For example, in this embodiment, a jump connection with a 1*1 convolution can be used to keep the number of feature channels of the input of the unit module consistent with the number of channels of the output of the unit module, thereby facilitating the feature fusion of the input and output of the unit module.

[0135] In this embodiment, the unit module extracts feature information, maintains a constant resolution of the feature map, increases the number of feature channels, and fuses input and output features, so that the entire model network structure composed of unit modules can avoid feature loss as much as possible, and further maintains the hair position information and semantic information during the model training process.

[0136] In combination with the above embodiments, in one implementation, the present invention further provides a 3D hairstyle generation method, in which the method further includes steps S81 and S82. The relationship between step S81 and step S82 is as follows: Figure 8 As shown, Figure 8 The figure is a flow chart of a 3D hairstyle generation method according to an embodiment of the present invention.

[0137] Step S81: Acquire the 2D head image to be processed.

[0138] In this embodiment, when 3D hairstyle reconstruction is required, a 2D head image for which 3D hairstyle reconstruction is required may be obtained, that is, a 2D head image to be processed may be obtained.

[0139] Step S82: inputting the 2D human head image to be processed into a 3D hair reconstruction model generated by the model generation method described in any of the above embodiments of the present invention, to obtain a 3D hairstyle image output by the 3D hair reconstruction model.

[0140] In this embodiment, after obtaining a 2D head image for which 3D hairstyle reconstruction is required, the 2D head image is input into a pre-trained 3D hair reconstruction model, and a 3D hairstyle image output by the 3D hair reconstruction model can be obtained. The pre-trained 3D hair reconstruction model in this embodiment is the 3D hair reconstruction model generated by the model generation method described in any of the above embodiments of the present invention and finally obtained through training.

[0141] In this embodiment, a 3D hairstyle is reconstructed for a 2D head image using a pre-trained 3D hair reconstruction model whose output results have been "refined", thereby obtaining a 3D hairstyle that is better in effect and closer to and more matching the input 2D head image.

[0142] In combination with the above embodiment, in another implementation manner, the present invention further provides another 3D hairstyle generation method, in which the above step S82 may include steps S91 to S92. Fig. 9 As shown, Fig. 9 FIG. 4 is a flow chart of a 3D hairstyle generating method according to another embodiment of the present invention.

[0143] Step S91: pre-processing the 2D head image to be processed to obtain second pre-processed data corresponding to the 2D head image to be processed.

[0144] In this embodiment, the acquired 2D head image to be processed needs to be preprocessed to obtain second preprocessed data corresponding to the 2D head image to be processed. The method of preprocessing in this step can be the same as the method of preprocessing in model training (such as the above step S52), and this embodiment does not impose any limitation on this.

[0145] Step S92: inputting the second pre-processed data into a 3D hair reconstruction model generated by the model generation method described in any of the above embodiments of the present invention, to obtain a 3D hairstyle image output by the 3D hair reconstruction model.

[0146] In this embodiment, the second preprocessed data obtained after the preprocessing is input into the pre-trained 3D hair reconstruction model to obtain a 3D hairstyle image output by the 3D hair reconstruction model. The pre-trained 3D hair reconstruction model in this embodiment is the last trained 3D hair reconstruction model generated by the model generation method described in any of the above embodiments of the present invention.

[0147] In this embodiment, a 3D hairstyle is reconstructed for a 2D head image through a pre-trained 3D hair reconstruction model whose position information is reduced as little as possible and whose output results are "refined", so that a 3D hairstyle with better effect, closer to and more matching the input 2D head image can be obtained.

[0148] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0149] Based on the same inventive concept, an embodiment of the present invention provides a model generation device 1000, referring to Fig.10 , Fig.10 FIG. 1 is a structural block diagram of a model generation device provided by an embodiment of the present invention. Fig.10 As shown, the device 1000 includes:

[0150] The first image acquisition module 1001 is used to acquire a 2D sample head image and a 3D hairstyle image corresponding to the 2D sample head image;

[0151] The first model training module 1002 is used to input the 2D sample head image as an input image into the backbone model to obtain a first image; the first image is a 3D hairstyle image, and the backbone model is used to generate a 3D hairstyle image according to the 2D head image, and the 3D hairstyle image corresponding to the 2D sample head image is a label of the backbone model during the training process;

[0152] The second model training module 1003 is used to input the first image into the generator of the generative adversarial network, obtain the second image generated by the generator, compare the second image with the input image, train the backbone model and the generative adversarial network until the second image meets a preset condition, and then terminate the training; the preset condition is that the pixel value error between the second image and the input image is less than a preset threshold;

[0153] The model generation module 1004 is used to determine the trunk model at the end of training as the 3D hair reconstruction model.

[0154] Optionally, the device 1000 further includes:

[0155] A first loss function value determination module, used to determine a first loss function value corresponding to a first image output by the backbone model through a first loss function during the training process of the backbone model and the generative adversarial network;

[0156] A second loss function value determination module, used to determine a second loss function value corresponding to a second image output by the generator through a second loss function;

[0157] A model parameter updating module is used to determine a total loss function value according to the first loss function value and the second loss function value, and to update the model parameters of the backbone model and the generative adversarial network according to the total loss function value.

[0158] Optionally, the second loss function at least includes: an adversarial loss function and a reconstruction loss function, and the first loss function, the adversarial loss function and the reconstruction loss function are all L1 loss functions;

[0159] The second loss function value determination module includes:

[0160] An adversarial loss function value determination module, used to determine an adversarial loss function value corresponding to the second image by using the adversarial loss function;

[0161] A reconstruction loss function value determination module, used to determine a reconstruction loss function value corresponding to the second image by using the reconstruction loss function;

[0162] The second loss function value determination submodule is used to determine the second loss function value based on the adversarial loss function value and the weight corresponding to the adversarial loss function, as well as the reconstruction loss function value and the weight corresponding to the reconstruction loss function.

[0163] Optionally, in the initial stage of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function is 0; in the middle and late stages of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function increases from 0 to 1.

[0164] Optionally, the first image acquisition module 1001 includes: a first image acquisition submodule, used to acquire a 3D hairstyle image, and map the 3D hairstyle image into a 2D sample head image;

[0165] The first model training module 1002 includes: a first model training submodule, which is used to preprocess the 2D sample head image to obtain first preprocessed data corresponding to the 2D sample head image, wherein the first preprocessed data at least includes a direction map corresponding to the 2D sample head image; and input the first preprocessed data into the backbone model to obtain a first image;

[0166] Among them, the input image of the backbone model is the directional image corresponding to the 2D sample head image, and the structure of the backbone model is: the backbone model includes an encoder and a decoder, the encoder and the decoder are both composed of multiple serial unit modules, and multiple parallel 3D convolution modules are connected between the encoder and the decoder; there are direct branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module, and each unit module fuses the feature information input in the unit module connected upstream of the data stream, so that the unit module can continuously fuse the feature information of the first preprocessed data, and the feature information at least includes position information, and the position information includes coordinate information of the hair point.

[0167] Based on the same inventive concept, an embodiment of the present invention provides a 3D hairstyle generating device 1100, referring to Fig.11 , Fig.11 FIG. 1 is a structural block diagram of a 3D hairstyle generating device provided by an embodiment of the present invention. Fig.11 As shown, the device 1100 includes:

[0168] The second image acquisition module 1101 is used to acquire a 2D head image to be processed;

[0169] The first 3D hairstyle acquisition module 1102 is used to input the 2D head image to be processed into a 3D hair reconstruction model generated by the model generation method described in any of the above embodiments of the present invention, and obtain a 3D hairstyle image output by the 3D hair reconstruction model.

[0170] Optionally, the first 3D hairstyle acquisition module 1102 includes:

[0171] A first data processing module, used for preprocessing the 2D head image to be processed to obtain second preprocessing data corresponding to the 2D head image to be processed;

[0172] The second 3D hairstyle acquisition module is used to input the second pre-processed data into the 3D hair reconstruction model generated by the model generation method described in any of the above embodiments of the present invention to obtain a 3D hairstyle image output by the 3D hair reconstruction model.

[0173] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the model generation method described in any of the above embodiments of the present invention or the steps in the 3D hairstyle generation method described in any of the above embodiments of the present invention are implemented.

[0174] Based on the same inventive concept, another embodiment of the present invention provides an electronic device 1200, such as Fig.12 shown. Fig.12 1 is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device includes a memory 1202, a processor 1201, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the model generation method according to any of the above embodiments of the present invention or the steps in the 3D hairstyle generation method according to any of the above embodiments of the present invention are implemented.

[0175] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0176] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0177] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, embodiments of the present invention may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware. Furthermore, embodiments of the present invention may take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0178] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0181] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0182] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0183] The above is a detailed introduction to a model generation, 3D hairstyle generation method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A model generation method, characterized in that: The method comprises: Acquire a 2D sample head image and a 3D hairstyle image corresponding to the 2D sample head image; The 2D sample head image is input as an input image to the backbone model to obtain a first image; the first image is a 3D hairstyle image, and the backbone model is used to generate a 3D hairstyle image according to the 2D head image, and the 3D hairstyle image corresponding to the 2D sample head image is a label of the backbone model during the training process; Inputting the first image to a generator of a generative adversarial network to obtain a second image generated by the generator, comparing the second image with the input image, training the backbone model and the generative adversarial network until the second image meets a preset condition, and then terminating the training; the preset condition is that the pixel value error between the second image and the input image is less than a preset threshold; The backbone model at the end of training is determined as the 3D hair reconstruction model.

2. The method according to claim 1, characterized in that: During the training of the backbone model and the generative adversarial network, determining a first loss function value corresponding to a first image output by the backbone model by using a first loss function; Determine a second loss function value corresponding to the second image output by the generator through a second loss function; A total loss function value is determined according to the first loss function value and the second loss function value, and model parameters of the backbone model and the generative adversarial network are updated according to the total loss function value.

3. The method according to claim 2, characterized in that The second loss function at least includes: an adversarial loss function and a reconstruction loss function, and the first loss function, the adversarial loss function and the reconstruction loss function are all L1 loss functions; and determining the second loss function value corresponding to the second image output by the generator through the second loss function includes: Determine the adversarial loss function value corresponding to the second image by using the adversarial loss function; Determine a reconstruction loss function value corresponding to the second image by using the reconstruction loss function; The second loss function value is determined according to the adversarial loss function value and the weight corresponding to the adversarial loss function, as well as the reconstruction loss function value and the weight corresponding to the reconstruction loss function.

4. The method according to claim 3, characterized in that In the initial stage of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function is 0; in the middle and late stages of the training process of the backbone model and the generative adversarial network, the weight of the reconstruction loss function increases from 0 to 1.

5. The method according to claim 1, characterized in that The obtaining of the 2D sample head image and the 3D hairstyle image corresponding to the 2D sample head image comprises: obtaining the 3D hairstyle image, and mapping the 3D hairstyle image into the 2D sample head image; The step of inputting the 2D sample head image as an input image into the backbone model to obtain a first image includes: Preprocessing the 2D sample head image to obtain first preprocessing data corresponding to the 2D sample head image, wherein the first preprocessing data at least includes a direction map corresponding to the 2D sample head image; inputting the first preprocessing data into a backbone model to obtain a first image; Among them, the input image of the backbone model is the directional image corresponding to the 2D sample head image, and the structure of the backbone model is: the backbone model includes an encoder and a decoder, the encoder and the decoder are both composed of multiple serial unit modules, and multiple parallel 3D convolution modules are connected between the encoder and the decoder; there are direct branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module, and each unit module fuses the feature information input in the unit module connected upstream of the data stream, so that the unit module can continuously fuse the feature information of the first preprocessed data, and the feature information at least includes position information, and the position information includes coordinate information of the hair point.

6. A 3D hairstyle generation method, characterized in that: The method comprises: Obtain the 2D human head image to be processed; The 2D human head image to be processed is input into a 3D hair reconstruction model generated by the model generation method according to any one of claims 1 to 5, to obtain a 3D hairstyle image output by the 3D hair reconstruction model.

7. The method according to claim 6, characterized in that The step of inputting the 2D head image to be processed into the 3D hair reconstruction model generated by the model generation method according to any one of claims 1 to 5 to obtain a 3D hairstyle image output by the 3D hair reconstruction model comprises: Preprocessing the 2D human head image to be processed to obtain second preprocessing data corresponding to the 2D human head image to be processed; The second preprocessed data is input into a 3D hair reconstruction model generated by the model generation method according to any one of claims 1 to 5 to obtain a 3D hairstyle image output by the 3D hair reconstruction model.

8. A model generation device, characterized in that: The device comprises: A first image acquisition module, used to acquire a 2D sample head image and a 3D hairstyle image corresponding to the 2D sample head image; A first model training module is used to input the 2D sample head image as an input image into the backbone model to obtain a first image; the first image is a 3D hairstyle image, and the backbone model is used to generate the 3D hairstyle image according to the 2D head image, and the 3D hairstyle image corresponding to the 2D sample head image is the label of the backbone model during the training process; A second model training module, used for inputting the first image into a generator of a generative adversarial network, obtaining a second image generated by the generator, comparing the second image with the input image, training the backbone model and the generative adversarial network until the second image meets a preset condition, and then terminating the training; the preset condition is that a pixel value error between the second image and the input image is less than a preset threshold; The model generation module is used to determine the backbone model at the end of training as a 3D hair reconstruction model.

9. A 3D hairstyle generating device, characterized in that: The device comprises: The second image acquisition module is used to acquire a 2D head image to be processed; The first 3D hairstyle acquisition module is used to input the 2D head image to be processed into the 3D hair reconstruction model generated by the model generation method according to any one of claims 1 to 5, and obtain a 3D hairstyle image output by the 3D hair reconstruction model.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images