Model Generation, 3D Hair Style Generation Method, Device, Electronic Device and Storage Medium

Abstract

An embodiment of the present invention provides a model generation, 3D hairstyle generation method, device, electronic device and storage medium. The model generation method includes: obtaining a 3D hairstyle map, and processing the 3D hairstyle map into a 2D hairstyle map; preprocessing the 2D hairstyle map to obtain first preprocessing data corresponding to the 2D hairstyle map; using the first preprocessing data as an input and using the 3D hairstyle map corresponding to the 2D hairstyle map as an output target to train an initial model; determining the trained initial model as a 3D hairstyle reconstruction model. Among them, both the encoder and decoder in the initial model are composed of multiple serial unit modules, and multiple parallel 3D convolutional modules are connected between the encoder and decoder. There are direct connection branches between the input ends and output ends of the unit modules and the 3D convolutional modules, so that the unit modules can continuously fuse the position information of the first preprocessing data, ensuring that as the network structure deepens, the initial model reduces the position information of the hair as little as possible to improve the model training effect.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of image processing, and in particular, to a method, apparatus, electronic device, and storage medium for model generation and 3D hairstyle generation. Background Art

[0002] 3D hair reconstruction refers to reconstructing a three-dimensional human hairstyle based on a planar human head image. Currently, the 3D hair reconstruction task is usually transformed into a regression task to calculate the coordinate information of 3D hair points. However, in the current traditional models for reconstructing 3D hair through regression, as the model network structure deepens, there will inevitably be a continuous loss of hair position information, resulting in a mismatch and a large gap between the hair position information in the 3D hairstyle generated by the model and the hair position information in the planar human head image. Eventually, the 3D hairstyle generated by the model has a poor effect and a large gap from the planar human head image. Summary of the Invention

[0003] The purpose of the embodiments of the present invention is to provide a method, apparatus, electronic device, and storage medium for model generation and 3D hairstyle generation, so as to minimize the loss of hair position information in the 3D hairstyle reconstruction model. The specific technical solutions are as follows:

[0004] In the first aspect of the present invention, a model generation method is first provided. The method includes:

[0005] Obtain a 3D hairstyle map;

[0006] Process the 3D hairstyle map into a 2D hairstyle map;

[0007] Preprocess the 2D hairstyle map to obtain first preprocessing data corresponding to the 2D hairstyle map;

[0008] Use the first preprocessing data as input and the 3D hairstyle map corresponding to the 2D hairstyle map as the output target to train an initial model. The structure of the initial model is as follows: the initial model includes an encoder and a decoder, both the encoder and the decoder are composed of multiple serial unit modules, and there are multiple parallel 3D convolution modules connected between the encoder and the decoder. There are direct connection branches between the input and output ends of the unit module and between the input and output ends of the 3D convolution module. Each unit module fuses the feature information input from the unit module connected upstream in the data stream, enabling the unit module to continuously fuse the feature information of the first preprocessing data. The feature information at least includes position information, and the position information includes the coordinate information of hair points;

[0009] Determine the trained initial model as a 3D hairstyle reconstruction model.

[0010] In a second aspect of the implementation of the present invention, a 3D hairstyle generation method is further provided. The method includes:

[0011] Obtain a 2D hairstyle image to be processed;

[0012] Preprocess the 2D hairstyle image to be processed to obtain second preprocessing data corresponding to the 2D hairstyle image to be processed;

[0013] Input the second preprocessing data into the 3D hairstyle reconstruction model generated by the model generation method as described in the first aspect of the present invention to obtain a 3D hairstyle image output by the 3D hairstyle reconstruction model.

[0014] In a third aspect of the implementation of the present invention, a model generation device is further provided. The device includes:

[0015] A first image acquisition module for acquiring a 3D hairstyle image;

[0016] A first image processing module for processing the 3D hairstyle image into a 2D hairstyle image;

[0017] A first data processing module for preprocessing the 2D hairstyle image to obtain first preprocessing data corresponding to the 2D hairstyle image;

[0018] A model training module for using the first preprocessing data as an input and the 3D hairstyle image corresponding to the 2D hairstyle image as an output target to train an initial model. Wherein, the structure of the initial model is: the initial model includes an encoder and a decoder, both the encoder and the decoder are composed of multiple serial unit modules, and multiple parallel 3D convolution modules are connected between the encoder and the decoder; there are direct connection branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module. Each unit module fuses the feature information input from the unit module connected upstream in the data stream, so that the unit module can continuously fuse the feature information of the first preprocessing data. The feature information at least includes position information, and the position information includes the coordinate information of hair points;

[0019] A model generation module for determining the trained initial model as a 3D hairstyle reconstruction model.

[0020] In a fourth aspect of the implementation of the present invention, a 3D hairstyle generation device is further provided. The device includes:

[0021] A second image acquisition module for acquiring a 2D hairstyle image to be processed;

[0022] A second data processing module, configured to preprocess the to-be-processed 2D hairstyle image to obtain second preprocessed data corresponding to the to-be-processed 2D hairstyle image;

[0023] A 3D hairstyle obtaining module, configured to input the second preprocessed data into a 3D hairstyle reconstruction model generated by the model generation method according to the first aspect of the present invention, and obtain a 3D hairstyle image output by the 3D hairstyle reconstruction model.

[0024] In a fifth aspect of the embodiments of the present invention, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the model generation method according to the first aspect of the embodiments of the present invention or the steps of the 3D hairstyle generation method according to the second aspect of the embodiments of the present invention are implemented.

[0025] In a sixth aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the model generation method according to the first aspect of the embodiments of the present invention or the steps of the 3D hairstyle generation method according to the second aspect of the embodiments of the present invention are implemented.

[0026] By using the model generation method provided in the embodiments of the present invention, the 3D hairstyle reconstruction model is designed as an encoder-decoder structure. Both the encoder and the decoder are serially composed of multiple unit modules, and multiple parallel 3D convolution modules are connected between the encoder and the decoder. There are direct connection branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module. Through the model structure and the direct connection branches proposed by this method, each serially connected unit module can fuse the feature information input from the unit module connected to its data stream upstream, and the decoder can continuously fuse the information of the encoder through multiple parallel 3D convolution modules, so that each unit module in the 3D hairstyle reconstruction model can fuse the feature information input from the unit module connected to its data stream upstream, which also enables the unit modules in the model to continuously fuse the position information of the lower layer of the model (i.e., the hair point coordinate information of the first preprocessed data), thereby ensuring that as the network structure deepens, the position information of the hair is reduced as little as possible, so that the trained 3D hairstyle reconstruction model can make full use of the position information of the hair in the image for processing, and the finally output 3D hairstyle image is closer and more matched to the input 2D hairstyle image, so as to obtain a 3D hairstyle reconstruction model that is well applicable to the regression task. Description of the Drawings

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0028] Figure 1 It is a flowchart of a model generation method shown in an embodiment of the present invention;

[0029] Figure 2 It is a schematic structural diagram of an initial model shown in an embodiment of the present invention;

[0030] Figure 3 It is a flowchart of a 3D hairstyle acquisition method shown in an embodiment of the present invention;

[0031] Figure 4 It is a schematic diagram of a hairstyle interpolation algorithm shown in an embodiment of the present invention;

[0032] Figure 5 It is a flowchart of a 3D hairstyle classification method shown in an embodiment of the present invention;

[0033] Figure 6 It is a flowchart of a data processing method shown in an embodiment of the present invention;

[0034] Figure 7 It is a flowchart of a unit module information processing shown in an embodiment of the present invention;

[0035] Figure 8 It is a schematic diagram of a model generation method shown in an embodiment of the present invention;

[0036] Figure 9 It is a flowchart of a 3D hairstyle generation method shown in an embodiment of the present invention;

[0037] Figure 10 It is a structural block diagram of a model generation device provided by an embodiment of the present invention;

[0038] Figure 11 It is a structural block diagram of a 3D hairstyle generation device provided by an embodiment of the present invention;

[0039] Figure 12 It is a schematic diagram of an electronic device shown in an embodiment of the present invention. Detailed implementation manners

[0040] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0041] Due to the invisibility of local hair in the planar head image and the high variability of hair structure, it is very difficult to perform 3D hair reconstruction, resulting in 3D hair reconstruction always being a relatively complex task. Therefore, converting the 3D hair reconstruction task into a regression task can make the 3D hair reconstruction task much clearer: 3D hair can be represented in the form of multiple hair bundles, and each hair bundle can be described as a combination of the connections of multiple points. Therefore, converting the complex 3D hair reconstruction task into a regression task of predicting multiple hair bundle points can greatly reduce the complexity of the 3D hair reconstruction task.

[0042] Among them, the regression task needs to fully consider the position information of the hair (hair point coordinates) and semantic information (texture, color, etc. of the hair). In the current traditional models for reconstructing 3D hair through regression, the lower-level structural parts (parts closer to the model input) often extract more position information. As the network structure deepens, the higher-level parts (parts closer to the model output) will obtain more semantic information. The deepening of the network structure will inevitably be accompanied by the continuous loss of low-level position information and the increase of high-level semantic information. The essence of 3D hair regression is to calculate the coordinate information of hair points, which is very sensitive to the position information of the hair. The continuous loss of position information in the model will inevitably make the hair position information in the 3D hairstyle generated by the model not match the hair position information in the planar head image and have a large gap, resulting in a poor 3D hairstyle effect finally output by the model and a large gap from the planar head image.

[0043] Therefore, in order to at least partially solve the above problems and one or more of other potential problems, the embodiments of the present invention propose a model generation method. This method designs the model network structure in the form of serial unit modules and parallel 3D convolution modules. There are direct connection branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module. Each module can perform feature fusion on its own input and output through the direct connection branch, so as to ensure as much as possible that the unit module can continuously fuse the low-level position information of the model, so as to ensure that while the semantic information in the unit module increases as much as possible, the position information decreases as little as possible.

[0044] Refer to Figure 1 , Figure 1 is a flowchart of a model generation method shown in an embodiment of the present invention. AsFigure 1 As shown in Figure 1 , the model generation method of this embodiment may include the following steps:

[0045] Step S11: Obtain a 3D hairstyle map.

[0046] Step S12: Process the 3D hairstyle map into a 2D hairstyle map.

[0047] In this embodiment, first obtain multiple 3D hairstyle maps, and then process the 3D hairstyle maps to obtain the corresponding 2D hairstyle maps. In one implementation manner, it may be to map the 3D hairstyle map into a 2D hairstyle map representation. For example, orthogonally project the 3D hairstyle map and remove the Z-axis in the 3D hairstyle map to convert it into a planar image. It can be understood that there are various ways to process the 3D hairstyle map into a 2D hairstyle map, and this embodiment does not limit this.

[0048] To build a model, it is necessary to first obtain a training sample set. The training sample set in this embodiment is: multiple 3D hairstyle maps, and the corresponding 2D hairstyle maps processed from these multiple 3D hairstyle maps. It can be understood that the training sample set in this embodiment is multiple groups of sample image pairs. Each group of sample image pairs is a 2D hairstyle map and the corresponding 3D hairstyle map. And during the training process, the label of the 2D hairstyle map is the coordinate information of each 3D hair point in the 3D hairstyle map corresponding to the 2D hairstyle map. In addition, it should be noted that the 3D hairstyle map in this embodiment is a 3D hairstyle map including a human head structure, and the 2D hairstyle map is a 2D hairstyle map including a human head structure, and whether the included human head structure includes a human face is optional.

[0049] Step S13: Preprocess the 2D hairstyle map to obtain the first preprocessed data corresponding to the 2D hairstyle map.

[0050] In this embodiment, considering that the 2D hairstyle maps in the sample training set are obtained by processing 3D hairstyle maps, there are differences between them and the 2D hairstyle maps (i.e., real human hairstyle planar images) input during the application of the 3D hairstyle reconstruction model. Therefore, in order to make the differences between the 2D hairstyle maps used during model training and model application have as little impact as possible on the 3D hairstyle maps generated during model application and not affect the training effect of the model, in this embodiment, after obtaining the 2D hairstyle maps, preprocess the 2D hairstyle maps to obtain the first preprocessed data corresponding to the 2D hairstyle maps. Among them, the preprocessing in this embodiment can be a processing method for extracting feature information with a relatively small gap from the 2D hairstyle maps compared to the real human hairstyle planar images. The first preprocessed data in this embodiment is: the data that is useful for model training in the 2D hairstyle maps and has a relatively small difference from the real human hairstyle planar images. Using the first preprocessed data for model training in this embodiment is more conducive to training a 3D hairstyle reconstruction model with better effects.

[0051] Step S14: Use the first preprocessed data as the input and the 3D hairstyle map corresponding to the 2D hairstyle map as the output target to train the initial model.

[0052] In this embodiment, after obtaining the first preprocessed data corresponding to the 2D hairstyle map, use the first preprocessed data as the input of the initial model and the 3D hairstyle map corresponding to the 2D hairstyle map in the training sample set as the output target to train the initial model. Among them, the structure of the initial model in this embodiment is designed as follows: The initial model includes an encoder and a decoder, and both the encoder and the decoder are composed of multiple serial unit modules. There is a direct connection branch between the input end and the output end of each unit module. And there are multiple parallel 3D convolutional modules connected between the encoder and the decoder. There is also a direct connection branch between the input end and the output end of each 3D convolutional module. The direct connection branch can be a skip connection, which is used to add the input of the module to the output after convolution through a direct connection edge, so as to perform feature fusion between the input and output of the module. As Figure 2 shown, Figure 2 Figure 7 is a schematic structural diagram of an initial model shown in an embodiment of the present invention. It should be noted that, Figure 2 the direct connection branches shown in Figure 7 are only for better indicating the flow direction of the data stream. In fact, the direct connection branches are all inside each module (each unit module and each 3D convolutional module) to perform feature fusion between the input and output of the module.

[0053] In this embodiment, through the entire initial model structure and the direct connection branches in the module, the unit module can perform feature fusion on its own input and output. Its own input is the final output of the unit module connected to its upstream in the data stream. And the final output of the unit module connected to its upstream in the data stream of this unit module is the result of feature fusion between the input and output of the unit module connected to its upstream in the data stream of this unit module. Therefore, each unit module in the initial model can fuse the feature information input in the unit module connected to its upstream in the data stream. Among them, the feature information at least includes: the position information of the hair (such as the coordinate information of the hair points) and the semantic information of the hair (such as the color, texture, etc. of the hair). Thus, during the model training process, the unit module of the initial model can continuously fuse the position information of the first preprocessed data, make full use of the position information of the hair in the 2D hairstyle map for training, so that the 3D hairstyle map finally output by the initial model can be closer and more matched to the input 2D hairstyle map.

[0054] Step S15: Determine the trained initial model as the 3D hairstyle reconstruction model.

[0055] In this embodiment, after training the initial model, when the training effect of the model reaches the target and meets the requirements, end the training of the model, and determine the initial model at the end of the training as the 3D hairstyle reconstruction model.

[0056] Among them, in an optional implementation, the training may end when the coordinate error between each hair point in the 3D hairstyle map output by the initial model and the label of the initial model (each 3D hair point in the 3D hairstyle map) is less than a preset threshold. The hair points in this embodiment may refer to the key points on the hair or the multiple points that make up the hair. Among them, the preset threshold is the maximum coordinate value error between the model output and the model label that is set in advance according to experience and meets the project requirements. The preset threshold can be adjusted accordingly according to the training effect and manual experience. This embodiment does not limit the specific value of the preset threshold.

[0057] Through the model generation method proposed in this embodiment, each serial unit module can fuse the feature information input from the unit module connected to its upstream data stream, and the decoder can continuously fuse the information of the encoder through multiple parallel 3D convolution modules, so that each unit module in the 3D hairstyle reconstruction model can fuse the feature information input from the unit module connected to its upstream data stream, which also enables the unit modules in the model to continuously fuse the position information of the lower layer of the model, thus ensuring that the model can minimize the reduction of hair position information as the network structure deepens, enabling the trained 3D hairstyle reconstruction model to make full use of the position information of the hair in the image for processing, so that the finally output 3D hairstyle map is closer and more matched to the input 2D human head image, thereby generating a 3D hairstyle reconstruction model that is well applicable to the regression task.

[0058] Combined with the above embodiments, in one implementation, as Figure 2 shown, the encoder and the decoder are respectively composed of multiple unit modules connected in series, and the last unit module in the encoder is connected in series with the first unit module in the decoder; among them, the unit modules in the encoder correspond one by one to the unit modules in the decoder; a 3D convolution module is connected between each unit module in the encoder and the unit module corresponding to it in the decoder.

[0059] In this embodiment, the encoder is composed of multiple identical unit modules connected in series, and the decoder is also composed of multiple identical unit modules connected in series. Moreover, the last unit module in the encoder is connected in series with the first unit module in the decoder, thus forming the encoder-decoder in the model. Among them, the unit modules in the encoder and the unit modules in the decoder correspond one by one. That is to say, each unit module in the encoder has a corresponding unit module in the decoder, and they are symmetric with respect to the connection between the encoder and the decoder. For example, if the encoder and the decoder are respectively composed of 3 unit modules connected in series, then the first unit module in the encoder corresponds to the third unit module in the decoder, the second unit module in the encoder corresponds to the second unit module in the decoder, and the third unit module in the encoder corresponds to the first unit module in the decoder.

[0060] In this embodiment, a 3D convolution module is connected between each unit module in the encoder and the corresponding unit module in the decoder. The 3D convolution module in this embodiment is essentially a 3D convolution operation, which is used to fuse the information of the encoder and the decoder. Continuing with the above example: If the encoder and the decoder are respectively composed of 3 unit modules connected in series, then there is a 3D convolution module connected between the first unit module in the encoder and the third unit module in the corresponding decoder; there is a 3D convolution module connected between the second unit module in the encoder and the second unit module in the corresponding decoder; and there is a 3D convolution module connected between the third unit module in the encoder and the first unit module in the corresponding decoder.

[0061] In this embodiment, the outputs of each module in the initial model are as follows:

[0062] For the unit module: The output obtained after the unit module processes the input will not be directly given to the next unit module or the 3D convolution module connected to it. Instead, the input and output of the unit module will be feature-fused through the direct connection branch (such as a skip connection, which is used to add the input of the module to the output after convolution through a direct connection edge) in the unit module, so as to obtain the final output of the unit module, and then the final output will be given to the next unit module or the 3D convolution module connected to it.

[0063] For the 3D convolution module: The output obtained after the 3D convolution module processes the input will not be directly given to the unit module in the decoder connected to it. Instead, the input and output of the 3D convolution module will be feature-fused through the direct connection branch (such as a skip connection) in the 3D convolution module, so as to obtain the final output of the 3D convolution module, and then the final output will be given to the unit module in the decoder connected to it.

[0064] Among them, the inputs of each module in this embodiment are as follows: The input of the first unit module in the encoder is: the first preprocessed data; the inputs of the other unit modules in the encoder except the first unit module are: the final output of the previous unit module; the input of the 3D convolution module is: the final output of the unit module in the encoder connected to it; the input of the unit module of the decoder is: the final output of the previous unit module, and the final output of the 3D convolution module connected to the unit module of the decoder.

[0065] Through the model structure proposed in this embodiment, each unit module in the encoder corresponds one by one to the unit module in the decoder, and a 3D convolution module is connected between the corresponding unit modules in the encoder and decoder, so that information fusion can be carried out between the encoder and decoder through multiple parallel 3D convolution modules, thereby further continuously fusing the low-level position information in the model, and minimizing the position information in the model as much as possible during the model training process. Each unit module in the encoder of this embodiment can continuously fuse the input information of the previous unit module (that is, the unit module connected upstream of the data stream), while each unit module in the decoder can not only fuse the input information of the previous unit module, but also fuse the input information of the corresponding module in the encoder through the 3D convolution module, so that during the model training process, the unit module can continuously fuse the low-level position information in the model (the position information of the first preprocessed data), so as to ensure that as the model network structure deepens, while ensuring that the semantic information in the unit module increases as much as possible, the position information decreases less, so as to improve the training effect of the model.

[0066] Combined with the above embodiments, in one implementation, considering that high-precision 3D hair data depends on professional manual design to achieve, the acquisition cost will be very high, deep learning requires a large amount of training data to have good results, and there is very little 3D hair data that can be used open source at present. Therefore, how to obtain more 3D hair data based on the existing 3D hair data is a problem that must be solved in this embodiment. Therefore, the embodiment of the present invention also provides a 3D hairstyle acquisition method. In this method, step S11 includes steps S31 to S35. The relationship between steps S31 to S35 is as Figure 3 shown, Figure 3 which is a flowchart of a 3D hairstyle acquisition method shown in an embodiment of the present invention.

[0067] Step S31: Classify each original 3D hairstyle in multiple original 3D hairstyle maps.

[0068] In this embodiment, multiple 3D hairstyle maps can be obtained from the existing 3D hair data that can be used open source as the original 3D hairstyle maps, and each original 3D hairstyle in the original 3D hairstyle maps is classified.

[0069] Step S32: Cluster the root points of two original 3D hairstyles of the same type respectively, and obtain multiple hair bundle center clusters for each of the two original 3D hairstyles.

[0070] In this embodiment, after classifying the original 3D hairstyles, multiple types of original 3D hairstyles can be obtained. Each type of original 3D hairstyle forms a group. Cluster the root points of two original 3D hairstyles belonging to the same type (i.e., belonging to the same group) respectively, and obtain multiple hair bundle center clusters for each of the two original 3D hairstyles. Here, the root point refers to the intersection of each hair and the scalp. In this embodiment, the specific number of clusters divided into hair bundle center clusters can be determined according to the style of the hairstyle and the actual effect. For example, all the hair bundles of each hairstyle can be clustered into 7 hair bundle center clusters. And there are various implementation means for clustering the root points. For example, the k-means algorithm can be used for clustering the root points. This embodiment does not impose any restrictions on this.

[0071] Step S33: Perform pairwise interpolation on the two original 3D hairstyles based on the multiple hair bundle center clusters as the standard, and obtain multiple 3D new hairstyles belonging to the same type.

[0072] In this embodiment, after obtaining multiple hair bundle center clusters for each of the two original 3D hairstyles of the same type, perform pairwise interpolation on the two original 3D hairstyles based on the multiple hair bundle center clusters as the standard: In this embodiment, pairwise interpolation calculations are respectively performed between any hair bundle center cluster in one original 3D hairstyle and any hair bundle center cluster in the other original 3D hairstyle among the two different original 3D hairstyles, so as to obtain multiple 3D new hairstyles belonging to this type. That is to say, each hair bundle center cluster in one original 3D hairstyle of the same type can perform interpolation calculations with each hair bundle center cluster in the other original 3D hairstyle of the same type. Thus, there are various interpolation combination methods, and each interpolation calculation can obtain a 3D new hairstyle. Therefore, multiple 3D new hairstyles belonging to the same type can be obtained through this method. And for multiple types of original 3D hairstyles, this method can be used to obtain multiple 3D new hairstyles belonging to each type.

[0073] For example, as Figure 4 shown, Figure 4It is a schematic diagram of a hairstyle interpolation algorithm shown in an embodiment of the present invention. After classifying the original 3D hairstyles, under the same Q type, there are two original 3D hairstyles, namely the original 3D hairstyle A and the original 3D hairstyle B. The original 3D hairstyle A has 7 hair bundle center clusters: A1, A2, A3, A4, A5, A6, A7, and the original 3D hairstyle B also has 7 hair bundle center clusters: B1, B2, B3, B4, B5, B6, B7. Then, a 3D new hairstyle can be generated by A1 and B1 through the interpolation algorithm, a 3D new hairstyle can be generated by A1 and B2 through the interpolation algorithm, a 3D new hairstyle can be generated by A1 and B3 through the interpolation algorithm... A 3D new hairstyle can be generated by A2 and B1 through the interpolation algorithm, a 3D new hairstyle can be generated by A2 and B2 through the interpolation algorithm... and so on. In this way, pairwise interpolation calculations are performed on the 7 hair bundle center clusters in the two original 3D hairstyles. Theoretically, a total of 49 3D new hairstyles can be generated from the original 3D hairstyle A and the original 3D hairstyle B.

[0074] It should be noted that the key point in this embodiment is to perform clustering on the root hair positions of each original 3D hairstyle to obtain multiple hair bundle center clusters for each original 3D hairstyle, and then perform pairwise interpolation on two original 3D hairstyles belonging to the same type based on the multiple hair bundle center clusters, so as to obtain multiple 3D new hairstyles under each type. Therefore, in this embodiment, it can be to perform clustering on the root hair positions of all original 3D hairstyles under all types to obtain multiple hair bundle center clusters, and then perform pairwise interpolation on the hair bundle center clusters of the original 3D hairstyles under the same type to generate new hairstyles; it can also be to process one type at a time: that is, to perform clustering on the root hair positions of all original 3D hairstyles under the same type to obtain multiple hair bundle center clusters, and then perform pairwise interpolation on the hair bundle center clusters of the original 3D hairstyles under this type to generate new hairstyles; then perform clustering on the root hair positions of all original 3D hairstyles under the next type to obtain multiple hair bundle center clusters, and then perform pairwise interpolation on the hair bundle center clusters of the original 3D hairstyles under the next type to generate new hairstyles. This embodiment does not make any restrictions on this.

[0075] Step S34: Add the 3D new hairstyle diagrams of the multiple 3D new hairstyles belonging to each type obtained to the 3D hairstyle library containing the multiple original 3D hairstyle diagrams.

[0076] In this embodiment, after obtaining multiple 3D new hairstyles belonging to each type, the 3D new hairstyle diagrams of the multiple 3D new hairstyles belonging to each type obtained (that is, the multiple 3D new hairstyle diagrams under each type obtained) are added to the 3D hairstyle library containing multiple original 3D hairstyle diagrams to further expand the 3D hairstyle data in the 3D hairstyle library, provide a training data set for model establishment, and improve the training effect of the model.

[0077] Step S35: Obtain the 3D hairstyle map from the 3D hairstyle library.

[0078] When performing initial model training, obtain the 3D hairstyle map from the 3D hairstyle library that has been augmented with 3D new hairstyle maps, so as to construct the subsequent training sample set.

[0079] In this embodiment, the original 3D hairstyle map is processed by an interpolation algorithm to obtain more 3D new hairstyle maps based on the existing original 3D hairstyle map, so that the initial model can be trained based on as much training data as possible to obtain a better training effect and further improve the output effect of the 3D hairstyle reconstruction model.

[0080] Combined with the above embodiments, in one implementation manner, the present invention also provides a 3D hairstyle classification method. In this method, step S31 includes steps S51 to S54. The relationship between steps S51 to S54 is as Figure 5 shown, Figure 5 which is a flowchart of a 3D hairstyle classification method shown in an embodiment of the present invention.

[0081] Step S51: Map the original 3D hairstyle map into an original 2D hairstyle map.

[0082] In this embodiment, multiple original 3D hairstyle maps are each mapped into an original 2D hairstyle map to obtain multiple original 2D hairstyle maps.

[0083] Step S52: Perform image segmentation on the original 2D hairstyle map to obtain a mask corresponding to the hair region in the original 2D hairstyle map, and divide the hairstyle length according to the mask to obtain a division result.

[0084] In this embodiment, the obtained original 2D hairstyle map can be segmented by an image segmentation algorithm or a dedicated image segmentation model to obtain a mask of the original 2D hairstyle map. The mask (Mask) in this embodiment is a grayscale image with the same size as the original 2D hairstyle map, and the floating-point value corresponding to each pixel is between 0 and 1. And determine the mask corresponding to the hair region in the original 2D hairstyle map from the mask of the original 2D hairstyle map. For example, the floating-point value of the mask corresponding to the hair region is 1, so as to determine the hair region in the original 2D hairstyle map. After determining the mask corresponding to the hair region in the original 2D hairstyle map, divide the hairstyle length according to this mask to obtain a division result: it can be a division of the hairstyle into long, medium, and short according to the hair mask. For example, if it is determined that the tail of the hair passes the chin, it is defined as medium hair; if it is determined that the tail of the hair passes the ear, it is defined as short hair; if it is determined that the tail of the hair passes the shoulder, it is defined as long hair, etc., so as to obtain the division result of whether the hair belongs to long hair, medium hair, or short hair.

[0085] Step S53: Calculate the orientation map for the hair region in the original 2D hairstyle map to determine the curvature of the hair.

[0086] In this embodiment, after determining the hair region in the original 2D hairstyle map through a segmentation algorithm, calculate the orientation map for the hair region in the original 2D hairstyle map to determine the curvature of the hair. For example, it can be to filter the hair region through a Gabor Filter to generate a series of orientation angles, and then determine the curvature of the hair according to the orientation angles, such as determining whether the hair is straight or curly.

[0087] Step S54: Determine the type to which each original 3D hairstyle belongs according to the curvature of the hair and the partitioning result.

[0088] In this embodiment, after determining the partitioning result and the curvature of the hair, the type to which each original 3D hairstyle belongs can be determined according to the partitioning result and the curvature of the hair: for example, it can be based on whether the hair is long, short or medium, and whether the hair is curly or straight, so as to divide the original 3D hairstyles into six types: short straight, short curly, medium straight, medium curly, long straight and long curly, thereby realizing the classification of the original 3D hairstyles.

[0089] In this embodiment, by mapping the original 3D hairstyle into the original 2D hairstyle map representation, and then performing image segmentation and orientation map calculation on the 2D hairstyle map, the automatic classification of the original 3D hairstyle can be accurately performed.

[0090] Combined with the above embodiments, in one implementation, the present invention also provides a data processing method. In this method, step S13 includes steps S61 to S64. The relationship between steps S61 to S64 is as Figure 6 shown, Figure 6 which is a flowchart of a data processing method shown in an embodiment of the present invention.

[0091] Step S61: Process the 2D hairstyle map through a filter to obtain the orientation map of the 2D hairstyle map.

[0092] In this embodiment, process the 2D hairstyle map through a filter to obtain the orientation map of the 2D hairstyle map. Among them, in this embodiment, it can be to process the 2D hairstyle map through a Gabor Filter to obtain the orientation map of the 2D hairstyle map.

[0093] Step S62: Perform image segmentation on the 2D hairstyle map to obtain the mask corresponding to the hair region of the 2D hairstyle map.

[0094] In this embodiment, the method for obtaining the mask corresponding to the hair region of the original 2D hairstyle map in step S62 is the same as that in the above step S52. It can be to perform image segmentation on the 2D hairstyle map through an image segmentation algorithm or a dedicated image segmentation model to obtain the mask of the 2D hairstyle map, and then determine the mask corresponding to the hair region in the 2D hairstyle map from the mask of the 2D hairstyle map.

[0095] Step S63: Process the 2D hairstyle map through a human body contour recognition model to obtain the head contour information of the 2D hairstyle map.

[0096] The human body contour recognition model in this embodiment is specifically used to extract the human body contour information in the image. By processing the 2D head image through the human body contour recognition model, the head contour information in the 2D head image can be obtained.

[0097] Step S64: Use the orientation map of the 2D hairstyle map, the mask corresponding to the hair region of the 2D hairstyle map, and the head contour information of the 2D hairstyle map as the first preprocessing data.

[0098] In this embodiment, after obtaining the orientation map of the 2D hairstyle map, the mask corresponding to the hair region of the 2D hairstyle map, and the head contour information of the 2D hairstyle map, the orientation map of the 2D hairstyle map, the mask corresponding to the hair region of the 2D hairstyle map, and the head contour information of the 2D hairstyle map are used as the first preprocessing data for model training.

[0099] In this embodiment, before model training, after processing the 2D hairstyle map to obtain the orientation map of the 2D hairstyle map, the mask corresponding to the hair region of the 2D hairstyle map, and the head contour information of the 2D hairstyle map, and then inputting the orientation map of the 2D hairstyle map, the mask corresponding to the hair region of the 2D hairstyle map, and the head contour information of the 2D hairstyle map into the initial model for model training, the 3D hairstyle reconstruction model trained through 3D hairstyle data can also be well applicable to the processing of 2D real head images, thus ensuring the reconstruction effect of the 3D hairstyle reconstruction model.

[0100] Combined with the above embodiments, in one implementation, the initial model further includes an initial convolution module, which is arranged upstream of the data stream outside the encoder. This initial convolution module is used to perform dimensionality increase processing on the first preprocessing data to obtain a preliminary feature map as the input of the first unit module in the encoder.

[0101] In this embodiment, the steps executed by each unit module in the encoder and each unit module in the decoder include steps S71 to S74, and the relationship between steps S71 to S74 is as Figure 7 shown, Figure 7 which is a flowchart of information processing of a unit module shown in an embodiment of the present invention.

[0102] Step S71: Extract information from the input of the unit module to obtain a first feature map with reduced resolution.

[0103] In this embodiment, each unit module first extracts information from the input of the unit module. For example, the information extraction of the input is completed through a convolution with a stride of 2, thereby obtaining a first feature map with reduced resolution. Among them, since the unit modules are divided into the first unit module of the encoder, other unit modules of the encoder except the first unit module, and the unit modules in the decoder, and the inputs of these unit modules are different. The following takes the first unit module as an example to illustrate step S71: For the first unit module, it can complete the information extraction of the preliminary feature map through a convolution with a stride of 2, and at this time, the resolution of the feature map will be reduced, such as becoming half of the preliminary feature map, thereby obtaining the first feature map.

[0104] Step S72: Restore the resolution of the first feature map to the input resolution by bilinear interpolation to obtain an intermediate feature map.

[0105] After obtaining the first feature map in this embodiment, the unit module then restores the resolution of the first feature map to the resolution of the unit module input by bilinear interpolation, thereby obtaining an intermediate feature map. When the unit module in this embodiment extracts information, the resolution will decrease. Since certain information will be lost due to the continuous decrease of the resolution, the unit module then restores the resolution of the feature map by bilinear interpolation, so that the unit module can maintain a constant resolution of the feature map and avoid information loss as much as possible.

[0106] Step S73: Increase the number of channels of the intermediate feature map to obtain a final feature map.

[0107] In this embodiment, after obtaining the intermediate feature map, the number of channels of the intermediate feature map can also be increased through a 1*1 convolution. For example, the number of channels is changed to twice that when the unit module inputs, thereby obtaining a final feature map. Since convolution will reduce the input feature information to a certain extent, the unit module in this embodiment makes up for the lost information by increasing the number of channels.

[0108] Step S74: Perform feature fusion on the final feature map and the input through the direct connection branch to obtain the final output of the unit module.

[0109] In this embodiment, there is a direct connection branch between the input and output ends of the unit module, and the unit module can fuse the final feature map and the input of the unit module through the direct connection branch to obtain the final output of the unit module. For example, in this embodiment, a jump connection with a 1*1 convolution can be used to keep the number of feature channels of the input of the unit module consistent with the number of channels of the output of the unit module, thereby facilitating the feature fusion of the input and output of the unit module.

[0110] In this embodiment, by extracting feature information, maintaining constant resolution of feature maps, increasing the number of feature channels, and fusing input and output features within the unit module, the entire model network structure composed of unit modules can avoid feature loss as much as possible, and further maintain the hair position information and semantic information during the model training process.

[0111] In addition, in one embodiment, if Figure 8 As shown, Figure 8 It is a schematic diagram of a model generation method shown in an embodiment of the present invention. Figure 8 The input image in is a 2D hairstyle image. The 2D hairstyle image is processed by Gabort filter, Segmentation and bust model to obtain 2D orientation map, Hair mask and Bust depth respectively. These are then input into the initial convolution module, processed and then input into the encoder-decoder structure. There are three 3D conv modules in parallel between the encoder and the decoder. The 3D convolution module is used to fuse the information between the encoder and the decoder. Figure 8 The “add” in the figure indicates that the 3D convolution module performs feature fusion between the input and output, and the decoder performs corresponding processing and outputs the final result.

[0112] In combination with the above embodiments, in one implementation, the present invention further provides a 3D hairstyle generation method, in which the method includes steps S91 to S93. The relationship between steps S91 and S93 is as follows: Figure 9 As shown, Figure 9 The figure is a flow chart of a 3D hairstyle generation method according to an embodiment of the present invention.

[0113] Step S91: Obtain the 2D hairstyle image to be processed.

[0114] In this embodiment, when 3D hairstyle reconstruction is required, a 2D hairstyle image for 3D hairstyle reconstruction can be obtained, that is, the 2D hairstyle image to be processed is obtained.

[0115] Step S92: Preprocess the 2D hairstyle image to be processed to obtain second preprocessing data corresponding to the 2D hairstyle image to be processed.

[0116] In this embodiment, the obtained 2D hairstyle image to be processed needs to be preprocessed to obtain second preprocessing data corresponding to the 2D hairstyle image to be processed. Among them, the preprocessing method in this step can be the same as that in the above step S13 or the method from step 61 to step 64, and this embodiment does not make any restrictions on this.

[0117] Step S93: Input the second preprocessing data into the 3D hairstyle reconstruction model generated by the model generation method described in any of the above embodiments of the present invention to obtain a 3D hairstyle image output by the 3D hairstyle reconstruction model.

[0118] In this embodiment, the second preprocessing data obtained after preprocessing is input into the pre-trained 3D hairstyle reconstruction model, and then a 3D hairstyle image output by the 3D hairstyle reconstruction model can be obtained. The pre-trained 3D hairstyle reconstruction model in this embodiment is the 3D hairstyle reconstruction model finally trained and generated by the model generation method described in any of the above embodiments of the present invention.

[0119] In this embodiment, the 2D hairstyle image is subjected to 3D hairstyle reconstruction through a pre-trained 3D hairstyle reconstruction model with as little position information as possible reduced, so that a 3D hairstyle with better effect, closer to and more matching the input 2D hairstyle image can be obtained.

[0120] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0121] Based on the same inventive concept, an embodiment of the present invention provides a model generation device 1000. Refer to Figure 10 , Figure 10 is the structural block diagram of the model generation device provided by an embodiment of the present invention. As Figure 10 shown, the device 1000 includes:

[0122] A first image acquisition module 1001, configured to acquire a 3D hairstyle image;

[0123] The first image processing module 1002 is configured to process the 3D hairstyle map into a 2D hairstyle map;

[0124] The first data processing module 1003 is configured to preprocess the 2D hairstyle map to obtain first preprocessing data corresponding to the 2D hairstyle map;

[0125] The model training module 1004 is configured to use the first preprocessing data as an input and the 3D hairstyle map corresponding to the 2D hairstyle map as an output target to train an initial model; wherein, the structure of the initial model is: the initial model includes an encoder and a decoder, both the encoder and the decoder are composed of multiple serial unit modules, and multiple parallel 3D convolution modules are connected between the encoder and the decoder; there are direct connection branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module, and each unit module fuses the feature information input from the unit module connected upstream in the data stream, so that the unit module can continuously fuse the feature information of the first preprocessing data, the feature information at least includes position information, and the position information includes the coordinate information of hair points;

[0126] The model generation module 1005 is configured to determine the trained initial model as a 3D hairstyle reconstruction model.

[0127] Optionally, both the encoder and the decoder are composed of multiple serial unit modules, including:

[0128] The encoder and the decoder are respectively composed of multiple unit modules connected in series, and the last unit module in the encoder is connected in series with the first unit module in the decoder; wherein, the unit modules in the encoder and the unit modules in the decoder correspond one by one;

[0129] Multiple parallel 3D convolution modules are connected between the encoder and the decoder, including: 3D convolution modules are connected between each unit module in the encoder and the corresponding unit module in the decoder.

[0130] Optionally, the first image acquisition module 1001 includes:

[0131] A hairstyle classification module, configured to classify each original 3D hairstyle in multiple original 3D hairstyle maps;

[0132] A clustering module, configured to respectively perform clustering on the root positions of two original 3D hairstyles belonging to the same type to respectively obtain multiple hair bundle center clusters of each original 3D hairstyle in the two original 3D hairstyles;

[0133] A new hairstyle synthesis module for pairwise interpolation of the two original 3D hairstyles based on the multiple hair bundle center clusters to obtain multiple 3D new hairstyles of the same type;

[0134] A 3D hairstyle library construction module for adding the 3D new hairstyle images of the multiple 3D new hairstyles obtained for each type to the 3D hairstyle library containing the multiple original 3D hairstyle images;

[0135] A 3D hairstyle image acquisition module for acquiring 3D hairstyle images from the 3D hairstyle library.

[0136] Optionally, the hairstyle classification module includes:

[0137] An image conversion module for mapping the original 3D hairstyle image into an original 2D hairstyle image;

[0138] A first image segmentation module for performing image segmentation on the original 2D hairstyle image to obtain a mask corresponding to the hair region in the 2D hairstyle image, and dividing the hairstyle length according to the mask to obtain a division result;

[0139] A hair curvature determination module for calculating a direction map of the hair region in the original 2D hairstyle image to determine the curvature of the hair;

[0140] A hairstyle classification sub-module for determining the type to which each original 3D hairstyle belongs according to the curvature of the hair and the division result.

[0141] Optionally, the first data processing module 1003 includes:

[0142] A direction map determination module for processing the 2D hairstyle image through a filter to obtain the direction map of the 2D hairstyle image;

[0143] A second image segmentation module for performing image segmentation on the 2D hairstyle image to obtain a mask corresponding to the hair region of the 2D hairstyle image;

[0144] A human head contour recognition module for processing the 2D hairstyle image through a human body contour recognition model to obtain the human head contour information of the 2D hairstyle image;

[0145] A preprocessing data determination module for using the direction map of the 2D hairstyle image, the mask corresponding to the hair region of the 2D hairstyle image, and the human head contour information of the 2D hairstyle image as the first preprocessing data.

[0146] Optionally, the initial model further includes an initial convolutional module, which is arranged upstream of the data stream outside the encoder. The initial convolutional module is used to perform dimensionality increase processing on the first preprocessed data to obtain a preliminary feature map as the input of the first unit module in the encoder. The apparatus 1000 further includes a unit execution module, which is used to execute the steps that each unit module in the encoder and the decoder needs to execute. The unit execution module includes:

[0147] A first execution module, which is used to extract information from the input of the unit module to obtain a first feature map with reduced resolution;

[0148] A second execution module, which is used to restore the resolution of the first feature map to the input resolution by means of bilinear interpolation to obtain an intermediate feature map;

[0149] A third execution module, which is used to increase the number of channels of the intermediate feature map to obtain a final feature map;

[0150] A fourth execution module, which is used to perform feature fusion on the final feature map and the input through the direct connection branch to obtain the final output of the unit module.

[0151] Based on the same inventive concept, an embodiment of the present invention provides a 3D hairstyle generation apparatus 1100. Refer to Figure 11 , Figure 11 which is the structural block diagram of the 3D hairstyle generation apparatus provided by an embodiment of the present invention. As Figure 11 shown, the apparatus 1100 includes:

[0152] A second image acquisition module 1101, which is used to acquire a to-be-processed 2D hairstyle map;

[0153] A second data processing module 1102, which is used to preprocess the to-be-processed 2D hairstyle map to obtain second preprocessed data corresponding to the to-be-processed 2D hairstyle map;

[0154] A 3D hairstyle acquisition module 1103, which is used to input the second preprocessed data into the 3D hairstyle reconstruction model generated by the model generation method described in any of the above embodiments of the present invention to obtain the 3D hairstyle map output by the 3D hairstyle reconstruction model.

[0155] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps in the model generation method described in any of the above embodiments of the present invention or the steps in the 3D hairstyle generation method described in the above embodiments of the present invention.

[0156] Based on the same inventive concept, another embodiment of the present invention provides an electronic device 1200, as Figure 12 shown. Figure 12 is a schematic diagram of an electronic device shown in an embodiment of the present invention. The electronic device includes a memory 1202, a processor 1201, and a computer program stored on the memory and executable on the processor. When the processor executes, it implements the steps in the model generation method described in any of the above embodiments of the present invention or the steps in the 3D hairstyle generation method described in the above embodiments of the present invention.

[0157] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment.

[0158] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same and similar parts among the embodiments, refer to each other.

[0159] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention can take the form of completely hardware embodiments, completely software embodiments, or embodiments combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0160] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0161] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions in Figure 1 one process or multiple processes and / or blocks Figure 1The functions specified in one or more boxes.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable terminal device provide for implementing the steps of the functions specified in one Figure 1 One process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes.

[0163] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

[0164] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0165] The above has introduced in detail a model generation, 3D hairstyle generation method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A model generation method, characterized in that, The method includes: Obtaining a 3D hairstyle map; Processing the 3D hairstyle map into a 2D hairstyle map; Preprocessing the 2D hairstyle map to obtain first preprocessing data corresponding to the 2D hairstyle map; Using the first preprocessing data as input and the 3D hairstyle map corresponding to the 2D hairstyle map as the output target to train an initial model; wherein, the structure of the initial model is: the initial model includes an encoder and a decoder, both the encoder and the decoder are composed of multiple serial unit modules, and there are multiple parallel 3D convolution modules connected between the encoder and the decoder; there are direct connection branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module, and each unit module fuses the feature information input in the unit module connected upstream in the data stream, so that the unit module can continuously fuse the feature information of the first preprocessing data, and the feature information at least includes position information, and the position information includes the coordinate information of hair points; Determining the trained initial model as a 3D hairstyle reconstruction model; The preprocessing the 2D hairstyle map to obtain first preprocessing data corresponding to the 2D hairstyle map includes: Processing the 2D hairstyle map through a filter to obtain the orientation map of the 2D hairstyle map; Performing image segmentation on the 2D hairstyle map to obtain a mask corresponding to the hair region of the 2D hairstyle map; Processing the 2D hairstyle map through a human body contour recognition model to obtain the human head contour information of the 2D hairstyle map; Using the orientation map of the 2D hairstyle map, the mask corresponding to the hair region of the 2D hairstyle map, and the human head contour information of the 2D hairstyle map as the first preprocessing data.

2. The method according to claim 1, wherein Both the encoder and the decoder are composed of multiple serial unit modules, including: the encoder and the decoder are respectively composed of multiple unit modules connected in series, and the last unit module in the encoder is connected in series with the first unit module in the decoder; wherein, the unit modules in the encoder correspond one by one to the unit modules in the decoder; There are multiple parallel 3D convolution modules connected between the encoder and the decoder, including: there are 3D convolution modules connected between each unit module in the encoder and the unit module corresponding to it in the decoder.

3. The method according to claim 1, wherein The obtaining a 3D hairstyle map includes: Classifying each of multiple original 3D hairstyles in the multiple original 3D hairstyle maps; Performing clustering on the hair root positions of two original 3D hairstyles belonging to the same type respectively to obtain multiple hair bundle center clusters for each of the two original 3D hairstyles; Performing pairwise interpolation on the two original 3D hairstyles based on the multiple hair bundle center clusters to obtain multiple 3D new hairstyles belonging to the same type; Adding the 3D new hairstyle maps of the multiple 3D new hairstyles obtained for each type to a 3D hairstyle library containing the multiple original 3D hairstyle maps; Obtaining a 3D hairstyle map from the 3D hairstyle library.

4. The method according to claim 3, wherein Classifying each of the multiple original 3D hairstyle images includes: Mapping the original 3D hairstyle image into an original 2D hairstyle image; Performing image segmentation on the original 2D hairstyle image to obtain a mask corresponding to the hair region in the original 2D hairstyle image, and dividing the hairstyle length according to the mask to obtain a division result; Calculating a direction map for the hair region in the original 2D hairstyle image to determine the curvature of the hair; Determining the type to which each original 3D hairstyle belongs according to the curvature of the hair and the division result.

5. The method according to claim 2, wherein The initial model further includes an initial convolution module, which is arranged upstream of the data stream outside the encoder. The initial convolution module is used to perform dimensionality increase processing on the first preprocessed data to obtain a preliminary feature map as the input to the first unit module in the encoder; The steps performed by each unit module in the encoder and the decoder include: Extracting information from the input of the unit module to obtain a first feature map with reduced resolution; Restoring the resolution of the first feature map to the input resolution by means of bilinear interpolation to obtain an intermediate feature map; Increasing the number of channels of the intermediate feature map to obtain a final feature map; Performing feature fusion on the final feature map and the input through the direct connection branch to obtain the final output of the unit module.

6. A 3D hairstyle generation method, characterized in that, The method includes: Obtaining a 2D hairstyle image to be processed; Performing preprocessing on the 2D hairstyle image to be processed to obtain second preprocessed data corresponding to the 2D hairstyle image to be processed; Inputting the second preprocessed data into a 3D hairstyle reconstruction model generated by the model generation method according to any one of claims 1 to 5 to obtain a 3D hairstyle image output by the 3D hairstyle reconstruction model.

7. A model generation device, characterized in that The device includes: A first image acquisition module for acquiring a 3D hairstyle image; A first image processing module for processing the 3D hairstyle image into a 2D hairstyle image; A first data processing module for performing preprocessing on the 2D hairstyle image to obtain first preprocessed data corresponding to the 2D hairstyle image; A model training module for using the first preprocessed data as an input and the 3D hairstyle image corresponding to the 2D hairstyle image as an output target to train an initial model; wherein, the structure of the initial model is: the initial model includes an encoder and a decoder, both the encoder and the decoder are composed of multiple serial unit modules, and there are multiple parallel 3D convolution modules connected between the encoder and the decoder; there are direct connection branches between the input end and the output end of the unit module and between the input end and the output end of the 3D convolution module. Each unit module fuses the feature information input from the unit module connected upstream of the data stream, so that the unit module can continuously fuse the feature information of the first preprocessed data. The feature information at least includes position information, and the position information includes the coordinate information of the hair points; A model generation module for determining the trained initial model as a 3D hairstyle reconstruction model; The first data processing module is specifically configured to process the 2D hairstyle image through a filter to obtain the orientation map of the 2D hairstyle image; perform image segmentation on the 2D hairstyle image to obtain a mask corresponding to the hair region of the 2D hairstyle image; process the 2D hairstyle image through a human contour recognition model to obtain the human head contour information of the 2D hairstyle image; and use the orientation map of the 2D hairstyle image, the mask corresponding to the hair region of the 2D hairstyle image, and the human head contour information of the 2D hairstyle image as the first preprocessed data.

8. A 3D hairstyle generation device, characterized in that, The device includes: A second image acquisition module, configured to acquire a 2D hairstyle image to be processed; A second data processing module, configured to preprocess the 2D hairstyle image to be processed to obtain second preprocessed data corresponding to the 2D hairstyle image to be processed; A 3D hairstyle acquisition module, configured to input the second preprocessed data into a 3D hairstyle reconstruction model generated by the model generation method according to any one of claims 1 to 5, and obtain a 3D hairstyle image output by the 3D hairstyle reconstruction model.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images