A multi-view hair modeling method and system based on deep learning

Abstract

The application provides a multi-view hair modeling method and system based on deep learning. The application first acquires a growth direction map of a two-dimensional image and a depth map of a head model under each view; constructs a volume space of hair growth for the head model, and acquires a plurality of query points in the volume space of hair growth; takes the growth direction map of the two-dimensional image, the depth map of the head model and three-dimensional coordinates of each query point as input of a trained multi-view reconstruction network, and obtains an occupancy value and a growth direction of each query point; and finally grows hair on the head model according to the occupancy value and the growth direction of the query point, and completes hair modeling. The application combines the advantages of two methods, can use fewer inputs, more efficiently reconstruct accurate hair geometry, and can be widely deployed and used in human digitization scenarios.

Description

Technical Field

[0001] This invention belongs to the fields of computer graphics and computer vision, and particularly relates to a multi-view hair modeling method and system based on deep learning. Background Technology

[0002] Existing methods for reconstructing hair geometry from images can be broadly categorized into two types based on the number of input images: hair modeling using multiple images as input and hair modeling using a single image as input. The former, based on traditional multi-view reconstruction techniques, uses feature matching to solve for depth values, reconstructing a coarse hair point cloud and then constructing hair strands within the point cloud according to the hair's growth direction. This type of method can reconstruct relatively accurate hair geometry, but it often requires complex acquisition equipment (simultaneous shooting by dozens of cameras, uniform lighting from all directions) and long processing times (the processing time for a single model is typically over one hour), making it difficult to deploy and use widely. On the other hand, the latter uses a frontal photograph of a person as input, employing data-driven or deep learning methods to construct a spatial vector field representing the hair's growth direction, allowing the construction of hair strands within this vector field. The advantage of this type of method is that it can use common portrait images as input, making it relatively easy to reconstruct reasonable hair structures. However, it often only reconstructs the approximate hair shape and structure, lacking detail, and it is difficult to model hair types missing from the training data. Furthermore, since the input information is only a frontal image, the hair reconstructed from a single image is only similar to the original image from the frontal view, and cannot be accurately modeled from the side and back views. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing a deep learning-based multi-view hair modeling method and system. This invention combines the advantages of two existing methods, enabling the efficient (processing time for a single model is approximately 1 minute) reconstruction of accurate hair geometry using fewer input images (sparse viewpoints). This invention aims to widely deploy and apply high-quality hair modeling technology in the context of human digitalization, efficiently reconstructing accurate hair geometry.

[0004] The specific technical solution adopted in this invention is as follows:

[0005] A deep learning-based multi-view hair modeling method includes:

[0006] Construct a head model and obtain two-dimensional images of the hair to be reconstructed from multiple perspectives;

[0007] Obtain the growth direction map of the 2D image and the depth map of the head model from each viewpoint;

[0008] Construct a volumetric space for hair growth in the head model, and obtain multiple query points within the volumetric space for hair growth.

[0009] The growth direction map of the 2D image from each viewpoint, the depth map of the head model, and the 3D coordinates of each query point are used as inputs to a trained multi-view reconstruction network to obtain the occupancy value σ and growth direction d of each query point.

[0010] Hair is grown on the head model based on the occupancy value σ of the query point and the growth direction d, thus completing the hair modeling.

[0011] The multi-view reconstruction network consists of a backbone feature extraction network, a feature aggregation module, an information aggregation module, and a decoding module. The backbone feature extraction network is used to extract the feature map f for each viewpoint based on the overlay map formed by superimposing the growth direction map and depth map along the channel dimension. i ;

[0012] The feature aggregation module is used to aggregate the query point X features f obtained from each perspective. i (x i ) and the corresponding camera coordinates p i (X), to obtain the representation vector of the query point X from each viewpoint. Wherein, the query point X feature f i (x i Based on the two-dimensional coordinates x of each query point X in the i-th viewpoint i Feature maps f from various perspectives i Found;

[0013] The information aggregation module is used to aggregate the representation vectors of query point X from all perspectives. The three-dimensional feature vector of the query point X is obtained by aggregating the representation vector q(X) corresponding to the query point X.

[0014] The decoding module is used to determine the three-dimensional feature vector of query point X. Decoding yields the occupancy value σ and the growth direction d;

[0015] The trained multi-view reconstruction network is obtained by training on a training dataset with the goal of minimizing the loss between the occupancy value σ and the growth direction d of each query point in the output and the ground truth.

[0016] Furthermore, the volumetric space for hair growth is constructed for the head model, and the size of the constructed volumetric space for hair growth is set to twice the bounding box of the head model.

[0017] Furthermore, the query point is the corner point of each voxel in the volume space of hair growth. The hair modeling is completed by growing hair on the head model based on the occupancy value σ of the query point and the growth direction d. Specifically, the points are sampled uniformly in the volume space of growth, and hair is grown bidirectionally along the growth direction field. The hair growth direction at any point in the space is the result of trilinear interpolation of the growth direction at the corner point of the voxel containing that point.

[0018] Furthermore, each set of training data in the training dataset includes:

[0019] Two-dimensional images of the same hair model from multiple perspectives, occupancy value and growth direction of each query point sampled from the hair model.

[0020] Furthermore, each query point based on the hair model sampling is obtained by the following method: the space around the hair model is voxelized, and if a voxel is passed through by any hair strand, it is considered a positive voxel, otherwise it is considered a negative voxel. Then, the positive voxels and the negative voxels adjacent to the positive voxels are densely sampled, and the remaining negative voxels are sparsely sampled as query points.

[0021] Furthermore, the loss function used for training is as follows:

[0022]

[0023]

[0024] Where Locc is the loss function for the occupied value, σ * d represents the truth value of the occupied value. * is the truth value of the growth direction, and N is the total number of query points.

[0025] Furthermore, after growing hair on the head model based on the occupancy value σ of the query point and the growth direction d, a fine-tuning step is also included:

[0026] Each 3D hair strand obtained after growing hair on the head model based on the occupancy value σ of the query point and the growth direction d is projected onto its visible viewpoint to form a 2D hair strand. Each projected 2D hair strand is deformed in segments according to the guide hair strand that best matches it. The deformed 2D hair strands from multiple viewpoints are aggregated into the final deformed 3D hair strand through back projection. The guide hair strand that best matches it is obtained by selecting from the 2D image corresponding to the projection viewpoint.

[0027] Furthermore, the specific steps for obtaining the growth direction map of the two-dimensional image from each viewpoint are as follows:

[0028] The two-dimensional image from each viewpoint is input into a directional Gabor filter for convolution. The most significant hair growth direction of each pixel is determined, resulting in a noisy growth direction map and a confidence map for measuring the accuracy of the growth direction map from the corresponding viewpoint.

[0029] A noisy growth orientation map and a confidence map for measuring the accuracy of the growth orientation map are used as inputs to a trained denoising convolutional network, and the output is the growth orientation map of the corresponding view.

[0030] The trained denoising convolutional network is obtained by training on a training dataset with the goal of minimizing the loss between the output growth direction map and the ground truth.

[0031] A deep learning-based multi-view hair modeling system, used to implement the aforementioned deep learning-based multi-view hair modeling method, includes:

[0032] The data preprocessing unit is used to construct a head model and obtain two-dimensional images of the hair to be reconstructed from multiple perspectives; it also obtains the growth direction map of the two-dimensional image and the depth map of the head model from each perspective.

[0033] The query point acquisition unit is used to construct the volume space for hair growth in the head model and acquire multiple query points within the volume space for hair growth.

[0034] The hair modeling unit takes the growth direction map of the 2D image from each viewpoint, the depth map of the head model, and the 3D coordinates of each query point as input to a trained multi-view reconstruction network to obtain the occupancy value σ and growth direction d of each query point; and grows hair on the head model based on the occupancy value σ and growth direction d of the query point to complete hair modeling.

[0035] A storage medium containing computer-executable instructions that, when executed by a computer processor, implement the deep learning-based multi-view hair modeling method described above.

[0036] The beneficial effects of this invention are as follows: Compared with traditional multi-view reconstruction methods, this invention reduces the number of input images required while achieving similar modeling quality. It can reconstruct the geometric structure of hair using sparser viewpoints (4 or more images), and the camera equipment is easier to deploy. Simultaneously, the construction time for a single hair model is reduced from over 1 hour to approximately 1 minute, significantly improving modeling efficiency and making it suitable for large-scale testing. Compared with monocular data-driven reconstruction methods, previous modeling results only provided a coarse reconstruction of the hair visible in the front, while the modeling results of this invention reconstruct the complete target hair structure (including the sides and back) and significantly possess more details consistent with the original image.

[0037] This invention combines the advantages of the two methods mentioned above, enabling the efficient (processing time for a single model is approximately 1 minute) reconstruction of accurate hair geometry using fewer input images (sparse viewpoints). The invention aims to widely deploy and apply high-quality hair modeling technology in human digitization scenarios, efficiently reconstructing accurate hair geometry. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the overall pipeline of the present invention;

[0039] Figure 2 This is a schematic diagram of hair strand deformation from multiple perspectives; where a is a two-dimensional image, b is a growth direction diagram, c is a schematic diagram of a two-dimensional hair strand formed after projection from each visible viewpoint, d is the deformation process of a two-dimensional hair strand, and e is a schematic diagram of two-dimensional hair strands being aggregated into a final deformed three-dimensional hair strand through back projection.

[0040] Figure 3 These are four sets of comparison images of the hair modeling results and two-dimensional images from this invention. Detailed Implementation

[0041] The present invention will be further described below with reference to specific embodiments and accompanying drawings.

[0042] This invention provides a deep learning-based multi-view hair modeling method, such as... Figure 1 As shown, it specifically includes:

[0043] Step 1: Obtain 2D images of the hair to be reconstructed from multiple perspectives, and obtain the growth direction map of the 2D image from each perspective; construct a head model based on the 2D images from multiple perspectives, and obtain the depth map of the head model;

[0044] In this context, the two-dimensional images of the hair to be reconstructed from multiple perspectives refer to two-dimensional images containing the hair to be reconstructed, which can be obtained by taking photos with a camera from different perspectives. To obtain better three-dimensional reconstruction results, at least two-dimensional images from the front, back, left, and right sides are generally included.

[0045] Obtaining the growth orientation map of a 2D image from each viewpoint can be achieved using conventional filters, but the directly obtained growth orientation map has low accuracy and contains significant noise in the dark and blurred areas of the original image, affecting the accuracy of subsequent hair modeling. Therefore, preferably, the growth orientation map can be further denoised to improve the accuracy of hair modeling. For example, this embodiment provides a method for obtaining the growth orientation map of a 2D image from each viewpoint based on a trained denoising convolutional network, as detailed below:

[0046] For each viewpoint of the 2D image, the hair region is first segmented, and then convolution is performed within the hair region using a directional Gaussian filter to determine the most significant hair growth direction for each pixel, resulting in a noisy orientation map and a confidence map for measuring the accuracy of the orientation map.

[0047] A noisy growth orientation map and a confidence map for measuring the accuracy of the growth orientation map are used as inputs to a trained denoising convolutional network, and the output is the growth orientation map of the corresponding view.

[0048] The trained denoising convolutional network is obtained by training on a simulation dataset with the goal of minimizing the loss between the output growth direction map and the ground truth.

[0049] Step 2: Construct a volumetric space for hair growth in the head model, and obtain multiple query points within the volumetric space for hair growth;

[0050] The volume space for hair growth should cover the entire hair area of ​​most hairstyles as much as possible, and is generally set to twice the bounding box of the head model. In this embodiment, a volume space size of 0.5m × 0.6m × 0.8m is specifically set.

[0051] Step 3: Use the growth direction map of the 2D image from each viewpoint, the depth map of the head model, and the 3D coordinates of each query point as input to a trained multi-view reconstruction network to obtain the occupancy value σ and growth direction d of each query point.

[0052] The multi-view reconstruction network HairMVSNet learns the mapping from the observed 2D features of a query point to the spatial occupancy value and 3D growth direction of that query point. HairMVSNet learns the mapping of local features; therefore, compared to directly learning global features, it can better preserve detailed information, and based on the generalization ability of local features, it is easier to transfer to new, unseen hairstyles. HairMVSNet is defined as follows:

[0053] H(X, {Dir, Dep}1,..., {Dir, Dep} n )=(σ,d)

[0054] In the formula, n is the number of views. HairMVSNet H uses an implicit expression. It takes a query point X and the previously preprocessed orientation map Dir and depth map Dep as input, and outputs the space occupancy value σ and growth direction d at the query point X.

[0055] Specifically, HairMVSNet consists of a backbone feature extraction network, a feature aggregation module, an information aggregation module, and a decoding module connected in sequence.

[0056] First, the growth direction map and depth map from each viewpoint are superimposed along the channel dimension and input into a backbone feature extraction network F to extract the feature map f from each viewpoint. i :

[0057] f i =F({Dir, Dep}) i )

[0058] i represents the i-th viewpoint; the present invention preferably uses a relatively lightweight U-Net as the backbone feature extraction network F, balancing efficiency and feature extraction quality. It can be replaced by other networks that can be used for prediction of dense features, such as the ResNet series backbone network, achieving similar results, but with some differences in efficiency and feature extraction quality.

[0059] Then, from the feature map f of each viewpoint i In the process, extract pixel-aligned features f at the camera projection of the query point X. i (x i Since the direction of hair growth is an anisotropic feature, meaning that the two-dimensional growth direction observed from different viewpoints is different, the image features f under each viewpoint are... i (x i In addition to this, features related to viewpoint information were also utilized, including the coordinates p of the query point X in the camera coordinate system corresponding to each viewpoint. i (X) is the point embedding, and a learnable viewpoint embedding parameter vector e i The query point X features f obtained from each viewpoint i (x i ) and the corresponding camera coordinates p i (X) is input to the feature aggregation module g to obtain the representation vector of the query point X from each viewpoint. The complete representation vector for each viewpoint is:

[0060] φ i =g(f i (x i ), p i (X))+e i

[0061] x i p represents the two-dimensional coordinates of query point X from the i-th viewpoint. i(X) represents the coordinates of sampling point X in the camera coordinate system corresponding to the i-th viewpoint. In this embodiment, g is an MLP network used to aggregate feature vectors. These viewpoint information feature parameters effectively help HairMVSNet learn the interrelationships between different viewpoints, significantly improving the results in terms of visual effects and statistical metrics.

[0062] To aggregate representation information from multiple perspectives, the aggregation method needs to be able to accept any number of unordered input feature vectors, and further aggregate the representation vectors of the query point X from all perspectives. The query point X and its corresponding representation vector q(X) are input to the information aggregation module E to aggregate and obtain the three-dimensional feature vector of the query point X.

[0063] E(q(X), φ1, ..., φ) n )=Φ X

[0064] Here, q(X) is the representation vector used for the query, which integrates the coordinate information of the query point X. Specifically, q is a learnable parameter vector, which is concatenated with the x-coordinate and passed through a fully connected layer to obtain q(X). n is the number of viewpoints; in this embodiment, a transformer model is used as the information aggregation module. This transformer model receives image information and viewpoint information as input, and therefore can effectively aggregate features from different viewpoints, ultimately outputting an aggregated three-dimensional feature vector representing the query point X. Finally, a decoding module is used to decode the three-dimensional feature vector Φ to obtain the spatial occupancy value σ of the query point X (1 for hair passing through, 0 for hair not passing through) and the growth direction d. In this embodiment, an MLP network is used as the decoding module, as shown below:

[0065] MLP(Φ X )=(σ,d)

[0066] The trained multi-view reconstruction network is obtained by training on a training dataset with the goal of minimizing the loss between the occupancy value σ and the growth direction d of each query point in the output and the ground truth.

[0067] The training dataset uses simulated hair models, such as the publicly available USC-HairSalon dataset. This dataset contains diverse hair models, including long, short, straight, and curly hair types, to train a model with good generalization ability. Each training dataset includes:

[0068] (1) Two-dimensional images of the same simulated hair model from multiple perspectives:

[0069] In order to ensure that the camera viewpoint roughly covers the entire hair area while maintaining versatility, this embodiment initially sets the virtual camera viewpoint to be evenly distributed around the hair model, and adds random perturbations to the camera pose.

[0070] (2) Occupancy value and growth direction of each query point sampled from the simulated hair model:

[0071] Since individual hair strands occupy very little physical space, this embodiment voxels the space surrounding the hair model. A voxel is considered positive if any hair strand passes through it, and negative otherwise. Then, dense sampling is performed on the positive voxels and their neighboring negative voxels, while sparse sampling is performed on the remaining negative voxels as query points. Query points sampled on positive voxels have a positive occupancy value, while those sampled on negative voxels have a occupancy value of 0. The hair growth direction is calculated for query points with positive occupancy values, yielding the required occupancy value and growth direction for each query point. This approach allows HairMVSNet to focus its learning on the hair region while roughly balancing the positive and negative samples. For predicting the hair growth direction, points on the original hair strands are used as sampling points to preserve the original high resolution.

[0072] As a preferred embodiment, this embodiment provides a training loss function. Specifically, the space occupancy value is defined as a binary classification problem, and cross-entropy loss is used for training.

[0073]

[0074] σ * This represents the true occupancy value, which in this embodiment is the occupancy value obtained through simulation; the three-dimensional hair growth direction is trained using the average L-1 loss function.

[0075]

[0076] d * is the truth value of the growth direction, and N is the total number of query points.

[0077] Step 4: Based on the occupancy value σ of the query point and the growth direction d, grow hair on the head model to complete hair modeling;

[0078] In this step, hair can be grown bidirectionally along the growth direction field based directly on the sampled query points. As a preferred approach, to efficiently generate hair strands, the space surrounding the target hair (the volume space for hair growth) can be voxelized first, and the occupancy value and growth direction of each voxel's corner point can be queried using HairMVSNet. Then, points are uniformly sampled within the growth volume space, and hair is grown bidirectionally along the growth direction field. The hair growth direction at any point in the space is the result of trilinear interpolation of the growth direction at the corner point of the voxel containing that point.

[0079] The hair strands generated in the above steps are roughly similar to the input 2D image, but may still be too smooth because voxel resolution cannot be infinitely high, otherwise it would greatly reduce reconstruction efficiency. Since the observed structure of the 3D hair strands should match the structure of its 2D projection from its visible viewpoint, as a preferred approach, after growing hair on the head model based on the occupancy value σ of the query point and the growth direction d, a fine-tuning step is also included, specifically:

[0080] like Figure 2 As shown, for each input 2D image, within its hair region, 2D hair strands are grown along a 2D growth pattern, as follows. Figure 2 As shown in b. These two-dimensional hair strands contain the original input two-dimensional image ( Figure 2 Most of the details in a) can be used to guide deformation. Then, each 3D hair strand is projected onto its visible viewpoint, forming a 2D hair strand after projection from each visible viewpoint. Figure 2 c). Each projected two-dimensional hair strand deforms segment by segment according to the guide hair strand that best matches it. Figure 2 d), this deformation process is not affected by intersecting, irrelevant hairs ( Figure 2 The influence of the horizontal curve (d). The deformed two-dimensional hair strands from multiple perspectives are back-projected and aggregated into the final deformed three-dimensional hair strands (d). Figure 2 e). By revisiting the input 2D images from various perspectives, the deformation of the 3D hair strands is guided by the 2D hair structure information of the 2D images to fine-tune the hair strand structure, making it closer to the original input 2D image, thereby effectively improving the accuracy of hair modeling.

[0081] Figure 3 These are four sets of comparison images of the hair modeling results of this invention with two-dimensional images. As can be seen from the images, the method of this invention can reconstruct accurate hair geometry more efficiently (the processing time for a single model is about 1 minute) using fewer input images (sparse viewpoints). Furthermore, the modeling results of this invention restore the complete target hair structure (including the side and back) and significantly have more details consistent with the original image.

[0082] Corresponding to the aforementioned embodiment of a deep learning-based multi-view hair modeling method, the present invention also provides an embodiment of a deep learning-based multi-view hair modeling system.

[0083] The present invention provides a deep learning-based multi-view hair modeling system, comprising:

[0084] The data preprocessing unit is used to construct a head model and obtain two-dimensional images of the hair to be reconstructed from multiple perspectives; it also obtains the growth direction map of the two-dimensional image and the depth map of the head model from each perspective.

[0085] The query point acquisition unit is used to construct the volume space for hair growth in the head model and acquire multiple query points within the volume space for hair growth.

[0086] The hair modeling unit takes the growth direction map of the 2D image from each viewpoint, the depth map of the head model, and the 3D coordinates of each query point as input to a trained multi-view reconstruction network to obtain the occupancy value σ and growth direction d of each query point; and grows hair on the head model based on the occupancy value σ and growth direction d of the query point to complete hair modeling.

[0087] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0088] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a deep learning-based multi-view hair modeling method as described in the above embodiments.

[0089] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., mounted on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0090] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A multi-view hair modeling method based on deep learning, characterized in that, include: Construct a head model and obtain two-dimensional images of the hair to be reconstructed from multiple perspectives; Obtain the growth direction map of the 2D image and the depth map of the head model from each viewpoint; Construct a volumetric space for hair growth in the head model, and obtain multiple query points within the volumetric space for hair growth. The growth direction map of the 2D image from each viewpoint, the depth map of the head model, and the 3D coordinates of each query point are used as inputs to a trained multi-view reconstruction network to obtain the occupancy value of each query point. and the growth direction d; Based on the occupancy value of the query point Hair is grown on the head model in the direction d, thus completing the hair modeling; The multi-view reconstruction network consists of a backbone feature extraction network, a feature aggregation module, an information aggregation module, and a decoding module. The backbone feature extraction network is used to extract the feature map f for each viewpoint based on the overlay map formed by superimposing the growth direction map and depth map along the channel dimension. i ; The feature aggregation module is used to aggregate the query point X features f obtained from each perspective. i (x i ) and the corresponding camera coordinate system coordinates p i (X), to obtain the representation vector of the query point X from each viewpoint. Wherein, the query point X features f i (x i Based on the two-dimensional coordinates x of each query point X in the i-th viewpoint i Feature maps f from various perspectives i Found; The information aggregation module is used to aggregate the representation vectors of query point X from all perspectives. The three-dimensional feature vector of the query point X is obtained by aggregating the representation vector q(X) corresponding to the query point X. ; The decoding module is used to determine the three-dimensional feature vector of query point X. Decode to obtain the occupied value and growth direction d; The trained multi-view reconstruction network is based on the training dataset and minimizes the occupancy value of each query point in the output. The loss is obtained by training with the growth direction d as the target and the true value as the objective.

2. The method according to claim 1, characterized in that, The volumetric space for hair growth is constructed on the head model, and the size of the constructed volumetric space for hair growth is set to twice the bounding box of the head model.

3. The method according to claim 1, characterized in that, The query point is the corner point of each voxel in the volume space of hair growth, and the occupancy value of the query point is used as a basis. Hair is grown on the head model along the growth direction d to complete the hair modeling. Specifically, points are sampled uniformly within the growth volume space, and hair is grown bidirectionally along the growth direction field. The hair growth direction at any point in the space is the result of trilinear interpolation of the growth direction at the corner of the voxel containing that point.

4. The method according to claim 1, characterized in that, Each set of training data in the training dataset includes: Two-dimensional images of the same hair model from multiple perspectives, occupancy value and growth direction of each query point sampled from the hair model.

5. The method according to claim 4, characterized in that, Each query point based on the hair model sampling is obtained by the following method: the space around the hair model is voxelized. If a voxel is passed through by any hair strand, it is considered a positive voxel, otherwise it is considered a negative voxel. Then, the positive voxels and the negative voxels adjacent to the positive voxels are densely sampled, and the remaining negative voxels are sparsely sampled as query points.

6. The method according to claim 1, characterized in that, The loss function used for training is as follows: Where Locc is the loss function for the occupancy value. Indicates the true value of the occupied value. is the truth value of the growth direction, N is the total number of query points, and Lori represents the loss function of the three-dimensional hair growth direction, using the average L-1 loss function.

7. The method according to claim 1, characterized in that, Based on the occupancy value of the query point After hair is grown on the head model in the growth direction d, fine-tuning steps are also included: Based on the occupancy value of the query point Each three-dimensional hair strand obtained after growing hair on the head model in the growth direction d is projected onto its visible viewpoint to form a two-dimensional hair strand. Each projected two-dimensional hair strand is deformed in segments according to the guide hair strand that best matches it. The deformed two-dimensional hair strands from multiple viewpoints are aggregated into the final deformed three-dimensional hair strand through back projection. The guide hair strand that best matches it is obtained by selecting from the two-dimensional images corresponding to the projection viewpoint.

8. The method according to claim 1, characterized in that, The specific steps for obtaining the growth direction map of the two-dimensional image from each viewpoint are as follows: The two-dimensional image from each viewpoint is input into a directional Gabor filter for convolution. The most significant hair growth direction of each pixel is determined, resulting in a noisy growth direction map and a confidence map for measuring the accuracy of the growth direction map from the corresponding viewpoint. A noisy growth orientation map and a confidence map for measuring the accuracy of the growth orientation map are used as inputs to a trained denoising convolutional network, and the output is the growth orientation map of the corresponding view. The trained denoising convolutional network is obtained by training on a training dataset with the goal of minimizing the loss between the output growth direction map and the ground truth.

9. A multi-view hair modeling system based on deep learning, characterized in that, To implement the deep learning-based multi-view hair modeling method according to any one of claims 1-8, comprising: The data preprocessing unit is used to construct a head model and obtain two-dimensional images of the hair to be reconstructed from multiple perspectives; it also obtains the growth direction map of the two-dimensional image and the depth map of the head model from each perspective. The query point acquisition unit is used to construct the volume space for hair growth in the head model and acquire multiple query points within the volume space for hair growth. The hair modeling unit takes the growth direction map of the 2D image from each viewpoint, the depth map of the head model, and the 3D coordinates of each query point as input to a trained multi-view reconstruction network to obtain the occupancy value of each query point. and growth direction d; based on the occupancy value of the query point Hair is grown on the head model in the direction of growth d to complete the hair modeling.

10. A storage medium containing computer-executable instructions that, when executed by a computer processor, implement the deep learning-based multi-view hair modeling method as described in any one of claims 1-8.

Images