Digital human hair style three-dimensional reconstruction method based on multi-view structure prior

Abstract

The invention discloses a digital human hair style three-dimensional reconstruction method based on multi-view structure prior. The method comprises the following steps: making a multi-view data set of a hair growth direction; training a hair direction prediction network by using the data set, randomly selecting a target visual angle and eight adjacent visual angles, extracting camera features through Nerf coding, splicing the camera features with multi-visual-angle image features extracted by a Dinov2 network, and combining depth and rough hair direction features extracted by a CNN network after cross-attention fusion to obtain a hair direction prediction result; decoding to generate a predicted fine hair pattern as a multi-view structure prior, and constructing a loss function by using the L1 distance and the SSIM loss to complete training; the multi-view structure is injected into the hidden codes of the StandDecoder network in a priori mode to generate hairlines, the growth direction of the hairlines is mapped to a two-dimensional plane, the loss in the cosine direction is calculated, and the three-dimensional hairstyle with the coherent direction is obtained. According to the method, the structured intermediate representation of the 2D appearance and the 3D hairline geometry is established, and the reconstruction quality is improved.

Description

Technical Field

[0001] This invention relates to the technical fields of deep learning and 3D modeling, and in particular to a 3D reconstruction method for digital human hairstyles based on multi-view structural priors. Background Technology

[0002] 3D digital human hair reconstruction is one of the core technologies for immersive scenes such as games, virtual-real fusion, and metaverse. However, the complex geometry of hair, frequent self-occlusion, and the ambiguous mapping relationship between 2D appearance and 3D structure pose significant challenges to existing reconstruction methods.

[0003] Existing hair reconstruction techniques mainly fall into two categories: one is conditional generation-based methods, which infer the global orientation field by learning from priors driven by a dataset, but struggle to capture fine-grained geometric details and local orientation changes; the other is optimization-based methods, which use orientation priors obtained from 2D images through Gabor filtering to regularize and constrain hair geometry, but are sensitive to visual artifacts such as occlusion, lighting changes, and background interference, easily producing orientational errors and tangled reconstruction results. Furthermore, some methods, due to their reliance on single-view input or the use of overly smoothing modeling strategies, fail to restore hair-level details, ultimately leading to fragmented local structures and inconsistent global orientations in the reconstruction results, making it difficult to meet the requirements of physical realism and immersive applications.

[0004] The lack of strong geometric priors is the core bottleneck of existing technologies. RGB images alone cannot solve the inverse problem of hair reconstruction ambiguity. There is an urgent need for a structured intermediate representation that can bridge 2D appearance and 3D hair geometry to provide stable and coherent directional guidance for reconstruction. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings and deficiencies of the existing technology and propose a digital human hairstyle 3D reconstruction method based on multi-view structural priors. This method can predict the hair growth direction of hairstyle images from multiple perspectives, serving as a powerful prior for higher quality digital human hairstyle 3D reconstruction and solving the problem that the hairstyle 3D reconstruction results do not conform to the laws of the physical world.

[0006] To achieve the above objectives, the technical solution provided by this invention is: a three-dimensional reconstruction method for digital human hair based on multi-view structural priors, comprising the following steps:

[0007] 1) A multi-view dataset of hair growth direction is created using publicly available 3D digital human hairstyle data. This multi-view dataset includes multi-view color-rendered images, depth maps, coarse hair direction maps, and fine hair direction maps. The color-rendered images are images rendered from the 3D digital human hairstyle data from multiple perspectives. The depth map is an image representing the distance between the hair and the camera. The coarse hair direction map is a hair direction map obtained through a Gabor filter. The fine hair direction map is a hair direction map obtained by mapping the hair strands of the 3D digital human hairstyle data.

[0008] 2) For the multi-view dataset, a viewpoint is randomly selected as the target viewpoint for prediction each time, and the eight nearest neighboring viewpoints are selected. The data corresponding to the selected viewpoints and the image-level loss function composed of L1 distance loss and SSIM similarity loss are used to train a Dinov2-based hair direction prediction network. This hair direction prediction network is used to predict the fine hair direction map in the target viewpoint. Its execution includes: ① performing Nerf encoding on the camera parameters of the target viewpoint and neighboring viewpoints to extract camera features; ② using the Dinov2 network to extract features from the selected multi-view color rendering images, and concatenating the extracted image features with the above camera features; ③ fusing the concatenated information through a cross-attention mechanism to generate cross-view image features; ④ inputting the depth map and coarse hair direction map of the target viewpoint into two CNN networks for feature extraction to obtain depth features and coarse hair direction features; ⑤ concatenating the cross-view image features, depth features, and coarse hair direction features, and then decoding them through the U-Net decoder network to obtain the fine hair direction map predicted by the target viewpoint.

[0009] 3) Input the test data into the trained hair direction prediction network to obtain the predicted fine hair direction map, which is used as a multi-view structure prior. The multi-view structure prior is injected into the hidden code of the StrandDecoder network, and then the hidden code injected with the multi-view structure prior is decoded into three-dimensional hair strands. The growth direction of the obtained three-dimensional hair strands is mapped onto a two-dimensional plane, and the cosine direction loss is calculated with the multi-view structure prior to obtain a digital human three-dimensional hairstyle that conforms to the direction guidance of the multi-view structure prior.

[0010] Furthermore, in step 1), the creation of the multi-view dataset includes building a virtual rendering scene and annotating the hair growth direction. Specifically, this involves placing the 3D digital human head model and hair model in a unified coordinate system, uniformly setting multiple virtual cameras with different viewpoints around the head, and recording the camera information for each viewpoint, including camera intrinsic and extrinsic parameters. The camera information is recorded as follows: ; Rendering yields color images at each viewpoint and depth map The pixel values ​​in the depth map represent the distance from the corresponding hair point to the camera's optical center; Gabor filtering is applied to the color rendered image, and a coarse hair direction map is generated based on the principal direction of the filter response. ;

[0011] Based on the geometric information of the hair model, for each hair strand in the 3D digital human hairstyle data The orderly vertex from the root to the tip of the hair , Indicates hair strands The number of nodes, Indicates hair strands The 1 node Indicates hair strands From the first node To the last node An ordered set of vertices is projected onto a pixel plane to obtain two-dimensional vertices. Furthermore, the two-dimensional growth direction is obtained by the difference between adjacent projection points, maintaining the directional nature from hair root to hair tip:

[0012] ;

[0013] In the formula, Indicates hair strands The The growth direction of each node in the two-dimensional plane Indicates hair strands The The coordinates of each node on a two-dimensional plane Indicates hair strands The The coordinates of each node on a two-dimensional plane Indicates L2 normal form;

[0014] Will Convert to continuous angle representation :

[0015] ;

[0016] In the formula, , They represent hair strands. The The x and y coordinates of the growth direction of each node on the two-dimensional plane;

[0017] Representing continuous angles Write the corresponding pixels to form a fine hair direction map When multiple hair segments are projected onto the same pixel, the foreground segment is selected based on the depth visibility and assigned a value, while non-hair areas are left blank with a mask.

[0018] Furthermore, in step 2), the selection is based on the target perspective. The eight nearest neighboring viewpoints are determined as follows: the camera pose distance between viewpoints is calculated based on the camera extrinsic parameters of each viewpoint, and the eight viewpoints with the closest relative pose distance to the target viewpoint are determined as neighboring viewpoints.

[0019] The selected camera information is Nerf encoded to obtain camera features. The color rendering image of the selected camera viewpoint is used to extract features using the Dinov2 network, and then the features are concatenated with the camera features.

[0020] The cross-attention mechanism is as follows: using the concatenated features from the target's perspective as the query vector. The splicing features of adjacent viewpoints are used as key vectors. Sum value vector Calculate the cross-view attention weights and sum the weighted features from adjacent views to obtain the cross-view image features. :

[0021] ;

[0022] In the formula, , , These represent the query vector, key vector, and value vector, respectively. Indicates cross-attention mechanism, Indicates the target perspective Cross-view image features, This represents the transpose function. Represents the normalization function;

[0023] The depth features are derived from the depth map of the target viewpoint. Input a CNN network The extracted coarse hair directional features are derived from the coarse hair directional map from the target viewpoint. Input another CNN network Extracted;

[0024] Image features, depth features, and coarse hair direction features from across different viewpoints are concatenated along the channel dimension:

[0025] ;

[0026] In the formula, Indicates the characteristics of splicing. Indicates the target perspective depth features, Indicates the target perspective The directional characteristics of coarse hair, Indicates the concatenation function;

[0027] Features of splicing The input U-Net decoder network decodes and outputs a detailed hair orientation map predicted from the target's viewpoint. ;

[0028] Calculated and predicted fine hair orientation map And the fine hair orientation map in the multi-view dataset in step 1). Image-level loss function Image-level loss function L1 distance loss The L1 distance loss, weighted by the SSIM similarity loss, is expressed as:

[0029] ;

[0030] In the formula, , These represent the height and width of the hair direction diagram. The x and y coordinates of the pixel Representing a detailed hair direction diagram In pixels The direction of hair growth Represents a detailed hair orientation map for prediction. In pixels Predicted direction of hair growth;

[0031] Image-level loss function Represented as:

[0032] ;

[0033] In the formula, , These are the weighting coefficients. This represents the SSIM similarity loss.

[0034] Furthermore, in step 3), the test data is input into the trained hair orientation prediction network, which outputs a detailed predicted hair orientation map. As a multi-view structure prior, the multi-view structure prior is encoded into a prior feature vector. and with the StrandDecoder network implicit coding After fusion, the injected implicit code is obtained. Specifically: First, Transformed to the form by linear mapping Adding them together along the same dimension satisfies ,in Represents a linear mapping function; injects implicit codes after a priori understanding of the multi-view structure. Input StrandDecoder network Output a three-dimensional hair strand set This is used to characterize the three-dimensional geometric shape of digital human hairstyles. The StrandDecoder network is a pre-trained hair generation model based on a large amount of three-dimensional hairstyle data, capable of generating hair strands from hidden codes. Decode the corresponding hairstyle from the middle, and implicitly encode it. This is the implicit representation of the corresponding three-dimensional hairstyle;

[0035] The tangential direction vector at any point on a 3D hair strand is transformed using camera coordinates from the target's viewpoint and projected onto the pixel plane, then normalized to obtain a 2D direction vector. ;

[0036] The cosine direction loss Including pixel-level orientation consistency loss With multi-scale patch-level orientation consistency loss Pixel-level directional consistency loss Represented as:

[0037] ;

[0038] In the formula, The multi-view structure prior obtained in step 2) is in pixels The two-dimensional direction vector at that location, After the decoded 3D hair strands are mapped onto a 2D plane, they are then pixel-wise. The two-dimensional direction vector at that location;

[0039] Multi-scale patch-level orientation consistency loss Defined as: the two-dimensional direction vector and multi-view structural priors after mapping three-dimensional hair strands onto a two-dimensional plane. Divided into a multi-scale grid set, at each scale The image is divided into the following categories: Non-overlapping grids ,in Indicates belonging to the scale set The scale, Indicates according to scale The number of grids divided, Representing scale The Each grid cell is used to calculate its two-dimensional direction vector. and multi-view structure prior Average unit direction within the grid and Calculate the multi-scale patch-level directional consistency loss. as follows:

[0040] ;

[0041] In the formula, For each scale Weighting coefficients;

[0042] The final cosine direction loss is:

[0043] ;

[0044] In the formula, Represents loss in the cosine direction. These are the weighting coefficients;

[0045] By training the StrandDecoder network by minimizing the cosine direction loss, the 3D hair strands are precisely aligned at the pixel level while maintaining consistency between local clustering and global flow direction, thereby obtaining a digital human 3D hairstyle that conforms to the direction guidance of the multi-view structure prior.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] 1. The data obtained by this invention provides a centralized and detailed hair orientation map, which can achieve a pixel-level two-dimensional representation of hair orientation. Compared with existing methods that rely on filter calculations, this method is more detailed and reliable.

[0048] 2. This invention uses a neural network to fuse various geometric structural information, including color-rendered images, depth maps, and coarse direction maps obtained from Gabor filters. This allows for better utilization of structural information and provides more accurate and diverse information features for complex hair direction prediction.

[0049] 3. The neural network training of this invention uses an image-level loss function, which further improves the accuracy of fine hair direction map prediction.

[0050] 4. This invention provides strong prior support for three-dimensional hair reconstruction by injecting the predicted fine hair orientation map into the hidden code, thereby further improving the quality of three-dimensional hair reconstruction.

[0051] 5. This invention proposes a cosine direction loss, which provides an effective constraint on the hair growth direction and further improves the accuracy of hair 3D reconstruction. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the logical flow of the method of the present invention.

[0053] Figure 2 This is a schematic diagram of the structure of a hair direction prediction network.

[0054] Figure 3 A flowchart for obtaining a digital human 3D hairstyle based on prior directional guidance from a multi-view structure. Detailed Implementation

[0055] The present invention will be further described below with reference to specific embodiments and accompanying drawings.

[0056] like Figures 1 to 3 As shown in the figure, this embodiment discloses a method for 3D reconstruction of digital human hairstyles based on multi-view structural priors, the specific details of which are as follows:

[0057] 1) A multi-view dataset of hair growth direction was created using publicly available 3D digital human hairstyle data. This multi-view dataset includes multi-view color-rendered images, depth maps, coarse hair direction maps, and fine hair direction maps. The color-rendered images are multi-view renderings of the 3D digital human hairstyle data; the depth map is an image representing the distance between the hair and the camera; the coarse hair direction map is a hair direction map obtained through a Gabor filter; and the fine hair direction map is a hair direction map obtained by mapping the hair strands of the 3D digital human hairstyle data. Specifically:

[0058] The creation of the multi-view dataset includes building a virtual rendering scene and annotating the hair growth direction. Specifically, a 3D digital human head model and hair model are placed in a unified coordinate system, and multiple virtual cameras with different perspectives are evenly set around the head. The camera information for each perspective is recorded, including camera intrinsic and extrinsic parameters. The camera information is denoted as... ; Rendering yields color images at each viewpoint and depth map The pixel values ​​in the depth map represent the distance from the corresponding hair point to the camera's optical center; Gabor filtering is applied to the color rendered image, and a coarse hair direction map is generated based on the principal direction of the filter response. ;

[0059] Based on the geometric information of the hair model, for each hair strand in the 3D digital human hairstyle data The orderly vertex from the root to the tip of the hair , Indicates hair strands The number of nodes, Indicates hair strands The 1 node Indicates hair strands From the first node To the last node An ordered set of vertices is projected onto a pixel plane to obtain two-dimensional vertices. Furthermore, the two-dimensional growth direction is obtained by the difference between adjacent projection points, maintaining the directional nature from hair root to hair tip:

[0060] ;

[0061] In the formula, Indicates hair strands The The growth direction of each node in the two-dimensional plane Indicates hair strands The The coordinates of each node on a two-dimensional plane Indicates hair strands The The coordinates of each node on a two-dimensional plane Indicates L2 normal form;

[0062] Will Convert to continuous angle representation :

[0063] ;

[0064] In the formula, , They represent hair strands. The The x and y coordinates of the growth direction of each node on the two-dimensional plane;

[0065] Representing continuous angles Write the corresponding pixels to form a fine hair direction map When multiple hair segments are projected onto the same pixel, the foreground segment is selected based on the depth visibility and assigned a value, while non-hair areas are left blank with a mask.

[0066] 2) For the multi-view dataset, a viewpoint is randomly selected each time as the target viewpoint for prediction, and the eight nearest neighboring viewpoints are selected. The data corresponding to the selected viewpoints and an image-level loss function weighted by L1 distance loss and SSIM similarity loss are used to train a Dinov2-based hair orientation prediction network. This network is used to predict the fine hair orientation map in the target viewpoint, and its specific structure is as follows: Figure 2As shown, the execution includes: ① Performing Nerf encoding on the camera parameters of the target viewpoint and adjacent viewpoints to extract camera features; ② Using the Dinov2 network to extract features from the selected multi-viewpoint color rendering images, and concatenating the extracted image features with the aforementioned camera features; ③ Fusing the concatenated information through a cross-attention mechanism to generate cross-viewpoint image features; ④ Inputting the depth map and coarse hair direction map of the target viewpoint into two CNN networks respectively for feature extraction to obtain depth features and coarse hair direction features; ⑤ Concatenating the cross-viewpoint image features, depth features, and coarse hair direction features, and then decoding them through the U-Net decoder network to obtain the detailed hair direction map predicted by the target viewpoint. Specifically, as follows:

[0067] Screening and Target Perspective The eight nearest neighboring viewpoints are determined as follows: the camera pose distance between viewpoints is calculated based on the camera extrinsic parameters of each viewpoint, and the eight viewpoints with the closest relative pose distance to the target viewpoint are determined as neighboring viewpoints.

[0068] The selected camera information is Nerf encoded to obtain camera features. The color rendering image of the selected camera view is used to extract features using the Dinov2 network, and then the features are concatenated with the camera features.

[0069] The cross-attention mechanism is as follows: using the concatenated features from the target's perspective as the query vector. The splicing features of adjacent viewpoints are used as key vectors. Sum value vector Calculate the cross-view attention weights and sum the weighted features from adjacent views to obtain the cross-view image features. :

[0070] ;

[0071] In the formula, , , These represent the query vector, key vector, and value vector, respectively. Indicates cross-attention mechanism, Indicates the target perspective Cross-view image features, This represents the transpose function. Represents the normalization function;

[0072] The depth features are derived from the depth map of the target viewpoint. Input a CNN network The extracted coarse hair directional features are derived from the coarse hair directional map from the target viewpoint. Input another CNN network The extracted data shows that the CNN network uses 3 convolutional layers with 3×3 kernels, stride 1, padding=1, ReLU activation function, and output dimension 256.

[0073] Image features, depth features, and coarse hair direction features from across different viewpoints are concatenated along the channel dimension:

[0074] ;

[0075] In the formula, Indicates the characteristics of splicing. Indicates the target perspective depth features, Indicates the target perspective The directional characteristics of coarse hair, Indicates the concatenation function;

[0076] Features of splicing The input U-Net decoder network decodes and outputs a detailed hair orientation map predicted from the target's viewpoint. ;

[0077] Calculated and predicted fine hair orientation map And the fine hair orientation map in the multi-view dataset in step 1). Image-level loss function Image-level loss function L1 distance loss The L1 distance loss, weighted by the SSIM similarity loss, is expressed as:

[0078] ;

[0079] In the formula, , These represent the height and width of the hair direction diagram. The x and y coordinates of the pixel Representing a detailed hair direction diagram In pixels The direction of hair growth Represents a detailed hair orientation map for prediction. In pixels Predicted direction of hair growth;

[0080] Image-level loss function Represented as:

[0081] ;

[0082] In the formula, , These are the weighting coefficients. This represents the SSIM similarity loss.

[0083] 3) Input the test data into the trained hair direction prediction network to obtain a detailed predicted hair direction map, which serves as a multi-view structure prior. This multi-view structure prior is then injected into the hidden encoding of the StrandDecoder network. The injected hidden encoding is then decoded into 3D hair strands. The growth direction of the obtained 3D hair strands is mapped onto a 2D plane, and a cosine direction loss is calculated with the multi-view structure prior to obtain a 3D digital human hairstyle that conforms to the direction guidance of the multi-view structure prior. Figure 3 As shown. Specifically:

[0084] The test data is input into the trained hair orientation prediction network, which outputs a detailed predicted hair orientation map. As a multi-view structure prior, the multi-view structure prior is encoded into a prior feature vector. and with the StrandDecoder network implicit coding After fusion, the injected implicit code is obtained. Specifically: First, Transformed to the form by linear mapping Adding them together along the same dimension satisfies ,in Represents a linear mapping function; injects implicit codes after a priori understanding of the multi-view structure. Input StrandDecoder network Output a three-dimensional hair strand set This is used to characterize the three-dimensional geometric shape of digital human hairstyles. The StrandDecoder network is a pre-trained hair generation model based on a large amount of three-dimensional hairstyle data, capable of generating hair strands from hidden codes. Decode the corresponding hairstyle from the middle, and implicitly encode it. This is the implicit representation of the corresponding three-dimensional hairstyle;

[0085] The tangential direction vector at any point on a 3D hair strand is transformed using camera coordinates from the target's viewpoint and projected onto the pixel plane, then normalized to obtain a 2D direction vector. ;

[0086] The cosine direction loss Including pixel-level orientation consistency loss With multi-scale patch-level orientation consistency loss Pixel-level directional consistency loss Represented as:

[0087] ;

[0088] In the formula, The multi-view structure prior obtained in step 2) is in pixels The two-dimensional direction vector at that location, After the decoded 3D hair strands are mapped onto a 2D plane, they are then pixel-wise. The two-dimensional direction vector at that location;

[0089] Multi-scale patch-level orientation consistency loss Defined as: the two-dimensional direction vector and multi-view structural priors after mapping three-dimensional hair strands onto a two-dimensional plane. Divided into a multi-scale grid set, at each scale The image is divided into the following categories: Non-overlapping grids ,in Indicates belonging to the scale set The scale, Indicates according to scale The number of grids divided, Representing scale The Each grid cell is used to calculate its two-dimensional direction vector. and multi-view structure prior Average unit direction within the grid and Calculate the multi-scale patch-level directional consistency loss. as follows:

[0090] ;

[0091] In the formula, For each scale Weighting coefficients;

[0092] The final cosine direction loss is:

[0093] ;

[0094] In the formula, Represents loss in the cosine direction. These are the weighting coefficients;

[0095] By training the StrandDecoder network by minimizing the cosine direction loss, the 3D hair strands are precisely aligned at the pixel level while maintaining consistency between local clustering and global flow direction, thereby obtaining a digital human 3D hairstyle that conforms to the direction guidance of the multi-view structure prior.

[0096] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A digital human hairstyle 3D reconstruction method based on multi-view structural prior, characterized by: Includes the following steps: 1) A multi-view dataset of hair growth direction is created using publicly available 3D digital human hairstyle data. This multi-view dataset includes multi-view color-rendered images, depth maps, coarse hair direction maps, and fine hair direction maps. The color-rendered images are images rendered from the 3D digital human hairstyle data from multiple perspectives. The depth map is an image representing the distance between the hair and the camera. The coarse hair direction map is a hair direction map obtained through a Gabor filter. The fine hair direction map is a hair direction map obtained by mapping the hair strands of the 3D digital human hairstyle data. 2) For the multi-view dataset, a viewpoint is randomly selected as the target viewpoint for prediction each time, and the eight nearest neighboring viewpoints are selected. The data corresponding to the selected viewpoints and the image-level loss function composed of L1 distance loss and SSIM similarity loss are used to train a Dinov2-based hair direction prediction network. This hair direction prediction network is used to predict the fine hair direction map in the target viewpoint. Its execution includes: ① performing Nerf encoding on the camera parameters of the target viewpoint and neighboring viewpoints to extract camera features; ② using the Dinov2 network to extract features from the selected multi-view color rendering images, and concatenating the extracted image features with the above camera features; ③ fusing the concatenated information through a cross-attention mechanism to generate cross-view image features; ④ inputting the depth map and coarse hair direction map of the target viewpoint into two CNN networks for feature extraction to obtain depth features and coarse hair direction features; ⑤ concatenating the cross-view image features, depth features, and coarse hair direction features, and then decoding them through the U-Net decoder network to obtain the fine hair direction map predicted by the target viewpoint. 3) Input the test data into the trained hair direction prediction network to obtain the predicted fine hair direction map, which is used as a multi-view structure prior. The multi-view structure prior is injected into the hidden code of the StrandDecoder network, and then the hidden code injected with the multi-view structure prior is decoded into three-dimensional hair strands. The growth direction of the obtained three-dimensional hair strands is mapped onto a two-dimensional plane, and the cosine direction loss is calculated with the multi-view structure prior to obtain a digital human three-dimensional hairstyle that conforms to the direction guidance of the multi-view structure prior.

2. The digital human hairstyle 3D reconstruction method based on multi-view structural prior as described in claim 1, characterized in that: In step 1), the creation of the multi-view dataset includes building a virtual rendering scene and annotating the hair growth direction. Specifically, the 3D digital human head model and hair model are placed in a unified coordinate system, and multiple virtual cameras with different perspectives are evenly set around the head. The camera information for each perspective is recorded, including camera intrinsic and extrinsic parameters. The camera information is recorded as follows: ; Rendering yields color images at each viewpoint and depth map The pixel values ​​in the depth map represent the distance from the corresponding hair point to the camera's optical center; Gabor filtering is applied to the color rendered image, and a coarse hair direction map is generated based on the principal direction of the filter response. ; Based on the geometric information of the hair model, for each hair strand in the 3D digital human hairstyle data The orderly vertex from the root to the tip of the hair , Indicates hair strands The number of nodes, Indicates hair strands The 1 node Indicates hair strands From the first node To the last node An ordered set of vertices is projected onto a pixel plane to obtain two-dimensional vertices. Furthermore, the two-dimensional growth direction is obtained by the difference between adjacent projection points, maintaining the directional nature from hair root to hair tip: ； In the formula, Indicates hair strands The The growth direction of each node in the two-dimensional plane Indicates hair strands The The coordinates of each node on a two-dimensional plane Indicates hair strands The The coordinates of each node on a two-dimensional plane Indicates L2 normal form; Will Convert to continuous angle representation : ； In the formula, , They represent hair strands. The The x and y coordinates of the growth direction of each node on the two-dimensional plane; Representing continuous angles Write the corresponding pixels to form a fine hair direction map When multiple hair segments are projected onto the same pixel, the foreground segment is selected based on the depth visibility and assigned a value, while non-hair areas are left blank with a mask.

3. The digital human hairstyle 3D reconstruction method based on multi-view structural prior as described in claim 2, characterized in that: In step 2), the selection is based on the target perspective. The eight nearest neighboring viewpoints are determined as follows: the camera pose distance between viewpoints is calculated based on the camera extrinsic parameters of each viewpoint, and the eight viewpoints with the closest relative pose distance to the target viewpoint are determined as neighboring viewpoints. The selected camera information is Nerf encoded to obtain camera features. The color rendering image of the selected camera viewpoint is used to extract features using the Dinov2 network, and then the features are concatenated with the camera features. The cross-attention mechanism is as follows: using the concatenated features from the target's perspective as the query vector. The splicing features of adjacent viewpoints are used as key vectors. Sum value vector Calculate the cross-view attention weights and sum the weighted features from adjacent views to obtain the cross-view image features. : ； In the formula, , , These represent the query vector, key vector, and value vector, respectively. Indicates cross-attention mechanism, Indicates the target perspective Cross-view image features, This represents the transpose function. Represents the normalization function; The depth features are derived from the depth map of the target viewpoint. Input a CNN network The extracted coarse hair directional features are derived from the coarse hair directional map from the target viewpoint. Input another CNN network Extracted; Image features, depth features, and coarse hair direction features from across different viewpoints are concatenated along the channel dimension: ； In the formula, Indicates the characteristics of splicing. Indicates the target perspective depth features, Indicates the target perspective The directional characteristics of coarse hair, Indicates the concatenation function; Features of splicing The input U-Net decoder network decodes and outputs a detailed hair orientation map predicted from the target's viewpoint. ; Calculated and predicted fine hair orientation map And the fine hair orientation map in the multi-view dataset in step 1). Image-level loss function Image-level loss function L1 distance loss The L1 distance loss, weighted by the SSIM similarity loss, is expressed as: ； In the formula, , These represent the height and width of the hair direction diagram. The x and y coordinates of the pixel Representing a detailed hair direction diagram In pixels The direction of hair growth Represents a detailed hair orientation map for prediction. In pixels Predicted direction of hair growth; Image-level loss function Represented as: ； In the formula, , These are the weighting coefficients. This represents the SSIM similarity loss.

4. The digital human hairstyle 3D reconstruction method based on multi-view structural prior as described in claim 3, characterized in that: In step 3), the test data is input into the trained hair orientation prediction network, which outputs a detailed predicted hair orientation map. As a multi-view structure prior, the multi-view structure prior is encoded into a prior feature vector. and with the StrandDecoder network implicit coding After fusion, the injected implicit code is obtained. Specifically: First, Transformed to the form by linear mapping Adding them together along the same dimension satisfies ,in Represents a linear mapping function; injects implicit codes after a priori understanding of the multi-view structure. Input StrandDecoder network Output a three-dimensional hair strand set This is used to characterize the three-dimensional geometric shape of digital human hairstyles. The StrandDecoder network is a pre-trained hair generation model based on a large amount of three-dimensional hairstyle data, capable of generating hair strands from hidden codes. Decode the corresponding hairstyle from the middle, and implicitly encode it. This is the implicit representation of the corresponding three-dimensional hairstyle; The tangential direction vector at any point on a 3D hair strand is transformed using camera coordinates from the target's viewpoint and projected onto the pixel plane, then normalized to obtain a 2D direction vector. ; The cosine direction loss Including pixel-level orientation consistency loss With multi-scale patch-level orientation consistency loss Pixel-level directional consistency loss Represented as: ； In the formula, The multi-view structure prior obtained in step 2) is in pixels The two-dimensional direction vector at that location, After the decoded 3D hair strands are mapped onto a 2D plane, they are then pixel-wise. The two-dimensional direction vector at that location; Multi-scale patch-level orientation consistency loss Defined as: the two-dimensional direction vector and multi-view structural priors after mapping three-dimensional hair strands onto a two-dimensional plane. Divided into a multi-scale grid set, at each scale The image is divided into the following categories: Non-overlapping grids ,in Indicates belonging to the scale set The scale, Indicates according to scale The number of grids divided, Representing scale The Each grid cell is used to calculate its two-dimensional direction vector. and multi-view structure prior Average unit direction within the grid and Calculate the multi-scale patch-level directional consistency loss. as follows: ； In the formula, For each scale Weighting coefficients; The final cosine direction loss is: ； In the formula, Represents loss in the cosine direction. These are the weighting coefficients; By training the StrandDecoder network by minimizing the cosine direction loss, the 3D hair strands are precisely aligned at the pixel level while maintaining consistency between local clustering and global flow, thereby obtaining a digital human 3D hairstyle that conforms to the direction guidance of the multi-view structure prior.

Images