Method for generating a three-dimensional hair model, electronic device and storage medium

Abstract

The application provides a three-dimensional hair model generation method, an electronic device and a storage medium, and relates to the technical field of computer graphics. The method comprises the following steps: obtaining a picture latent vector; the picture latent vector is used for describing the image characteristics of a three-dimensional hair model to be generated; inputting the picture latent vector into a hair generation model to obtain a guide line model; the guide line model comprises a plurality of first guide lines, and each first guide line is composed of a plurality of first three-dimensional coordinates; the hair generation model is obtained by training a preset model, and the preset model is established based on an implicit diffusion model; and a three-dimensional hair model is generated according to the guide line model, so that the three-dimensional hair is digitally expressed by using the guide line model, the details of the hair can be retained, and a high-quality three-dimensional hair model is generated.

Description

Technical Field

[0001] This application relates to the field of computer graphics technology, and more particularly to a method for generating a three-dimensional hair model, an electronic device, and a storage medium. Background Technology

[0002] With the rapid development of computer vision and graphics technology, people have increasingly higher visual requirements for virtual hair modeling. Hair is a very important part of virtual characters and digital figures, and due to its complexity, a lot of time and effort is required to model it.

[0003] Currently, 3D hair models can typically be generated based on principal component analysis (PCA) technology. Specifically, an image of the hair to be generated is input into a parametric model of PCA. By projecting the 3D hair onto a 2D plane and continuously adjusting the parameter weights to achieve alignment, a 3D hair model is obtained.

[0004] However, in the above-mentioned methods, the ability to express details is limited when generating a 3D hair model based on PCA, thus failing to extract the fine parts of the hair, resulting in a low quality of the generated 3D hair model. Summary of the Invention

[0005] This application provides a method, electronic device, and storage medium for generating a three-dimensional hair model, in order to solve the problem that when generating a three-dimensional hair model based on PCA, the ability to express details is limited, thus making it impossible to extract the fine parts of the hair, resulting in a low quality of the generated three-dimensional hair model, thereby achieving the goal of generating a high-quality three-dimensional hair model.

[0006] In a first aspect, this application provides a method for generating a three-dimensional hair model, the method comprising:

[0007] Obtain the latent vectors of the image; the latent vectors of the image are used to describe the image features of the 3D hair model to be generated.

[0008] The latent vectors of the image are input into the hair generation model to obtain the guide line model; the guide line model includes multiple first guide lines, each of which is composed of multiple first three-dimensional coordinates; the hair generation model is obtained by training a preset model, which is based on an implicit diffusion model.

[0009] A three-dimensional hair model is generated based on the guide line model.

[0010] In one possible design, the hair generation model includes: an implicit diffusion model processing module and a hair model decoding module;

[0011] The implicit diffusion model processing module is used to perform noise restoration processing on the latent vectors of the image to obtain the hair latent vectors, and input the hair latent vectors into the hair model decoding module;

[0012] The hair model decoding module is used to reconstruct the hair latent vectors to obtain the guide line model.

[0013] In one possible design, generating a three-dimensional hair model based on the guide line model includes:

[0014] A first region is determined from a description image, which is an image input by the user to describe the three-dimensional hair model to be generated. The first region includes one or more regions in the description image that contain a special shape.

[0015] Based on the first region, a corresponding second region is determined in the guide line model;

[0016] For each second region, a special hair bundle is generated based on the multiple first guide lines within the second region and the Frenet-Serret framework of the Frenet formula;

[0017] The special hair strands corresponding to each second region are input into the Dino V2 model for latent vector extraction to obtain the special latent vectors corresponding to each second region.

[0018] For each second region, the hidden vector corresponding to the second region in the hair hidden vector is replaced with the special hidden vector;

[0019] The replaced hair hidden vectors are then re-inputted into the hair generation model to obtain the three-dimensional hair model.

[0020] In one possible design, the method further includes:

[0021] Multiple sets of sample data are acquired. Each set of sample data includes a sample latent vector, a guide vector, and a standard latent vector. The standard latent vector is obtained by sampling and compressing the sample hair model. The sample latent vector is obtained by adding noise to the standard latent vector. The guide vector is used to guide the preset model to denoise the sample latent vector.

[0022] The preset model is trained based on the multiple sets of sample data until it converges, thus obtaining the hair generation model.

[0023] In one possible design, for each set of sample data, the sample latent vector and the standard latent vector are obtained, including:

[0024] Obtain a sample hair model, wherein the sample hair model includes multiple second guide lines;

[0025] For each second guide line, the second guide line is divided equally according to a first preset ratio to obtain multiple second three-dimensional coordinates;

[0026] The result vector field is sampled and compressed in the scalp region of the sample hair model at a second preset ratio to obtain the standard latent vector. The result vector field is a vector composed of multiple second three-dimensional coordinates corresponding to multiple second guide lines in the sample hair model.

[0027] The standard latent vector is noise-added to obtain the sample latent vector.

[0028] In one possible design, for each set of sample data, the guiding vector is obtained, including:

[0029] The sample hair model is rendered from multiple perspectives to obtain a multi-view rendering image;

[0030] The multi-view rendered image is converted into a multi-view line sketch using a line detector.

[0031] The multi-viewline sketch is encoded into the guiding vector using the Dino V2 model.

[0032] In one possible design, obtaining the image latent vector includes:

[0033] Receive a description image, which is an image input by the user to describe the three-dimensional hair model to be generated;

[0034] The image description is input into the Dino V2 model for latent vector extraction to obtain the latent vector of the image.

[0035] The method provided in the first aspect obtains latent vectors from images, thereby extracting key features from high-dimensional two-dimensional hair images through mathematical transformations and compressing them into low-dimensional latent vectors. This significantly reduces data dimensionality while retaining core information, facilitating subsequent processing and improving the efficiency of generating 3D hair models. The latent vectors are then input into the hair generation model to obtain a guide line model. This allows for further processing of the latent vectors, enhancing the structural features of the hair, such as hair strand boundaries and flow direction, resulting in more structured latent vectors. Based on these more structured latent vectors, the guide line model is obtained, enabling the digital representation of 3D hair while preserving hair details for a more accurate 3D hair model. The 3D hair model can then be generated based on the guide line model, greatly improving the precision of the generated 3D hair model.

[0036] Secondly, this application provides a three-dimensional hair model generation apparatus, comprising: a module for performing the methods of the first aspect and any possible design of the first aspect.

[0037] The beneficial effects of the apparatus provided in the second aspect above can be seen in the beneficial effects brought about by the first aspect and the various possible designs of the first aspect, and will not be repeated here.

[0038] Thirdly, this application provides an electronic device, including: a memory and a first processor; the memory is used to store program instructions; the first processor is used to invoke the program instructions in the memory to cause the electronic device to perform the methods of the first aspect and any possible design of the first aspect.

[0039] Fourthly, this application provides an electronic device, including: a second processor;

[0040] The second processor is used to execute a computer-executable program or instructions in memory, causing the electronic device to perform the first aspect and any possible design of the first aspect.

[0041] Fifthly, this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being processed by a processor to cause an electronic device to implement the first aspect and any possible design of the first aspect when executed.

[0042] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating a method for generating a three-dimensional hair model according to an embodiment of this application.

[0045] Figure 2 This is a schematic diagram of the structure of a hair generation model provided in an embodiment of this application.

[0046] Figure 3 This is a schematic flowchart of a method for generating a three-dimensional hair model according to an embodiment of this application.

[0047] Figure 4 This is a schematic flowchart of a method for training a hair generation model according to an embodiment of this application.

[0048] Figure 5 This is a schematic flowchart illustrating a method for obtaining sample latent vectors and standard latent vectors according to an embodiment of this application.

[0049] Figure 6 This is a schematic flowchart of a method for obtaining a guiding vector according to an embodiment of this application.

[0050] Figure 7 This is a schematic flowchart illustrating a method for obtaining latent vectors of an image, provided in one embodiment of this application.

[0051] Figure 8 This is a schematic diagram of a three-dimensional hair model generation device provided in an embodiment of this application.

[0052] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0053] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c alone can mean: a alone, b alone, c alone, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0055] The terms “center,” “longitudinal,” “lateral,” “up,” “down,” “left,” “right,” “front,” and “rear,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0056] The terms "connected" and "connected" should be interpreted broadly. For example, in circuit structures, "connected" or "connected" can refer not only to physical connections but also to electrical or signal connections. This could be a direct connection (physical connection) or an indirect connection via at least one intermediate component, as long as the circuit is connected. It could also refer to the internal connection between two components. Similarly, a signal connection can refer to a connection via a circuit or a medium, such as radio waves. Those skilled in the art will understand the specific meaning of these terms in this application based on the specific circumstances.

[0057] For example, this application provides a method, apparatus, electronic device, computer-readable storage medium, computer program product, and chip for generating a three-dimensional hair model. Based on an image describing the image features of the three-dimensional hair model to be generated, an image latent vector is obtained. The image latent vector is input into a hair generation model to generate a guide line model. The guide line model is used to digitally represent three-dimensional hair, which can preserve the details of the hair and generate a high-quality three-dimensional hair model.

[0058] The method for generating the 3D hair model can be executed by an electronic device, or by a 3D hair model generation device within an electronic device. The 3D hair model generation device can be implemented through a combination of software and / or hardware. For example, the 3D hair model generation device can be an application (APP), a webpage, or a public account. Alternatively, the 3D hair model generation device can be a computer, tablet, or mobile phone.

[0059] The electronic device can be a server, desktop computer, mobile phone, tablet computer, laptop computer, wearable device, in-vehicle device, or augmented reality (AR) / virtual reality (VR) device, etc. For simplicity, this application embodiment uses an apparatus for generating a three-dimensional hair model (hereinafter referred to as the generating apparatus) as an example.

[0060] Below, in conjunction with Figures 1 to 7 The method for generating a three-dimensional hair model provided in the embodiments of this application will be described.

[0061] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for generating a three-dimensional hair model according to an embodiment of this application. Figure 1 As shown, the method includes:

[0062] S101, The generating device acquires the latent vector of the image.

[0063] Among them, the latent vectors of the image are used to describe the image features of the 3D hair model to be generated.

[0064] The generator can extract latent vectors from a descriptive image.

[0065] The description image includes a 2D image of the hair for which the 3D hair model is to be generated. The description image can be an image uploaded by the user. For example, the description image could be: an image of long, curly red hair.

[0066] Two-dimensional hair images have high data dimensionality and redundancy, making them difficult to process. After receiving the description image uploaded by the user, the generation device can extract latent vectors from the description image to obtain the image's latent vectors. This allows the high-dimensional two-dimensional hair image to be transformed mathematically to extract key features and compressed into low-dimensional latent vectors, significantly reducing data dimensionality while retaining core information, thus facilitating subsequent processing.

[0067] Furthermore, latent vectors can transform low-level visual features, such as hair color and texture, into high-level semantic information, improving the accuracy of image recognition and thus helping to improve the accuracy of generating 3D hair models.

[0068] As a feasible implementation, the generation device can use the Dino V2 model to capture semantic information and key features in the descriptive image, obtaining image latent vectors. These image latent vectors can describe the visual features of the 3D hair model to be generated. For example, if the descriptive image includes an image of long, curly red hair, then the image latent vectors can indicate at least the following features: red color, long hair, curly hair, and the distribution of hair strands in the 2D hair image.

[0069] Based on this, the generating device obtains the latent vector of the image.

[0070] S102. The generating device inputs the latent vector of the image into the hair generation model to obtain the guide line model.

[0071] After obtaining the latent vectors of the image, the generation device can further process the latent vectors to enhance the structural features of the hair, such as hair strand boundaries and flow direction, thereby obtaining more structured latent vectors. Based on the more structured latent vectors, the guide line model is obtained.

[0072] Specifically, the generation device can input the latent vectors of the image into the hair generation model. First, the hair generation model can iteratively denoise the latent vectors in the latent space, converting the blurred hair strand outlines into clear latent vectors. Based on the converted clear latent vectors, the hair generation model can further perform uniform sampling within the scalp area according to a certain proportion. After dividing the sampled latent vectors into multiple dividing points according to a specific ratio, the first three-dimensional coordinates of each dividing point are obtained. Connecting the first three-dimensional coordinates of each dividing point yields a first guide line. Based on this, the above processing is performed on all image latent vectors, and the generation device can obtain a guide line model.

[0073] The guide line model includes multiple first guide lines, each composed of multiple first three-dimensional coordinates. Each first guide line is the center line of the hair bundle, allowing the generation device to obtain a three-dimensional hair model.

[0074] The hair generation model is trained from a pre-defined model, which is based on an implicit diffusion model. The implicit diffusion model is a generative model based on latent space that improves the efficiency and quality of image generation and reconstruction through non-Markovian processes and low-dimensional feature compression mechanisms.

[0075] Based on this, the generating device obtains a guide line model, and then uses the guide line model to digitally represent three-dimensional hair, preserving the details of the hair, so as to obtain a high-precision three-dimensional hair model.

[0076] S103. The generating device generates a three-dimensional hair model based on the guide line model.

[0077] After obtaining the guide line model, the generation device can generate a three-dimensional hair model in a variety of ways.

[0078] As a feasible approach, the generation device can directly define the guide line model as a 3D hair model. The guide line model already possesses the structural features of a 3D hair model. In scenarios with lower precision requirements, the generation device can output the guide line model as the final 3D hair model, thereby efficiently and quickly generating the 3D hair model required by the user.

[0079] As another feasible implementation, the generation device can perform operations such as hair bundle replacement and enhancement based on the guide line model. The generation device can replace the hair bundle structure in the guide line model with special hair bundles, such as braids, curls, or hair bundles with other colors. Furthermore, the generation device can modify the width, thickness, and other attributes of the hair bundles based on the guide line model to further enhance its expressive effect, thereby obtaining a higher quality 3D hair model.

[0080] In this embodiment, the generation device acquires latent vectors from an image, thereby extracting key features from a high-dimensional two-dimensional hair image through mathematical transformation and compressing it into low-dimensional latent vectors. This significantly reduces data dimensionality while retaining core information, facilitating subsequent processing and improving the efficiency of generating a three-dimensional hair model. The generation device inputs the latent vectors into the hair generation model to obtain a guide line model, which can then be further processed to enhance the structural features of the hair, such as hair strand boundaries and flow direction, resulting in more structured latent vectors. Based on these more structured latent vectors, a guide line model is obtained, enabling the digital representation of three-dimensional hair while preserving hair details, thus facilitating the generation of a high-precision three-dimensional hair model. The generation device can then generate a three-dimensional hair model based on the guide line model, greatly improving the precision of the generated three-dimensional hair model.

[0081] Based on the above exemplary description, the structure of the hair generation model can be as follows: Figure 2 As shown. Please refer to [the original text]. Figure 2 , Figure 2 This is a schematic diagram of the structure of a hair generation model provided in one embodiment of this application. Figure 2 As shown, the hair generation model includes: an implicit diffusion model processing module 10 and a hair model decoding module 20.

[0082] The implicit diffusion model processing module 10 is used to perform noise restoration processing on the latent vectors of the image to obtain the hair latent vectors, and then inputs the hair latent vectors into the hair model decoding module 20.

[0083] Specifically, the implicit diffusion model processing module 10 can gradually recover the implicit vector used to describe the hair image from the noise based on the image's implicit vector, thus obtaining the hair's implicit vector.

[0084] In some examples, the implicit diffusion model processing module 10 can employ a Diffusion Transformer (DiT) model. The DiT model can progressively add noise to the data by simulating a diffusion process, and then learn to reverse this process to construct the desired data sample from the noise. The generation device can utilize the DiT model to efficiently generate hair latent vectors from the image latent vectors, achieving precise spatial localization and fine-grained control.

[0085] Among them, the hair model decoding module 20 is used to reconstruct the hair latent vectors to obtain the guide line model.

[0086] Specifically, the hair model decoding module 20 can uniformly sample the scalp area of ​​a standard human head template at a certain ratio. For example, if 10,000 hairs need to be generated, the sampling ratio can be 100×100. The generation device divides the sampled latent vectors into multiple boundary points according to a specific equal division, obtains the first three-dimensional coordinates of each boundary point, and connects the first three-dimensional coordinates of each boundary point to obtain a first guide line, thus obtaining a guide line model. Based on this, the hair model decoding module 20 can reconstruct the hair latent vectors into a guide line model, thereby realizing the modeling of a three-dimensional hair model.

[0087] In some examples, the hair model decoding module 20 may employ a neural network decoder.

[0088] Based on the above exemplary description, the generating apparatus can be configured according to the following: Figure 3 The method shown generates a 3D hair model.

[0089] Please see Figure 3 , Figure 3 This is a schematic flowchart illustrating a method for generating a three-dimensional hair model according to an embodiment of this application. Figure 3 As shown, the method includes:

[0090] S201, The generating device determines a first region from the description picture.

[0091] The description image is an image input by the user to describe the 3D hair model to be generated. The description image includes a 2D image of the hair from the 3D hair model. For example, the description image could be an image of long, red, curly hair.

[0092] The first area includes one or more areas in the image that contain a special design. Special designs include, for example, braids, curls, or hair strands in other colors.

[0093] As a feasible implementation, the generation device can combine an open set object detector (GroundingDINO) and a segmentation anything model (SAM) to locate special structures in the description image, thereby determining the first region. This method can accurately locate special results in the image, thus helping to improve the accuracy of generating 3D hair models.

[0094] S202. The generating device determines the corresponding second region in the guide line model based on the first region.

[0095] After determining the first region in the description image, the generating device determines the corresponding second region in the guide line model based on the first region, that is, in the guide line model corresponding to the first region, the position of the special structure in the first region in the guide line model is obtained.

[0096] When the first region includes one or more regions with special shapes in the description image, the second region includes one or more regions in the guide line model.

[0097] S203. The generating device generates special hair strands for each second region based on multiple first guide lines within the second region and the Frenet-Serret formula framework.

[0098] For each second region, the generating device can calculate an average curve as a guide hair bundle based on multiple first guide lines within the second region. Based on the guide hair bundle, a pattern or color corresponding to a special shape is introduced along the guide hair bundle through the Frenet-Serret framework, thereby generating a special hair bundle.

[0099] For example, if the special style is a braid, the generating device is based on a guide hair bundle and introduces a spiral pattern along the guide hair bundle through the Frenet-Serret frame, thereby generating a special hair bundle in the style of a braid.

[0100] Furthermore, the shape of special hair strands can be adjusted by modifying the parameters of width, thickness, and cross-sectional oscillation to meet the user's needs.

[0101] Furthermore, the generating device can also apply Laplace smoothing to specific hair strands to reduce high-frequency noise and retain key features, thereby enhancing the visual smoothness and geometric fidelity of specific hair strands.

[0102] For example, when the hair strand is braided, Laplace smoothing can preserve key features such as the braid's intersections and twists.

[0103] S204. The generating device inputs the special hair bundle corresponding to each second region into the Dino V2 model for latent vector extraction, and obtains the special latent vector corresponding to each second region.

[0104] Based on this, the generating device obtains the latent vector corresponding to the special hair bundle, so as to replace the hair latent vector at the corresponding position of the special hair bundle in the guide line model with the special latent vector, so as to realize the replacement of the hair bundle.

[0105] S205. For each second region, the generating device replaces the hidden vector corresponding to the second region in the hair hidden vector with a special hidden vector.

[0106] S206. The generation device will re-input the replaced hair latent vector into the hair generation model to obtain a three-dimensional hair model.

[0107] The generation device re-inputs the replaced hair latent vectors into the hair generation model for processing, thereby obtaining a 3D hair model that allows specific hair strands to fill the corresponding positions in the guide line model. After further processing by the hair generation model, during the denoising process, the latent vectors corresponding to the specific hair strands are replaced with their corresponding latent vectors, ensuring that the geometric structure of the specific hair strands is preserved and that they seamlessly blend with other surrounding hair strands, further improving the quality and accuracy of the 3D hair model.

[0108] Based on the above exemplary description, the training method for the hair generation model is described below. The generation device can be based on the following... Figure 4 The hair generation model is trained in the manner shown.

[0109] Please see Figure 4 , Figure 4 This is a schematic flowchart illustrating a method for training a hair generation model according to an embodiment of this application. Figure 4 As shown, the method includes:

[0110] S301, The generating device acquires multiple sets of sample data.

[0111] Each set of sample data includes a sample latent vector, a guiding vector, and a standard latent vector.

[0112] The standard latent vector is obtained by sampling and compressing the sample hair model, while the sample latent vector is obtained by adding noise to the standard latent vector.

[0113] Specifically, the generation device can obtain sample hair models from a preset database, sample and compress each sample hair model to obtain a standard latent vector, and add noise to the standard latent vector to obtain a sample latent vector.

[0114] The guiding vector is used to guide the preset model to denoise the latent vectors of the samples.

[0115] In each set of sample data, the latent vector, guiding vector, and standard latent vector should be generated based on the same sample hair model to ensure accurate training of the preset model.

[0116] S302. The generation device trains the preset model based on multiple sets of sample data until the preset model converges, thus obtaining the hair generation model.

[0117] The generation device inputs the sample latent vector and the guiding vector into the preset model. The guiding vector guides the preset model to denoise the sample latent vector, resulting in a denoised sample latent vector. The denoised sample latent vector is then decoded by the preset model, outputting the guiding line model corresponding to the sample latent vector. This model is then compared with the true value (i.e., the standard latent vector). Based on the comparison results, the parameters of the preset model are adjusted, and the preset model is trained until it converges, thus obtaining the hair generation model.

[0118] Based on the above exemplary description, the method for the generation device to obtain sample latent vectors and standard latent vectors is described below. For each set of sample data, the generation device can, according to the following... Figure 5 The method shown is used to obtain the sample latent vector and the standard latent vector.

[0119] Please see Figure 5 , Figure 5 This is a schematic flowchart illustrating a method for obtaining sample latent vectors and standard latent vectors according to an embodiment of this application. Figure 5 As shown, the method includes:

[0120] S401, The generator acquires a sample hair model.

[0121] The sample hair model includes multiple second guide lines. The generation device can obtain the sample hair model from a preset database.

[0122] S402. For each second guide line, the generating device divides the second guide line into equal parts according to a first preset ratio to obtain multiple second three-dimensional coordinates.

[0123] Specifically, the generating device divides each second guide line into equal parts according to a first preset ratio, obtaining multiple dividing points corresponding to each second guide line. The generating device records the second three-dimensional coordinates of each dividing point. By performing the above processing on each second guide line, multiple second three-dimensional coordinates corresponding to each second guide line can be obtained.

[0124] S403. The generating device samples and compresses the resulting vector field in the scalp region of the sample hair model at a second preset ratio to obtain a standard latent vector.

[0125] After obtaining multiple second three-dimensional coordinates corresponding to each second guide line, the generating device generates a result vector field.

[0126] The resulting vector field is a vector composed of multiple second three-dimensional coordinates corresponding to the multiple second guide lines in the sample hair model. The resulting vector field contains all the second three-dimensional coordinates.

[0127] The generation device samples the scalp region according to a second preset ratio based on the result vector field, and inputs the sampled results into the encoder, which compresses the sampled results to obtain the standard latent vector.

[0128] S404. The generator adds noise to the standard latent vector to obtain the sample latent vector.

[0129] The generation device progressively adds noise to the standard latent vector to obtain sample latent vectors. Based on this, the generation device can train a preset model using the sample latent vectors and the standard latent vectors to obtain a hair generation model.

[0130] Based on the above exemplary description, the method for the generation device to obtain the guiding vector is described below. For each set of sample data, the generation device can obtain the guiding vector according to the following... Figure 6 The guiding vector is obtained in the manner shown.

[0131] Please see Figure 6 , Figure 6 This is a schematic flowchart illustrating a method for obtaining a guiding vector according to an embodiment of this application. Figure 6 As shown, the method includes:

[0132] S501, the generator produces a multi-view rendering sample hair model to obtain a multi-view rendering image.

[0133] The generation device can render the sample hair model from multiple perspectives using the rendering engine to obtain the multi-view rendering image corresponding to the sample hair model.

[0134] The generation device can use rendering engines such as Arnold renderer or Three.js engine to render the sample hair model from multiple perspectives.

[0135] S502, the generating device converts the multi-view rendering into a multi-view line draft using a line detector.

[0136] The line detector can be a Canny line detector or a HED line detector, etc. The generation device can use the line detector to convert the multi-view rendering image into a multi-view line sketch image, thereby highlighting the structural information in the multi-view rendering image and removing other useless details, so as to improve the training effect during training.

[0137] S503, the generating device uses the Dino V2 model to encode the multi-view line sketch into guiding vectors.

[0138] Based on this, the generation device realizes the conversion from multi-view rendering images to guiding vectors, and extracts key features from the multi-view rendering images. In order to use the guiding vectors to guide the pre-set model to denoise the sample latent vectors in the subsequent training process of the pre-set model, thereby improving the training speed and training effect.

[0139] Based on the above exemplary description, the generating apparatus can be configured according to the following: Figure 7 The method shown is used to obtain the latent vectors of the image.

[0140] Please see Figure 7 , Figure 7 This is a schematic flowchart illustrating a method for obtaining latent vectors of an image according to an embodiment of this application. Figure 7 As shown, the method includes:

[0141] S601, Receive the description image.

[0142] The description image is an image input by the user to describe the 3D hair model to be generated.

[0143] S602. Input the description image into the Dino V2 model to extract the latent vectors of the image.

[0144] The user-input description image is a two-dimensional hair image. Two-dimensional hair images have high data dimensionality and high redundancy, making them difficult to process. After receiving the user-uploaded description image, the generation device can use the Dino V2 model to extract latent vectors from the image, obtaining latent vectors. This allows the high-dimensional two-dimensional hair image to be mathematically transformed to extract key features, compressing it into low-dimensional latent vectors, significantly reducing data dimensionality while retaining core information, facilitating subsequent processing.

[0145] Furthermore, latent vectors can transform low-level visual features, such as hair color and texture, into high-level semantic information, improving the accuracy of image recognition and thus helping to improve the accuracy of generating 3D hair models.

[0146] Figure 8 This is a schematic diagram of a device for generating a three-dimensional hair model according to an embodiment of this application. Figure 8 As shown, the device includes: an acquisition module 101, a first generation module 102, and a second generation module 103.

[0147] The acquisition module 101 is used to acquire the latent vectors of the image; the latent vectors of the image are used to describe the image features of the 3D hair model to be generated.

[0148] The first generation module 102 is used to input the latent vector of the image into the hair generation model to obtain the guide line model; the guide line model includes multiple first guide lines, each of which is composed of multiple first three-dimensional coordinates; the hair generation model is obtained by training a preset model, which is based on an implicit diffusion model.

[0149] The second generation module 103 is used to generate a three-dimensional hair model based on the guide line model.

[0150] It should be noted that the three-dimensional hair model generation device of this application embodiment can be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, so it will not be repeated here.

[0151] In some examples, the hair generation model includes: an implicit diffusion model processing module and a hair model decoding module;

[0152] The implicit diffusion model processing module is used to perform noise restoration processing on the latent vectors of the image to obtain the hair latent vectors, and then input the hair latent vectors into the hair model decoding module;

[0153] The hair model decoding module is used to reconstruct the hair latent vectors to obtain the guide line model.

[0154] In some examples, the second generation module 103 is specifically used for:

[0155] The first region is determined from the description image, which is an image input by the user to describe the 3D hair model to be generated. The first region includes one or more regions in the description image that contain a special shape.

[0156] Based on the first region, determine the corresponding second region in the guide line model;

[0157] For each second region, a special hair bundle is generated based on the multiple first guide lines within the second region and the Frenet-Serret framework of the Frenet formula;

[0158] The special hair strands corresponding to each second region are input into the Dino V2 model for latent vector extraction to obtain the special latent vectors corresponding to each second region.

[0159] For each second region, replace the hidden vector corresponding to the second region in the hair hidden vector with a special hidden vector;

[0160] The replaced hair latent vectors are then re-inputted into the hair generation model to obtain a 3D hair model.

[0161] In some examples, the device also includes a training module;

[0162] The training module is used to acquire multiple sets of sample data. Each set of sample data includes sample latent vectors, guide vectors, and standard latent vectors. The standard latent vectors are obtained by sampling and compressing the sample hair model. The sample latent vectors are obtained by adding noise to the standard latent vectors. The guide vectors are used to guide the preset model to denoise the sample latent vectors.

[0163] Based on multiple sets of sample data, the preset model is trained until it converges, thus obtaining the hair generation model.

[0164] In some examples, for each set of sample data, a training module is specifically used to obtain a sample hair model, which includes multiple second guide lines.

[0165] For each second guide line, the second guide line is divided equally according to a first preset ratio to obtain multiple second three-dimensional coordinates;

[0166] The resulting vector field is sampled and compressed in the scalp region of the sample hair model at a second preset ratio to obtain a standard latent vector. The resulting vector field is a vector composed of multiple second three-dimensional coordinates corresponding to multiple second guide lines in the sample hair model.

[0167] Noise is added to the standard latent vector to obtain the sample latent vector.

[0168] In some examples, for each set of sample data, a training module is specifically used to render sample hair models from multiple perspectives to obtain multi-view rendering images.

[0169] The multi-view rendered image is converted into a multi-view line sketch using a line detector.

[0170] The multi-viewline line sketches are encoded into guiding vectors using the Dino V2 model.

[0171] In some examples, module 101 is specifically used to receive a description image, which is an image input by the user to describe the 3D hair model to be generated;

[0172] The image description is input into the Dino V2 model for latent vector extraction, resulting in the image latent vector.

[0173] Figure 9 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Figure 9 As shown, the electronic device may include a first processor 201 and a memory 202. The memory 202 stores a computer program. When the first processor 201 executes the computer program, it implements the embodiments of this application. Figures 1 to 7 The method for generating a 3D hair model is shown.

[0174] Figure 10This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Figure 10 As shown, the electronic device may include a second processor 301. When the second processor 301 executes a computer-executable program or instructions stored in the memory to execute a computer program, it implements the embodiments of this application. Figures 1 to 7 The method for generating a 3D hair model is shown.

[0175] Another embodiment of this application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the embodiments of this application. Figures 1 to 7 The method for generating a 3D hair model is shown.

[0176] Another embodiment of this application provides a computer program product, including: execution instructions stored in a readable storage medium, at least one processor of an electronic device can read the execution instructions from the readable storage medium, and the at least one processor executes the execution instructions to cause the electronic device to implement the embodiments of this application. Figures 1 to 7 The method for generating a 3D hair model is shown.

[0177] Another embodiment of this application provides a chip that is connected to a memory, or a chip that integrates a memory. When a software program stored in the memory is executed, it implements the embodiments of this application. Figures 1 to 7 The method for generating a 3D hair model is shown.

[0178] In the above embodiments, all or part of the functionality can be implemented by software, hardware, or a combination of software and hardware. When implemented using software, it can be implemented wholly or partially in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0181] Those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0182] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for generating a three-dimensional hair model, characterized in that, The method includes: Obtain the latent vector of the image; the latent vector of the image is a low-dimensional vector obtained by extracting the latent vector of the description image, the description image is an image input by the user to describe the three-dimensional hair model to be generated, the description image includes a two-dimensional hair image of the three-dimensional hair model to be generated, the latent vector of the image is used to describe the image features of the three-dimensional hair model to be generated, the image features include hair color, hair length, hair strand shape and hair bundle distribution; The latent vectors of the image are input into a hair generation model to obtain a guide line model. The guide line model includes multiple first guide lines, each composed of multiple first three-dimensional coordinates. The hair generation model is obtained by training a preset model, which is based on an implicit diffusion model. The hair generation model includes an implicit diffusion model processing module and a hair model decoding module. The implicit diffusion model processing module performs noise restoration processing on the latent vectors of the image to obtain hair latent vectors, and inputs the hair latent vectors into the hair model decoding module. The hair model decoding module performs reconstruction processing on the hair latent vectors to obtain the guide line model. The reconstruction processing of the hair latent vectors includes: uniformly sampling the hair latent vectors in the scalp area of ​​a standard human head template according to a preset ratio to obtain multiple sampled latent vectors; for each sampled latent vector, dividing the sampled latent vector into multiple boundary points according to a specific equal division, and obtaining the first three-dimensional coordinates of each boundary point; connecting the first three-dimensional coordinates of the boundary points of each sampled latent vector to obtain multiple first guide lines. Based on the guide line model, a three-dimensional hair model is generated; The step of generating a three-dimensional hair model based on the guide line model includes: A first region is determined from the description image, the first region including one or more regions in the description image containing a special shape; Based on the first region, a corresponding second region is determined in the guide line model; For each second region, a special hair bundle is generated based on the multiple first guide lines within the second region and the Frenet-Serret framework of the Frenet formula; The special hair strands corresponding to each second region are input into the Dino V2 model for latent vector extraction to obtain the special latent vectors corresponding to each second region. For each second region, the hidden vector corresponding to the second region in the hair hidden vector is replaced with the special hidden vector; The replaced hair hidden vectors are then re-inputted into the hair generation model to obtain the three-dimensional hair model.

2. The method according to claim 1, characterized in that, The method further includes: Multiple sets of sample data are acquired. Each set of sample data includes a sample latent vector, a guide vector, and a standard latent vector. The standard latent vector is obtained by sampling and compressing the sample hair model. The sample latent vector is obtained by adding noise to the standard latent vector. The guide vector is used to guide the preset model to denoise the sample latent vector. The preset model is trained based on the multiple sets of sample data until it converges to obtain the hair generation model. The training of the preset model based on the multiple sets of sample data until it converges to obtain the hair generation model includes: guiding the preset model to denoise the sample latent vectors based on the guiding vectors to obtain denoised sample latent vectors; decoding the denoised sample latent vectors to obtain the guiding line model corresponding to the sample latent vectors; comparing the denoised sample latent vectors with the standard latent vectors to obtain a comparison result; and adjusting the parameters of the preset model based on the comparison result until the preset model converges to obtain the hair generation model.

3. The method according to claim 2, characterized in that, For each set of sample data, the sample latent vector and the standard latent vector are obtained, including: A sample hair model is obtained, the sample hair model including multiple second guide lines, the second guide lines being used to represent the center lines of hair bundles in the sample hair model, the sample hair model being obtained from a preset database; For each second guide line, the second guide line is divided equally according to a first preset ratio to obtain multiple second three-dimensional coordinates; The result vector field is sampled and compressed in the scalp region of the sample hair model at a second preset ratio to obtain the standard latent vector. The result vector field is a vector composed of multiple second three-dimensional coordinates corresponding to multiple second guide lines in the sample hair model. The standard latent vector is noise-added to obtain the sample latent vector.

4. The method according to claim 2, characterized in that, For each set of sample data, the guiding vector is obtained, including: The sample hair model is rendered from multiple perspectives to obtain a multi-view rendering image; The multi-view rendered image is converted into a multi-view line sketch using a line detector. The multi-viewline sketch is encoded into the guiding vector using the Dino V2 model.

5. The method according to claim 1, characterized in that, The process of obtaining the latent vector of the image includes: Receive the described image; The image description is input into the Dino V2 model for latent vector extraction to obtain the latent vector of the image.

6. An electronic device, characterized in that, include: At least one memory and at least one first processor; The memory is used to store computer-executable programs or instructions; The first processor is used to invoke a computer-executable program or instruction in the memory, causing the electronic device to execute the method for generating a three-dimensional hair model according to any one of claims 1-5.

7. An electronic device, characterized in that, include: Second processor; The second processor is configured to execute a computer-executable program or instructions in the memory, causing the electronic device to perform the method for generating a three-dimensional hair model according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer-executable program or instructions, the computer-executable program or instructions being configured to perform the method for generating a three-dimensional hair model according to any one of claims 1-5.

Images