A three-dimensional model drawing method, storage medium, electronic equipment and device

Abstract

The present application relates to a three-dimensional model drawing method, storage medium, electronic device and device, and to the technical field of three-dimensional model drawing. The method includes: using a preset depth perception 2D diffusion model to process the first image information of the model to be constructed, and obtaining a corresponding view condition image, wherein the view condition image includes texture visual feature information; constructing an initial rough texture map based on multiple view condition images of the model to be constructed; obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed; and obtaining a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed. In the process of three-dimensional model construction, the present application optimizes the rough texture generation and UV space texture refinement, effectively improves the quality of the texture map, and improves its consistency and applicability, and effectively ensures the effect of three-dimensional model construction.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional model drawing, and in particular to a three-dimensional model drawing method, storage medium, electronic equipment and device. Background Art

[0002] Currently, the generated rough texture maps often involve lighting shadows, which are caused by the 2D image diffusion model, or texture holes caused by self-occlusion during the rendering process. Furthermore, existing technologies also face the problem of texture discontinuity caused by UV mapping. This is because the 3D surface texture is cut into a series of separate texture fragments on the UV plane, which complicates learning the 3D adjacency relationship between fragments on the UV plane. There is still a lack of effective improvement methods for the technical problems involved in the rendering of 3D models.

[0003] Therefore, in order to solve the above technical problems, a three-dimensional model drawing technology is now provided. Summary of the Invention

[0004] The present application provides a three-dimensional model drawing method, storage medium, electronic device and apparatus, which optimizes rough texture generation and UV space texture refinement during the three-dimensional model construction process, effectively improves the quality of texture images, and enhances their consistency and applicability, effectively ensuring the effect of three-dimensional model construction.

[0005] In a first aspect, the present application provides a three-dimensional model drawing method, the method comprising the following steps:

[0006] Processing first image information of the model to be constructed using a preset depth perception 2D diffusion model to obtain a corresponding view condition image, wherein the view condition image includes texture visual feature information;

[0007] constructing an initial rough texture map based on the plurality of view condition images of the model to be constructed;

[0008] Obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed;

[0009] Based on the refined texture map and the UV space coordinate information of the model to be constructed, a textured 3D model is obtained; wherein,

[0010] The first image information is image description text information or a depth map corresponding to the model to be constructed or a set input image.

[0011] Furthermore, the method further comprises the following steps:

[0012] The depth-aware 2D diffusion model is constructed and trained based on a depth-aware algorithm.

[0013] Furthermore, obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed includes the following steps:

[0014] Mapping the surface of the model to be constructed corresponding to the UV space position map onto the initial rough texture map;

[0015] The texture holes of the initial rough texture map are filled using UV completion technology, thereby enhancing the texture details of the initial rough texture map to obtain the refined texture map.

[0016] Furthermore, the step of obtaining a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed includes the following steps:

[0017] Based on the refined texture map and the UV space coordinate information of the model to be constructed, the refined texture map is mapped back to the surface of the model to be constructed, thereby obtaining the textured 3D model.

[0018] Furthermore, the method further comprises the following steps:

[0019] Rendering is performed based on different perspectives of the model to be constructed to obtain corresponding depth information, and then the depth maps corresponding to the different perspectives are obtained.

[0020] Furthermore, the method further comprises the following steps:

[0021] Using a preset depth perception 2D diffusion model, processing the first image information of the model to be constructed to obtain view condition images corresponding to different viewing angles;

[0022] An initial rough texture map is constructed based on the multiple view condition images corresponding to different viewing angles of the model to be constructed.

[0023] Furthermore, the method further comprises the following steps:

[0024] The UV space position map is obtained based on the 3D point coordinates of the model to be constructed.

[0025] In a second aspect, the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements a three-dimensional model drawing method mentioned in the first aspect.

[0026] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program running on the processor, and when the processor executes the computer program, a three-dimensional model drawing method mentioned in the first aspect is implemented.

[0027] In a fourth aspect, the present application provides a three-dimensional model drawing device, the device comprising:

[0028] a view condition image generation module, configured to process first image information of a to-be-constructed model using a preset depth perception 2D diffusion model to obtain a corresponding view condition image, wherein the view condition image includes texture visual feature information;

[0029] a multi-view texture fusion module, configured to construct an initial rough texture map based on a plurality of view condition images of the model to be constructed;

[0030] A UV space texture refinement module, which is used to obtain a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed;

[0031] A texture mapping module is used to obtain a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed; wherein,

[0032] The first image information is image description text information or a depth map corresponding to the model to be constructed or a set input image.

[0033] The beneficial effects of the technical solution provided by this application include:

[0034] During the three-dimensional model construction process, this application optimizes the rough texture generation and UV space texture refinement, effectively improving the quality of the texture map, and improving its consistency and applicability, effectively ensuring the effect of three-dimensional model construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Explanation of terms:

[0036] UV: U-VEEZ, a two-dimensional texture coordinate with vertex component information for polygonal and subdivision surface meshes.

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 A flowchart of the steps of the three-dimensional model drawing method provided in an embodiment of the present application;

[0039] Figure 2 This is a structural block diagram of the three-dimensional model drawing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] The embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0042] The embodiments of the present application provide a three-dimensional model drawing method, storage medium, electronic device and apparatus, which optimize rough texture generation and UV space texture refinement during the three-dimensional model construction process, effectively improve the quality of the texture map, and enhance its consistency and applicability, thereby effectively ensuring the effect of the three-dimensional model construction.

[0043] To achieve the above technical effects, the overall idea of ​​this application is as follows:

[0044] A three-dimensional model drawing method, the method comprising the following steps:

[0045] S1. Processing first image information of a model to be constructed using a preset depth perception 2D diffusion model to obtain a corresponding view condition image, where the view condition image includes texture visual feature information;

[0046] S2. constructing an initial rough texture map based on multiple view condition images of the model to be constructed;

[0047] S3, obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed;

[0048] S4, obtaining a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed; wherein,

[0049] The first image information is image description text information or a depth map corresponding to the model to be constructed or a set input image.

[0050] The embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0051] First, see Figure 1 As shown, the embodiment of the present application provides a three-dimensional model drawing method, which includes the following steps:

[0052] S1. Processing first image information of a model to be constructed using a preset depth perception 2D diffusion model to obtain a corresponding view condition image, where the view condition image includes texture visual feature information;

[0053] S2. constructing an initial rough texture map based on multiple view condition images of the model to be constructed;

[0054] S3, obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed;

[0055] S4, obtaining a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed; wherein,

[0056] The first image information is image description text information or a depth map corresponding to the model to be constructed or a set input image.

[0057] Specifically, during the specific operation of step S1, the 3D model may be rendered from different perspectives to obtain depth information, i.e., a depth map, and then a texture image is sampled from the 2D diffusion model according to the depth map and the image description text information / input image;

[0058] In the specific operation of step S2, the sampled texture image is back-projected onto the surface of the model to be constructed, that is, the surface of the 3D model, and texture information is accumulated from each viewing angle to form an initial rough texture map;

[0059] In the specific operation of step S3, the texture coordinates of the surface of the model to be constructed, that is, the 3D model, are first expanded into 2D space, and then the texture is refined in the UV space using UV completion and UV super-resolution techniques. Finally, the depth-aware 2D diffusion model is used to further optimize the texture quality and eliminate the influence of lighting;

[0060] In the specific operation of step S4, the texture of the UV space in the refined texture map is mapped back to the surface of the model to be constructed, that is, the surface of the 3D model, and the textured 3D model is rendered into a final image, that is, a textured 3D model, using a rendering engine.

[0061] It should be noted that the embodiment of the present application requires initialization of the pre-trained depth perception 2D diffusion model. The parameters of the depth perception 2D diffusion model can be image description text information or the depth map corresponding to the model to be constructed or the set input image. The processing process is to load the pre-trained depth perception 2D diffusion model, prepare to receive image description text information or input image, and then complete the initialization of the model to be ready and wait for receiving input.

[0062] In the embodiment of the present application, during the three-dimensional model construction process, optimization is performed on rough texture generation and UV space texture refinement, which effectively improves the quality of the texture map and increases its consistency and applicability, effectively ensuring the effect of three-dimensional model construction.

[0063] Furthermore, the method further comprises the following steps:

[0064] The depth-aware 2D diffusion model is constructed and trained based on a depth-aware algorithm.

[0065] Furthermore, obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed includes the following steps:

[0066] Mapping the surface of the model to be constructed corresponding to the UV space position map onto the initial rough texture map;

[0067] The texture holes of the initial rough texture map are filled using UV completion technology, thereby enhancing the texture details of the initial rough texture map to obtain the refined texture map.

[0068] Specifically, in this processing process, the received parameters are the initial rough texture map and the UV space position map corresponding to the model to be constructed;

[0069] Specifically, the processing includes:

[0070] UV mapping: maps the surface of a 3D model to a 2D UV space;

[0071] UV completion: Use UV completion technology to fill texture holes in the UV plane;

[0072] UV super-resolution: Enhance texture details and improve texture quality.

[0073] Furthermore, a refined high-quality texture map is obtained without the influence of lighting.

[0074] Furthermore, the step of obtaining a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed includes the following steps:

[0075] Based on the refined texture map and the UV space coordinate information of the model to be constructed, the refined texture map is mapped back to the surface of the model to be constructed, thereby obtaining the textured 3D model.

[0076] Specifically, in this operation process, the processed parameters include the refined texture map and the UV space coordinate information of the model to be constructed, where the UV space coordinate information is the UV coordinate of the 3D model;

[0077] The specific processing content of the operation flow is: mapping the refined texture image back to the surface of the model to be constructed, that is, mapping the 2D texture image back to the surface of the 3D model;

[0078] Finally, the textured 3D model is obtained, which can be used for rendering or other applications later.

[0079] Furthermore, the method further comprises the following steps:

[0080] Rendering is performed based on different perspectives of the model to be constructed to obtain corresponding depth information, and then the depth maps corresponding to the different perspectives are obtained.

[0081] Furthermore, the method further comprises the following steps:

[0082] Using a preset depth perception 2D diffusion model, processing the first image information of the model to be constructed to obtain view condition images corresponding to different viewing angles;

[0083] An initial rough texture map is constructed based on the multiple view condition images corresponding to different viewing angles of the model to be constructed.

[0084] Furthermore, the method further comprises the following steps:

[0085] The 3D point coordinates of the model to be constructed are used to obtain the UV space position map.

[0086] Based on the technical solution of the embodiment of this application, the specific operations are as follows:

[0087] First stage, rough texture generation:

[0088] First, initialize the pre-trained 2D diffusion model:

[0089] Use SDXL's text-based graph model as the backbone network for texture generation and load a pre-trained depth-aware 2D diffusion model.

[0090] It should be noted that the embodiments of this application use SDXL's text-based graph model as the backbone for texture generation. This depth-aware 2D diffusion model can generate semantically meaningful texture maps based on text or image input, making the generated textures richer and more diverse. The use of this technology greatly improves the flexibility and quality of texture generation.

[0091] SDXL's text-generated graph model is a deep learning-based generative model specifically designed to generate high-quality images from textual descriptions or image inputs. The core of this model lies in its ability to understand the semantic information contained in text or images and convert it into visual content. In this paper, the SDXL model is used as the backbone network for texture generation, responsible for generating an initial rough texture map based on the input conditions.

[0092] The process begins by inputting the 3D model's view or depth information, along with a text description or target image, into the SDXL model. The model then uses internal neural network layers to gradually generate texture images that match the input conditions from random noise. These images are then used to construct a rough texture map, laying the foundation for subsequent UV space texture refinement.

[0093] Second, render the depth map:

[0094] Render the 3D mesh model from different camera perspectives and generate corresponding depth maps.

[0095] Third, conditionally sample the texture image:

[0096] Using textual hints or image inputs, combined with depth maps as conditions, a view-conditioned image is generated through 2D diffusion model sampling.

[0097] Fourth, multi-view texture fusion:

[0098] The view-condition images generated from different perspectives are fused, and the entire rough texture map is gradually generated by alternately performing rendering, sampling and back-projection processes.

[0099] It should be noted that the multi-view texture fusion strategy of the embodiment of the present application is:

[0100] During the rough texture generation phase, this embodiment of the present application defines six axis-aligned primary viewpoints and samples two texture images from a pair of symmetrical viewpoints in a single diffusion process. This multi-view texture fusion strategy is a key feature of this embodiment of the present application, helping to ensure texture consistency and coherence when viewed from different angles. This strategy significantly improves the visual quality and practicality of the texture image, providing a crucial safeguard.

[0101] The specific principles are as follows: The multi-view texture fusion strategy is a key technology used in the embodiments of this application to generate high-quality, consistent textures. This strategy is based on texture images captured from multiple perspectives and creates a comprehensive texture map by fusing these images, thereby ensuring that the texture remains consistent and coherent across different perspectives. In the embodiments of this application, the implementation of this strategy involves the following steps:

[0102] The first step is initial view texture generation: First, the 3D model is rendered from an initial viewpoint to generate a depth map and texture image.

[0103] The second step is texture generation for subsequent views: For subsequent viewpoints, image inpainting techniques are used to generate textures, which ensures that the newly generated textures match the textures of the previous viewpoints.

[0104] The third step is texture fusion: the texture images generated by all viewpoints are fused to form a unified rough texture map. This process takes into account the overlapping areas between viewpoints to ensure the continuity and consistency of the texture.

[0105] Specifically, the operation process of the above content is described as follows:

[0106] 1. Define viewpoints: Determine a series of viewpoints around the 3D model, usually selecting six axis-aligned main viewpoints.

[0107] 2. Rendering depth map: For each viewpoint, render the 3D model to generate a depth map.

[0108] 3. Conditional sampling: Using the depth map and text or image conditions, the texture image is generated by sampling the pre-trained 2D diffusion model.

[0109] 4. Multi-view fusion: The texture images of all viewpoints are fused, and the entire rough texture map is gradually generated by alternating rendering, sampling and back-projection processes.

[0110] 5. Texture accumulation: For non-initial viewpoints, the newly generated texture is merged with the texture of the previous viewpoint using the accumulation method to ensure texture continuity.

[0111] In addition, the technical advantages of the above multi-view texture fusion strategy are:

[0112] 1. Perspective consistency: The multi-view texture fusion strategy ensures that the texture is highly consistent under different perspectives, avoiding visual incoherence and mismatch problems.

[0113] 2. Texture integrity: By fusing texture information from multiple perspectives, a complete texture map without obvious seams or gaps can be generated.

[0114] 3. Illumination invariance: Since the texture is generated without considering specific lighting conditions, the generated texture is not affected by lighting changes and has better adaptability.

[0115] 4. Detail richness: The multi-view fusion strategy can capture more details of the 3D model, making the generated texture richer and more realistic.

[0116] 5. Flexibility: This strategy allows texture generation from different viewpoints and under different lighting conditions, providing higher flexibility and the ability to adapt to different application scenarios.

[0117] Second stage, texture refinement in UV space:

[0118] First, mapping:

[0119] Use the UV-Atlas tool or other UV unfolding technology to map the vertices of the 3D model to the 2D UV space to form a UV mesh.

[0120] Second, texture refinement in UV space:

[0121] In UV space, a position encoder is used to represent the 3D adjacency information of texture fragments, which exists in the form of a position map in UV space;

[0122] Apply UV completion technology to fill texture holes in the UV plane according to the guidance of the position encoder to solve the self-occlusion problem;

[0123] Through UV super-resolution technology, the details and visual beauty of the texture are further enhanced, and high-resolution texture maps are generated.

[0124] It should be noted that the texture refinement process of the UV space in the embodiment of the present application is as follows:

[0125] The embodiment of the present application performs a diffusion process in UV space to alleviate the effects of lighting and self-occlusion, and enhance the visual aesthetics of the texture map. In particular, by introducing position encoders, UV completion, and UV super-resolution technology, the texture discontinuity problem caused by UV mapping is effectively solved. This series of texture refinement technologies is the core innovation of the present invention, representing a major breakthrough in improving texture quality and should be protected.

[0126] Specifically, the technical principles of the texture refinement process in UV space are as follows:

[0127] The UV space texture refinement process is a key technology used in the embodiments of this application to improve texture quality and resolve texture discontinuities. This process is performed within the UV space and utilizes position encoders and a specific diffusion model to guide the refinement of texture generation, thereby resolving the texture fragmentation problem caused by UV mapping.

[0128] The texture refinement process in UV space involves the following key technologies:

[0129] 1. UV mapping: Map the surface texture of the 3D model to the 2D UV space to form a UV grid.

[0130] 2. Position encoder: Use the position encoder to represent the 3D adjacency information of texture fragments in UV space to help the model understand the relationship between texture fragments.

[0131] 3. UV completion: Use UV completion technology to fill texture holes in the UV plane and solve the self-occlusion problem.

[0132] 4. UV super-resolution: Use UV super-resolution technology to enhance the details and visual beauty of textures and improve texture quality.

[0133] Specifically, the operation process of the texture refinement process in UV space is described as follows:

[0134] 1. UV mapping: Convert the vertices of the 3D model to 2D UV space through UV mapping to create a UV mesh.

[0135] 2. Rough texture mapping: Map the rough texture map obtained from multi-view fusion onto the UV grid.

[0136] 3. Position map generation: Based on the geometric information of the 3D model, a position map is generated, which represents the 3D adjacency relationship of the texture fragments in the UV space.

[0137] 4. UV completion: Use UV completion technology to fill holes and incomplete areas in the UV plane according to the position map and rough texture map.

[0138] 5. UV super-resolution: Apply UV super-resolution technology to further refine textures and enhance the visual details and quality of textures.

[0139] 6. Texture optimization: Optimize in UV space through the diffusion model to generate the final high-quality texture map.

[0140] In addition, the technical advantages of the texture refinement process in UV space are as follows:

[0141] 1. Texture continuity: Through UV completion and UV super-resolution technology, the present invention can solve the texture discontinuity problem caused by UV mapping and ensure the visual integrity and coherence of the texture.

[0142] 2. High-quality texture generation: UV super-resolution technology can significantly improve the details and quality of textures, making them more realistic and rich.

[0143] 3. Lighting independence: The generated texture is not affected by lighting conditions and can maintain consistent visual effects in any lighting environment.

[0144] 4. Flexibility and adaptability: The use of position encoders provides fine control over the texture generation process, enabling the present invention to adapt to a variety of different 3D models and texture requirements.

[0145] 5. Automation and efficiency: The entire process is automated, reducing manual intervention and improving the efficiency and scalability of texture generation.

[0146] In addition, the embodiment of the present application also has the ability to generate textures without being affected by lighting;

[0147] The embodiments of this application can generate texture maps regardless of lighting conditions, meaning that the generated textures maintain consistent visual effects under varying lighting conditions. This capability addresses the lighting sensitivity issue of texture maps in the prior art, significantly improving their usability and flexibility. This is another key feature of the embodiments of this application, demonstrating significant market application value and technological innovation.

[0148] Detailed description of the technical principle of the texture generation process without the influence of lighting:

[0149] Illumination-independent texture generation capability refers to the ability of embodiments of this application to generate textures that are unaffected by lighting conditions, meaning that textures maintain consistent visual effects regardless of lighting conditions. This capability is achieved through a specially designed diffusion model during texture refinement in UV space, which learns and generates texture data that does not include lighting information.

[0150] The implementation of the texture generation process without the influence of lighting is based on the following principles:

[0151] 1. Texture representation in UV space: The surface texture of a 3D model is converted to 2D UV space through UV mapping, so that texture generation and editing no longer depend on the specific shape and complex lighting conditions of the 3D model.

[0152] 2. Illumination independence of the diffusion model: In UV space, a specific diffusion model is used to generate textures. The model does not depend on specific lighting conditions during training, thus ensuring that the generated texture does not contain lighting information.

[0153] 3. Texture prior learning: The diffusion model acquires the ability to generate high-quality, unaffected textures by learning a large amount of unilluminated texture data.

[0154] The operation flow of the texture generation process without the influence of lighting is as follows:

[0155] 1. UV mapping: Map the surface texture of the 3D model to the 2D UV space to create a UV grid.

[0156] 2. Rough texture generation: Use the pre-trained 2D diffusion model to generate the initial rough texture map.

[0157] 3. Texture refinement: In UV space, the rough texture is refined through a specific diffusion model to generate a texture without the influence of lighting.

[0158] 4. Texture optimization: Improve the details and quality of textures through further optimization processes such as UV completion and UV super-resolution.

[0159] 5. Texture mapping back to 3D model: Map the refined texture map back to the surface of the 3D model in preparation for rendering.

[0160] Furthermore, the technical advantages of the texture generation process without the influence of lighting are as follows:

[0161] 1. Lighting independence: The generated texture does not depend on specific lighting conditions and can be used in any lighting environment, which greatly improves the applicability and flexibility of the texture.

[0162] 2. Texture quality: The ability to generate textures without being affected by lighting ensures high quality and realism of textures, maintaining consistent visual effects regardless of lighting conditions.

[0163] 3. Editing flexibility: Since the texture is generated in UV space, it makes it easier to edit and modify the texture without being restricted by the complex shape of the 3D model.

[0164] 4. Compatibility: Unlit textures are compatible with various modern graphics rendering pipelines, including Physically Based Rendering (PBR), allowing textures to be seamlessly integrated into different rendering systems.

[0165] 5. Freedom of artistic creation: Artists and designers can freely create textures without considering lighting conditions, which provides them with greater creative space.

[0166] Third, the texture is mapped back to the 3D model:

[0167] Map the refined texture map in UV space back to the surface of the 3D model, use Kaolin tools for rendering and texture projection to ensure the correct display of the texture on the 3D model.

[0168] In view of the above technical content, the following technical principles are supplemented:

[0169] First, rough texture generation:

[0170] Pre-trained 2D image diffusion model: This paper utilizes a pre-trained depth-aware 2D image diffusion model that can generate high-quality 2D images based on text or image conditions. This model is a generative model based on deep learning, which can gradually generate clear images from random noise.

[0171] View-conditional image generation: By rendering a 3D model into a depth map, combined with text or image conditions and depth conditions, a view-dependent texture image is generated.

[0172] Multi-view texture fusion: To generate the initial coarse texture map, the present invention uses multi-view image fusion technology. By alternately performing rendering, sampling, and back-projection processes, texture information is generated from multiple viewpoints and fused into a unified texture map to form a coarse texture map.

[0173] Second, texture refinement in UV space:

[0174] UV mapping and position encoding: In UV space, this embodiment uses a position encoder to represent the 3D adjacency information of texture fragments. This information exists in the form of a position map in UV space, where each non-background element is a 3D point coordinate that can be obtained from the 3D model through UV mapping.

[0175] UV completion and UV super-resolution: To address the texture discontinuity problem caused by UV mapping, this embodiment introduces UV completion and UV super-resolution technologies. UV completion is used to fill texture holes in the UV plane, while UV super-resolution technology is used to enhance the visual beauty and details of the texture.

[0176] Texture Diffusion Model: In UV space, the present invention uses a texture diffusion model to learn the continuity and details of textures. This model generates textures by predicting the noise added to the noise potential state.

[0177] Through the principles described above, the embodiments of the present application can achieve efficient conversion from coarse textures to fine textures and generate high-quality 3D model textures. These principles ensure semantic consistency and high-quality visual effects of textures, while also addressing issues such as lighting effects and texture discontinuities.

[0178] It should be noted that, in the specific implementation of the embodiment of the present application, a variety of advanced encoders and tools are comprehensively applied.

[0179] Throughout the texture generation and refinement process, the present embodiment utilizes a comprehensive set of advanced encoders and tools, including but not limited to the IP-Adapter image encoder, the ControlNet domain encoder, the PyTorch framework, and the Kaolin tool. This comprehensive application is a unique feature of the present embodiment, significantly improving the efficiency and quality of texture generation through the rational utilization and integration of multiple advanced technologies. This technology integration strategy is a key feature of the present invention and represents a cutting-edge advancement in 3D object texturing technology.

[0180] Detailed description of the technical principles of the above technology:

[0181] A key aspect of the present invention is the integrated application of multiple advanced encoders and tools to convert text or image input into high-quality 3D model textures. The integrated use of these encoders and tools not only improves the efficiency and quality of texture generation, but also enhances the flexibility and scalability of the system.

[0182] Specifically, these components include but are not limited to:

[0183] 1. Image encoder: used to process image conditions and convert the input image information into a format that the model can understand.

[0184] 2. Text encoder: used to process text descriptions and extract semantic information to guide texture generation.

[0185] 3. Position encoder: Represents the 3D adjacency information of texture fragments in UV space, helping the model understand the relationship between texture fragments.

[0186] 4. Diffusion model: used to generate and optimize textures in UV space, learn and generate high-quality textures.

[0187] 5. Rendering and texture projection tools: such as Pytorch3D, used to map the generated texture back to the 3D model and render it.

[0188] Specifically, when applying a variety of advanced encoders and tools, the operation process is as follows:

[0189] 1. Input processing: Receive text or image input and convert it into a format that the model can process through the corresponding encoder.

[0190] 2. Texture generation: Use the pre-trained diffusion model and the encoded input conditions to generate the initial rough texture map.

[0191] 3. UV space processing: The surface texture of the 3D model is converted to 2D UV space through UV mapping, and texture refinement is performed using position encoders and diffusion models.

[0192] 4. Texture optimization: Apply UV completion and UV super-resolution technology to further improve the details and quality of textures.

[0193] 5. Texture mapping and rendering: Use rendering and texture projection tools to map the refined texture map back to the 3D model and perform final rendering.

[0194] Specifically, when combining multiple advanced encoders and tools, the technical advantages are as follows:

[0195] 1. Efficiency: Integrated advanced encoders and tools make the entire texture generation process more efficient, reducing the need for manual intervention and post-processing.

[0196] 2. Flexibility: The combination of multiple encoders and tools provides a variety of capabilities for processing different types of inputs, allowing the system to adapt to a variety of different texture generation needs.

[0197] 3. High-quality output: By comprehensively applying multiple advanced technologies, the present invention can generate high-quality, detail-rich textures, thus improving the visual quality of the final product.

[0198] 4. Ease of use: The integrated tools and encoders simplify the operation process, making it easy for non-professionals to generate and edit textures.

[0199] 5. Scalability: The modular design of the system allows new encoders and tools to be easily added in the future to adapt to changing technology needs and market trends.

[0200] Based on the above technical content, the embodiments of this application generate high-quality, consistent, and applicable texture maps, which is achieved through the combined effect of all the above technical points. This is not only the key point of the technical solution, but also the core value that the embodiments of this application aim to protect. This high-quality texture map has important value in a variety of application scenarios, such as game development, film production, virtual reality, etc., and is an important protection content of the embodiments of this application.

[0201] In a second aspect, the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements a three-dimensional model drawing method mentioned in the first aspect.

[0202] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program running on the processor, and when the processor executes the computer program, a three-dimensional model drawing method mentioned in the first aspect is implemented.

[0203] Fourthly, see Figure 2 As shown, the present application provides a three-dimensional model drawing device, which includes:

[0204] a view condition image generation module, configured to process first image information of a to-be-constructed model using a preset depth perception 2D diffusion model to obtain a corresponding view condition image, wherein the view condition image includes texture visual feature information;

[0205] a multi-view texture fusion module, configured to construct an initial rough texture map based on a plurality of view condition images of the model to be constructed;

[0206] A UV space texture refinement module, which is used to obtain a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed;

[0207] A texture mapping module is used to obtain a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed; wherein,

[0208] The first image information is image description text information or a depth map corresponding to the model to be constructed or a set input image.

[0209] In the embodiment of the present application, during the three-dimensional model construction process, optimization is performed on rough texture generation and UV space texture refinement, which effectively improves the quality of the texture map and increases its consistency and applicability, effectively ensuring the effect of three-dimensional model construction.

[0210] Furthermore, the device also includes a diffusion model construction module, which is used to construct and train the depth perception 2D diffusion model based on the depth perception algorithm.

[0211] Furthermore, the UV space texture refinement module is further used to map the surface of the model to be constructed corresponding to the UV space position map onto the initial rough texture map;

[0212] The UV space texture refinement module is further configured to fill texture holes of the initial rough texture map using a UV completion technique, thereby enhancing texture details of the initial rough texture map to obtain the refined texture map.

[0213] Furthermore, the texture mapping module is further configured to map the refined texture map back to the surface of the model to be constructed based on the refined texture map and UV space coordinate information of the model to be constructed, thereby obtaining the textured 3D model.

[0214] Furthermore, the device also includes a depth map acquisition module, which is used to render based on different perspectives of the model to be constructed, obtain corresponding depth information, and then obtain the depth maps corresponding to different perspectives.

[0215] Furthermore, the view condition image generation module is further configured to process the first image information of the model to be constructed using a preset depth perception 2D diffusion model to obtain view condition images corresponding to different viewing angles;

[0216] The multi-view texture fusion module is further configured to construct an initial rough texture map based on a plurality of view condition images corresponding to different viewing angles of the model to be constructed.

[0217] Furthermore, the device also includes a UV space position map acquisition module, which is used to obtain the UV space position map based on the 3D point coordinates of the model to be constructed.

[0218] It should be noted that the three-dimensional model drawing device provided in the embodiment of the present application, and its corresponding technical problems, technical means and technical effects are similar to the principles of the three-dimensional model drawing method at the principle level.

[0219] It should be noted that, in this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises", "includes" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0220] The above are merely specific embodiments of the present application to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but rather is intended to conform to the widest scope consistent with the principles and novel features of the present application.

Claims

1. A three-dimensional model drawing method, characterized in that: The method comprises the following steps: using a preset depth perception 2D diffusion model, processing first image information of a model to be constructed, and obtaining a corresponding view condition image, wherein the view condition image includes texture visual feature information; constructing an initial rough texture map based on the plurality of view condition images of the model to be constructed; Obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed; Obtaining a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed; in, The first image information is image description text information or a depth map corresponding to the model to be constructed or a set input image; The step of obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed comprises the following steps: mapping the surface of the model to be constructed corresponding to the UV space position map onto the initial rough texture map; Filling the texture holes of the initial rough texture map by using UV completion technology, thereby enhancing the texture details of the initial rough texture map to obtain the refined texture map; The process of obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed includes: UV mapping: Map the surface of a 3D model to a 2D UV space; UV completion: Use UV completion technology to fill texture holes in the UV plane; UV super-resolution: enhance texture details and improve texture quality; Then, a refined high-quality texture map is obtained; The step of obtaining a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed comprises the following steps: based on the refined texture map and the UV space coordinate information of the model to be constructed, mapping the refined texture map back to the surface of the model to be constructed, that is, mapping the 2D texture map back to the surface of the 3D model to obtain the textured 3D model.

2. The three-dimensional model drawing method according to claim 1, characterized in that: The method further comprises the following steps: constructing and training the depth perception 2D diffusion model based on a depth perception algorithm.

3. The three-dimensional model drawing method according to claim 1, characterized in that: The method further comprises the following steps: performing rendering based on different viewing angles of the model to be constructed to obtain corresponding depth information, and then obtaining the depth maps corresponding to the different viewing angles.

4. The three-dimensional model drawing method according to claim 3, characterized in that: The method further comprises the following steps: using a preset depth perception 2D diffusion model, processing the first image information of the model to be constructed, and obtaining view condition images corresponding to different viewing angles; An initial rough texture map is constructed based on the multiple view condition images corresponding to different viewing angles of the model to be constructed.

5. The three-dimensional model drawing method according to claim 1, characterized in that: The method further comprises the following steps: obtaining the UV space position map based on the 3D point coordinates of the model to be constructed.

6. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program running on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A three-dimensional model drawing device, characterized in that: The device comprises: a view condition image generation module, which is used to process first image information of a model to be constructed using a preset depth perception 2D diffusion model to obtain a corresponding view condition image, wherein the view condition image includes texture visual feature information; A multi-view texture fusion module, which is used to construct an initial rough texture map based on a plurality of view condition images of the model to be constructed; A UV space texture refinement module, which is used to obtain a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed; A texture mapping module, which is used to obtain a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed; wherein the first image information is image description text information or a depth map corresponding to the model to be constructed or a set input image; The step of obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed comprises the following steps: mapping the surface of the model to be constructed corresponding to the UV space position map onto the initial rough texture map; Filling the texture holes of the initial rough texture map by using UV completion technology, thereby enhancing the texture details of the initial rough texture map to obtain the refined texture map; The process of obtaining a refined texture map based on the initial rough texture map and the UV space position map corresponding to the model to be constructed includes: UV mapping: Map the surface of a 3D model to a 2D UV space; UV completion: Use UV completion technology to fill texture holes in the UV plane; UV super-resolution: enhance texture details and improve texture quality; Then, a refined high-quality texture map is obtained; The step of obtaining a textured 3D model based on the refined texture map and the UV space coordinate information of the model to be constructed comprises the following steps: based on the refined texture map and the UV space coordinate information of the model to be constructed, mapping the refined texture map back to the surface of the model to be constructed, that is, mapping the 2D texture map back to the surface of the 3D model to obtain the textured 3D model.

Images