Texture image generation method and device, computer equipment and storage medium

Abstract

The invention provides a texture image generation method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. Comprising the following steps: performing UV expansion on a target three-dimensional white mold to obtain UV space information of the target three-dimensional white mold; by taking the UV space information and the texture control information as reference information, carrying out multiple times of denoising on a random noise image based on a denoising network to obtain an intermediate texture image; and performing up-sampling on the intermediate texture image to obtain a UV texture image of the target three-dimensional white mold. According to the scheme, different texture images of the three-dimensional white mold can be generated only by changing the texture control information, and the generation efficiency and diversity of the texture images are greatly improved. Compared with an existing scheme of rendering images of different visual angles and then performing fusion, the accuracy of the generated texture image is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer equipment, and storage medium for generating a texture image. Background Art

[0002] One of the key stages of 3D (3D) development is modeling, which involves building models of characters, animals, and assets for games, animations, and applications. Texture is essentially a feature on the surface of a model that surrounds it and makes it appear realistic. A 3D model's texture refers to a two-dimensional image mapped onto the surface of a three-dimensional object, enhancing the object's visual effects and making it appear more realistic and detailed. Textures typically include properties such as color, smoothness, and reflectivity, and they play a key role in the 3D rendering process. Existing methods for generating texture images are often cumbersome and inefficient. Therefore, efficiently generating texture images for 3D models is a pressing issue. Summary of the Invention

[0003] The embodiment of the present application provides a method, apparatus, computer device and storage medium for generating a texture image. The technical solution is as follows:

[0004] In one aspect, a method for generating a texture image is provided, the method comprising:

[0005] Performing UV unfolding on the target 3D white model to obtain UV space information of the target 3D white model, wherein the UV unfolding is used to unfold the surface of the 3D model onto a 2D plane, and the UV space information is used to indicate the position and normal of the vertices on the target 3D white model in the UV space;

[0006] Using the UV spatial information and the texture control information as reference information, the random noise image is denoised multiple times based on a denoising network to obtain an intermediate texture image, wherein the texture control information is used to describe the style and color of the texture, and the denoising network is used to generate an image that meets the reference information;

[0007] The intermediate texture image is up-sampled to obtain a UV texture image of the target three-dimensional white model, where pixels in the UV texture image correspond to vertices on the target three-dimensional white model.

[0008] In another aspect, a texture image generation device is provided, the device comprising:

[0009] a two-dimensional information acquisition module, configured to perform UV unfolding on a target three-dimensional white model to obtain UV spatial information of the target three-dimensional white model, wherein the UV unfolding is used to unfold the surface of the three-dimensional model onto a two-dimensional plane, and the UV spatial information is used to indicate the position and normal of the vertices on the target three-dimensional white model in UV space;

[0010] a denoising module, configured to perform multiple denoising operations on the random noise image based on a denoising network using the UV spatial information and the texture control information as reference information to obtain an intermediate texture image, wherein the texture control information is used to describe the style and color of the texture, and the denoising network is used to generate an image that satisfies the reference information;

[0011] An upsampling module is configured to upsample the intermediate texture image to obtain a UV texture image of the target three-dimensional white model, wherein the pixels in the UV texture image correspond to vertices on the target three-dimensional white model.

[0012] In some embodiments, the UV space information includes a UV position map and a UV normal map, wherein the UV position map is used to indicate positions of vertices on the target 3D white model in the UV space, and the UV normal map is used to indicate normal directions of vertices on the target 3D white model in the UV space.

[0013] The denoising module includes:

[0014] A first feature extraction unit is configured to splice the UV position map and the UV normal map in a channel dimension to obtain UV geometric features;

[0015] A second feature extraction unit is used to extract features from the texture control information to obtain texture control features;

[0016] A denoising unit is used to input the UV geometric features, the texture control features and the random noise image into the denoising network to obtain the intermediate texture image.

[0017] In some embodiments, the denoising network includes a first sub-network and a second sub-network, the first sub-network is a stable diffusion SD large model, and the second sub-network includes an encoding part of the SD large model and a plurality of zero convolution layers;

[0018] The denoising unit is used to input the random noise image and the UV geometric features into the second sub-network, and the second sub-network performs multiple denoising on the random noise image to obtain multiple first intermediate features; input the texture control features and the random noise image into the first sub-network, and the first sub-network performs multiple denoising on the random noise image to obtain multiple second intermediate features; decode the multiple first intermediate features and the second intermediate features through the first sub-network to obtain the intermediate texture image; wherein, for any encoding layer in the second sub-network, the first intermediate features output by the encoding layer are the input of the decoding layer corresponding to the encoding layer in the first sub-network.

[0019] In some embodiments, the denoising unit is further configured to initialize the convolution weights of the three bottom channels in the first convolutional layer of the second sub-network to zero.

[0020] In some embodiments, the denoising network performs multiple denoising on the random noise image N times, where N is a positive integer; the denoising unit is configured to process the UV geometric features, the texture control features, and the random noise image through the denoising network for the first denoising process to obtain a first noise image; for the i-th denoising process, process the UV geometric features, the texture control features, and the second noise image through the denoising network to obtain a third noise image, where the second noise image is the noise image obtained after the i-1-th denoising process, where i is a positive integer greater than 1 and less than N; for the N-th denoising process, process the UV geometric features, the texture control features, and the fourth noise image through the denoising network to obtain the intermediate texture image, where the fourth noise image is the noise image obtained after the N-1-th denoising process.

[0021] In some instances, the second feature extraction unit is used to perform feature extraction on the text information to obtain the texture control feature when the texture control information is text; or to perform feature extraction on the image to obtain the texture control feature when the texture control information is an image; or to perform feature extraction on the text and image separately when the texture control information includes text and image, and splice the extracted text features and image features to obtain the texture control feature.

[0022] In some embodiments, the apparatus further comprises:

[0023] The image processing module is used to scale the coordinate values ​​corresponding to the pixel points in the UV texture image to a target range; for any pixel point, when the coordinate value of the pixel point after scaling is greater than 0, the pixel point is mapped to the vertex corresponding to the pixel point in the target three-dimensional white model.

[0024] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded and executed by the processor to implement the texture image generation method in the embodiment of the present application.

[0025] On the other hand, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the method for generating a texture image in an embodiment of the present application.

[0026] On the other hand, a computer program product is provided, which includes at least one program segment, wherein the at least one program segment is stored in a computer-readable storage medium, and a processor of a computer device reads the at least one program segment from the computer-readable storage medium, and the processor executes the at least one program segment, so that the computer device executes the texture image generation method described in any of the above implementations.

[0027] The present invention provides a method for generating texture images that incorporates UV space information and texture control information. Because UV space information contains all the geometric information of a three-dimensional object, the texture control information allows the generation of texture images to be directly controlled in UV space. Consequently, for the same three-dimensional white model, different texture images can be generated for the same model simply by changing the texture control information, significantly improving the efficiency and diversity of texture image generation. Compared to existing solutions that render images from different perspectives and then fuse them, this method addresses shortcomings such as occlusion and multi-perspective inconsistencies, and improves the accuracy of the generated texture images. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 A schematic diagram of an implementation environment of a texture image generation method provided in an embodiment of the present application;

[0030] Figure 2A flowchart of a method for generating a texture image provided in an embodiment of the present application;

[0031] Figure 3 A flowchart of another method for generating a texture image provided in an embodiment of the present application;

[0032] Figure 4 is a schematic diagram of a three-dimensional white mold provided according to an embodiment of the present application;

[0033] Figure 5 is a schematic diagram of texture control information provided according to an embodiment of the present application;

[0034] Figure 6 This is a schematic diagram of the principle of a diffusion generation model provided according to an embodiment of the present application;

[0035] Figure 7 is a structural diagram of a denoising network provided according to an embodiment of the present application;

[0036] Figure 8 is a flowchart of another method for generating a texture image according to an embodiment of the present application;

[0037] Figure 9 is a block diagram of a texture image generation device provided according to an embodiment of the present application;

[0038] Figure 10 is a structural block diagram of a terminal 1000 provided according to an embodiment of the present application;

[0039] Figure 11 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0041] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0042] In the present application, the term "at least one" means one or more, and the term "plurality" means two or more.

[0043] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, and display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the target 3D white model and texture control information involved in this application were obtained with full authorization.

[0044] The following is an introduction to the implementation environment involved in this application:

[0045] The method for generating a texture image provided in the embodiment of the present application can be executed by a computer device, which can be provided as a server or a terminal. The following is a schematic diagram of the implementation environment of the method for generating a texture image provided in the embodiment of the present application.

[0046] See also Figure 1 , Figure 1 A schematic diagram of an implementation environment of a method for generating a texture image provided in an embodiment of the present application, the implementation environment including a terminal 101 and a server 102. The terminal 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In some embodiments, the server 102 is used to train a denoising network. A target application is installed on the terminal 101, and the target application is used to render a three-dimensional model. In some embodiments, the terminal 101 is embedded with a trained denoising network, and the terminal 101 generates a UV texture image of a target three-dimensional white model through the denoising network. In other embodiments, the terminal 101 generates a UV texture image of a target three-dimensional white model through the denoising network on the server 102.

[0047] In some embodiments, the terminal 101 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a VR (Virtual Reality) device, an AR (Augmented Reality) device, etc., but is not limited thereto. In some embodiments, the server 102 is an independent server or a server cluster or distributed system composed of multiple servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some embodiments, the server 102 is primarily responsible for computing tasks, and the terminal 101 is responsible for secondary computing tasks; alternatively, the server 102 is responsible for secondary computing services, and the terminal 101 is responsible for primary computing tasks; alternatively, the server 102 and the terminal 101 use a distributed computing architecture for collaborative computing.

[0048] See also Figure 2 , Figure 2 A flowchart of a method for generating a texture image provided in an embodiment of the present application is provided. The method is executed by a computer device and includes the following steps.

[0049] 201. UV unfolding is performed on the target 3D white model to obtain UV space information of the target 3D white model. UV unfolding is used to unfold the surface of the 3D model onto a 2D plane. The UV space information is used to indicate the position and normal of the vertices on the target 3D white model in the UV space.

[0050] In an embodiment of the present application, the target three-dimensional white mold is a white mold of a three-dimensional model for which a texture image is to be generated. Among them, a white mold refers to an unpainted sculptural figure mold, an unpainted handmade model or a CNC machined model. This model is mainly used in the early stages of product design and industrial manufacturing to verify the design concept and the basic shape and size of the product. The white mold of a three-dimensional model refers to a three-dimensional model without texture mapping. UV (pronounced U-VEEZ) is a two-dimensional texture coordinate used to define a two-dimensional texture coordinate system, called "UV texture space", or simply UV space for short. Through UV unfolding, the surface of the three-dimensional model can be unfolded into UV space, that is, onto a two-dimensional plane. The three-dimensional white mold of the target three-dimensional model can obtain a corresponding two-dimensional image after UV unfolding. This application uses the two-dimensional image obtained by UV unfolding as UV space information.

[0051] 202. Using UV spatial information and texture control information as reference information, the random noise image is denoised multiple times based on the denoising network to obtain an intermediate texture image. The texture control information is used to describe the style and color of the texture, and the denoising network is used to generate an image that meets the reference information.

[0052] In an embodiment of the present application, a texture image is generated using a denoising network. The denoising network is an image generation network that can generate an image that meets the reference information by performing multiple denoising operations on a random noise image based on reference information. The present application uses the UV spatial information determined in the above steps and the acquired texture control information as reference information to generate an intermediate texture image. The texture control information can be text or an image that describes the style and color of the texture, which is not limited in the embodiment of the present application.

[0053] 203. Up-sample the intermediate texture image to obtain a UV texture image of the target 3D white model. Pixels in the UV texture image correspond to vertices on the target 3D white model.

[0054] In the embodiment of the present application, by upsampling the intermediate texture image, the spatial resolution of the texture image can be increased, so that it can fit the size of the target three-dimensional white model and then be fitted onto the target three-dimensional white model to form a three-dimensional model.

[0055] The present invention provides a method for generating texture images that incorporates UV space information and texture control information. Because UV space information contains all the geometric information of a three-dimensional object, the texture control information allows the generation of texture images to be directly controlled in UV space. Consequently, for the same three-dimensional white model, different texture images can be generated for the same model simply by changing the texture control information, significantly improving the efficiency and diversity of texture image generation. Compared to existing solutions that render images from different perspectives and then fuse them, this method addresses shortcomings such as occlusion and multi-perspective inconsistencies, and improves the accuracy of the generated texture images.

[0056] above Figure 2 The basic process of the texture image generation method provided in this application is as follows: Figure 3 Further introduction to the method of generating texture images. Figure 3 , Figure 3 A flowchart of another method for generating a texture image provided in an embodiment of the present application, which is executed by a computer device and includes the following steps.

[0057] 301. UV unfolding is performed on the target 3D white model to obtain UV space information of the target 3D white model. UV unfolding is used to unfold the surface of the 3D model onto a 2D plane. The UV space information is used to indicate the position and normal of the vertices on the target 3D white model in the UV space.

[0058] In an embodiment of the present application, a computer device may display an input page, on which a 3D white model and reference information are input to obtain a corresponding 3D model. The target 3D white model is the input 3D white model, i.e., a pure geometric model without textures. The reference information is an input text or image. The computer device may perform UV unwrapping on the target 3D white model, mapping each 3D vertex on the 3D model to a pixel on a 2D UV map, thereby unwrapping the 3D model surface onto a 2D plane.

[0059] Alternatively, the computer device may be a terminal that can run a useful application, and the user can input the 3D white model and reference information through a page in the application. Alternatively, the user can access a web page provided by a server through the terminal and input the 3D white model and reference information through the web page.

[0060] For example, Figure 4 Schematic diagram of a three-dimensional white mold provided according to an embodiment of the present application. Figure 4 As shown, the three-dimensional white mold is a white mold of a three-dimensional model of a hoodie.

[0061] It should be noted that 3D white models have a wide range of application scenarios in many fields. For example, in game development scenarios, 3D white models are used to build initial models of game scenes, characters, and props. In film and television special effects scenarios, 3D white models are used to provide basic models for special effects production. In architectural design scenarios, 3D white models are used to display the appearance and structure of buildings. In product design scenarios, 3D white models are used to help designers evaluate the shape and size of products in the early stages. In animation production scenarios, 3D white models are used as the basis for animated characters and scenes. In virtual reality scenarios, 3D white models are used to create the basic framework of virtual environments. In industrial design scenarios, 3D white models are used to design mechanical parts, equipment, etc. In education and training scenarios, 3D white models are used to help students understand the structure of complex objects. In artistic creation scenarios, artists can express more creatively based on 3D white models. In medical scenarios, 3D white models are used to simulate human structures or medical equipment. Therefore, the texture image generation method provided in the embodiments of the present application can be applied to various scenarios that require modeling 3D models using 3D white models to generate texture images that match the 3D white models.

[0062] UV unwrapping is a crucial step in the 3D modeling process. Its primary function is to unfold the surface of a 3D model into a flat 2D image. The following lists ten main uses of UV unwrapping and their respective importance. Use 1: Mapping: Ensures that texture images are accurately applied to the model surface. Use 2: Improving Texture Quality: Ensures that texture resolution and detail are optimally rendered on the model. Use 3: Saving Resources: Optimizes the use of texture space and avoids waste. Use 4: Model Export: Prepares the model for import into other software or engines. Use 5: Lighting Calculation: Facilitates the correct calculation and display of lighting effects. Use 6: Improving Efficiency: Reduces time spent on texture processing and adjustment. Use 7: Material Assignment: Facilitates the assignment of different materials to different parts. Use 8: Model Optimization: Helps reduce model complexity and file size. Use 9: Animation: Ensures that textures display correctly during animation. Use 10: Visual Effects: Enhances the model's realism and artistic quality. Among them, in the solution provided by this application, UV unfolding can enable the texture image generated in the subsequent steps to be accurately fitted to the model surface, while ensuring that the resolution and details of the texture are best presented on the model, and can also reduce the time for texture processing and adjustment, thereby improving the efficiency of three-dimensional modeling.

[0063] 302. The UV position map and the UV normal map are spliced ​​in the channel dimension to obtain UV geometric features.

[0064] In an embodiment of the present application, during the UV unfolding process, each vertex on the three-dimensional model is assigned a two-dimensional coordinate (U, V), which is used to correspond pixels on the texture image to points on the surface of the model. UV unfolding can obtain a UV position map and a UV normal map. Among them, the UV position map is used to indicate the position of the vertices on the target three-dimensional white model in the UV space. The vertex position (Position) represents the spatial coordinates (x, y, z) of each vertex in the three-dimensional model, which is used to determine the shape of the model in the scene. The UV coordinates represent the two-dimensional coordinates (U, V) of the vertex on the texture, which are used to determine the distribution of the texture on the model. The UV normal map is used to indicate the normal of the vertex on the target three-dimensional white model in the UV space. The main functions of the UV normal map (UV Normal) are to determine the distribution of the texture on the model, improve the texture quality, and enhance the realism and artistic effect of the model.

[0065] Optionally, the UV position value range is typically between 0 and 1.0. The UV coordinates corresponding to the lower left corner of the texture map are (0,0), and the coordinates corresponding to the upper right corner are (1,1). The rendering engine can normalize the position coordinates and normal vectors, and then represent the position and normal using color values ​​from 0-255. This allows the position attribute to be rasterized on the UV position map, and the normal attribute to be rasterized on the UV normal map, thus achieving a visual display.

[0066] Since both the UV position map and the UV normal map are three-channel images, by concatenating them in the channel dimension, we can obtain a six-channel feature. For ease of description, this is called a UV geometric feature. It should be noted that concatenating images in the channel dimension typically involves merging multiple images of the same size and resolution in the channel dimension to produce a new image with more channels.

[0067] For example, let's use the OpenCV library in Python to stitch images in the channel dimension. First, use the imread function to read two three-channel images. Then, use the merge function to stitch these two images in the channel dimension to obtain a six-channel image. Finally, use the imshow function to display the stitched image. The code example is as follows:

[0068] importcv2

[0069] #Read two three-channel images

[0070] image1=cv2.imread('image1.png')

[0071] image2=cv2.imread('image2.png')

[0072] #Splice into six-channel images

[0073] merged_image=cv2.merge((image1,image2))

[0074] # Display the stitched image

[0075] cv2.imshow('MergedImage',merged_image)

[0076] cv2.waitKey(0)

[0077] cv2.destroyAllWindows().

[0078] In other words, if the three channels of the UV position map are exemplarily represented as R1, G1, and B1, and the three channels of the UV normal map are exemplarily represented as R2, G2, and B2. When the UV position map is on top and the UV normal map is on the bottom, the six channels of the UV geometric features can be represented from top to bottom as R1, G1, B1, R2, G2, and B2. Similarly, when the UV normal map is on top and the UV position map is on the bottom, the six channels of the UV geometric features can be represented from top to bottom as R2, G2, B2, R1, G1, and B1.

[0079] 303. Perform feature extraction on the texture control information to obtain texture control features.

[0080] In the embodiment of the present application, since the texture control information is used to describe the style and color of the texture, the texture control feature can accordingly represent the style and color of the texture. For example, if the texture control information is "a blue hoodie with yellow sleeves", that is, a blue hoodie with yellow sleeves, then the texture control feature can be "blue", "yellow" and "hoodie". For example, Figure 5 Schematic diagram of texture control information provided according to an embodiment of the present application. Figure 5 As shown, the texture control information is an image including a blue hoodie with yellow sleeves.

[0081] In some embodiments, the texture control information can be text or an image. Accordingly, the feature extraction method varies depending on the texture control information. When the texture control information is text, feature extraction is performed on the text information to obtain a texture control feature. The texture control feature consists of two parts: a first part consisting of text features and a second part consisting of image features. When the texture control information consists only of text, the features extracted from the text are added to the first part, and then the second part is reset to zero, meaning all elements representing image features are filled with zeros. Alternatively, when the texture control information consists of an image, feature extraction is performed on the image to obtain a texture control feature. Similarly, when the texture control information consists only of an image, features extracted from the image are added to the second part, and then the first part is reset to zero, meaning all elements representing text features are filled with zeros. Alternatively, when the texture control information consists of both text and image, feature extraction is performed on both the text and image separately, and the extracted text and image features are concatenated to obtain a texture control feature. By extracting at least one of the text and image features, the texture control feature can be used as a control signal to control the denoising network to generate an image with these texture features, thereby improving the accuracy of the generated texture image.

[0082] Optionally, when the texture control information includes an image, the computer device extracts features of the image using an IP-Adapter. The IP-Adapter is an efficient and lightweight adapter for implementing image cue features in a pre-trained text-to-image diffusion model. The key design of the IP-Adapter is a decoupled cross-attention mechanism that separates the cross-attention layers for text features and image features.

[0083] 304. Input the UV geometric features, texture control features, and random noise image into a denoising network to obtain an intermediate texture image.

[0084] In this embodiment, the denoising network improves upon the control network (ControlNet) based on the Stable Diffusion (SD) model. The SD model stands for High-Resolution Image Synthesis with Latent Diffusion Models (Robin). The SD model is a powerful diffusion generative model trained using billions of image-text pairs and a large number of graphics cards.

[0085] The SD model is a text-to-image generative model based on latent diffusion models (LDMs). It can generate high-quality, high-resolution, and highly realistic images from arbitrary text input. LDMs are generative models widely used in fields such as image generation. Key features of LDMs include strong generative capabilities, flexibility, text-to-image generation, and low model complexity. Their strong generative capabilities refer to their ability to generate high-quality and diverse images. Their flexibility refers to their ability to process a wide range of data types. Their text-to-image generation refers to their ability to generate images based on text descriptions. Their low model complexity indicates their high computational efficiency compared to other models. LDMs generally operate by learning a latent representation of data in a latent space to capture semantic information. The diffusion process simulates the diffusion of information from sharp to blurred. Finally, back-diffusion is performed to gradually restore the blurred state to a sharp state, generating new data. LDMs are widely used in image generation to create realistic images, in data augmentation to increase data diversity, and in artistic creation to assist artists in their creative work.

[0086] The role of ControlNet is to guide Stable Diffusion to generate images according to the creative ideas of the creators by adding additional control conditions, thereby improving the controllability and accuracy of AI image generation. ControlNet is flexible in use, and can be applied as a single model or combined with multiple models.

[0087] In the improved manner of the control network in the embodiments of the present application, the encoding part of the SD large model is copied out, and the weight initial values of the control network and the encoding part of the SD large model are kept consistent. Then, a zero convolution module is added to the control network, and the geometric control signal, i.e., the UV geometric feature, is processed through the multi-layer convolution in the zero convolution module. In the training and inference, the weight of the original encoding part of the SD large model is locked, so that the copied part can generate corresponding features according to the geometric control signal, and the generated features are input into the original SD large model to achieve the purpose of control. Optionally, the computer device takes a random noise image of 4*64*64 as the initial input.

[0088] In some embodiments, the process of the denoising network for generating an intermediate texture image based on the input is introduced from the perspective of model structure. Correspondingly, the denoising network includes a first sub-network and a second sub-network, the first sub-network is a stable diffusion SD large model, and the second sub-network includes an encoding part of the SD large model and multiple zero convolution layers. The steps of the computer device for inputting the UV geometric feature, the texture control feature and the random noise image into the denoising network to obtain the intermediate texture image include: the computer device inputs the random noise image and the UV geometric feature into the second sub-network, and performs multiple denoising on the random noise image by the second sub-network to obtain multiple first intermediate features. And input the texture control feature and the random noise image into the first sub-network, and perform multiple denoising on the random noise image by the first sub-network to obtain multiple second intermediate features; finally, the first sub-network decodes the multiple first intermediate features and the second intermediate features to obtain the intermediate texture image. For any encoding layer in the second sub-network, the output of the encoding layer is the input of the decoding layer corresponding to the encoding layer in the first sub-network. By improving the control network, the UV geometric feature is input into the second sub-network, and the output of the second sub-network is taken as the input of the first sub-network, i.e., the encoding part of the SD large model, and then the intermediate texture image is generated. Compared with the control signal of three-channel depth, mask and the like before improvement, since the UV space contains all the geometric information of the three-dimensional object, the control can be directly performed on the UV space after improvement of the control network, thereby improving the accuracy of the control.

[0089] It should be noted that since the UV geometric feature is a six-channel feature obtained by concatenating the UV position map and the UV normal map in the channel dimension, the UV geometric feature is modified from three channels to six channels in the first convolutional layer of the second sub-network. Then, the convolution weights of the three channels at the bottom of the first convolutional layer of the second sub-network are initialized to zero. That is, the convolution weights of the three newly added channels (channel 4, channel 5, and channel 6) are initialized to zero, and the weights of the SD large model are used as the initial values ​​for the other convolutional channels and other convolutional layers to maintain generalization.

[0090] In some embodiments, the process of generating an intermediate texture image based on the input of a denoising network is described from the perspective of data flow. Accordingly, the denoising network uses a random noise image as the initial input and generates the intermediate texture image through multiple iterations. For ease of description, the number of times the denoising network denoises the random noise image is N, where N is a positive integer. Typically, the number of denoising cycles can be set to 20-50. A computer device inputs UV geometric features, texture control features, and a random noise image into the denoising network to generate an intermediate texture image. The steps include: for the first denoising cycle, the computer device processes the UV geometric features, texture control features, and random noise image through the denoising network to obtain a first noise image. For the i-th denoising cycle, the computer device processes the UV geometric features, texture control features, and second noise image through the denoising network to obtain a third noise image, where the second noise image is the noise image obtained after the i-1th denoising cycle. For example, in the second denoising cycle, the second noise image is the noise image obtained in the first denoising cycle, i.e., the first noise image described above. In the third denoising cycle, the second noise image is the noise image obtained in the second denoising cycle. The same logic applies and will not be repeated here. For the Nth denoising process, the computer device processes the UV geometric features, texture control features, and the fourth noise image based on the denoising network to obtain an intermediate texture image. The fourth noise image is the noise image obtained after the N-1th denoising process. By performing multiple denoising operations on the random noise image, the resulting intermediate texture image is more consistent with the geometric information in the UV geometric features and the texture information in the texture control features, thereby improving the accuracy of the generated texture image.

[0091] It should be noted that, in order to make the solution provided by the embodiment of the present application easier to understand, the denoising network is further explained below.

[0092] First, let's introduce the principles of the SD model. Stable Diffusion uses a novel text encoder (OpenCLIP) to convert text input into a vector representation. This vector representation captures the semantic information of the text and aligns with the image space. Stable Diffusion uses a diffusion model to gradually transform a random noise image into a target image. A diffusion model is a generative model that learns a probability distribution from training data and samples new data from it. During the diffusion process, Stable Diffusion uses the text vector and the noise image as conditional inputs to provide a probability distribution for each transformation step. In this way, Stable Diffusion can guide the noise image to converge towards the target image based on the text while maintaining image clarity and coherence. Finally, Stable Diffusion uses a super-resolution upscaler (upscaler) to upscale the generated low-resolution image to a higher resolution. Super-resolution upscaling is also a diffusion model that can recover details from low-resolution images and enhance image quality. Stable Diffusion can realize the generation from text to image, and has the following advantages: Stable Diffusion can process text input of any field and topic, and generate diverse and creative images that are consistent with it. Stable Diffusion can generate images with a resolution of up to 2048×2048 or higher, and maintain good visual effects and realism. Stable Diffusion can also perform depth-guided and structure-preserving image conversion and synthesis. It should be noted that in addition to the field of image generation, the Stable Diffusion model is also widely used in natural language processing, audio and video and other generation fields. Accordingly, the solution provided in the embodiment of the present application can also be applied to natural language processing, audio and video and other generation fields.

[0093] Figure 6 This is a schematic diagram of the principle of a diffusion generation model provided according to an embodiment of the present application. Figure 6 As shown in the figure, the random noise image x3 is denoised multiple times to obtain the image x0. Each denoising process is processed once using the DiffusionUNet network. θ represents the probability distribution of the reverse diffusion process, and q(x2|x1) represents the result of getting x2 from x1. During the model training process, the following formula (1) can be used to obtain x0 by superimposing noise of different degrees with the original image x0 as input. tThen the model is tried to denoise, that is, to recover the noise, as shown in the following formula (2).

[0094]

[0095] where q(x t |x0) represents obtaining x t from x0. represents a normal distribution. α t represents the parameter of the t-th step, which is a floating point number. I represents a unit matrix.

[0096]

[0097] where L γ (∈ θ ) represents the training loss. γ t represents the parameter in the training process. ∈ t represents the noise true value. represents the UNet network.

[0098] Then the network structure of the denoising network provided in the present application is introduced. Figure 7 is a structural schematic diagram of a denoising network according to an embodiment of the present application. Referring to Figure 7 . Figure 7 The left side is the original structure of the SD model, that is, the first sub-network. Figure 7 The part in the right dashed box is the newly added model structure, that is, the second sub-network. As Figure 7 can be seen, the input of the decoding layer in the first sub-network has a part of the output of the second sub-network. Figure 7 in the figure represents c f . The texture control feature, that is, the prompt in the figure, is represented as c t . The random noise image, that is, the input in the figure, is represented as z t .

[0099] 305、The intermediate texture image is up-sampled to obtain a UV texture image of the target three-dimensional white model, and the pixel points in the UV texture image correspond to the vertices on the target three-dimensional white model.

[0100] In an embodiment of the present application, a computer device can obtain a 3*512*512 generated UV texture image by upsampling and decoding the pre-trained VAE-Decoder. Among them, VAE (variational auto-encoder) refers to a variational autoencoder, which is often used to generate data and is one of the three common generative models. VAE can model the distribution of real data from the training data, and then use the learned model and distribution to generate and construct new data. Optionally, upsampling can also be performed in other ways, which is not limited by the embodiment of the present application.

[0101] The VAE-Decoder is the decoder of a variational autoencoder. The decoder samples latent vectors from the latent space, then learns to identify these sampled points and decodes them into a dataset (usually an image). A variational autoencoder is a neural network model that combines the concepts of autoencoders and probabilistic graphical models. The primary goal of a variational autoencoder is to learn a low-dimensional latent variable representation that reconstructs the input data well and allows for meaningful interpolation and generation of new data in the latent space.

[0102] Upsampling is an image processing technique used to increase the resolution or size of an image. The main functions of upsampling include improving resolution, image magnification, and filling in missing information. Among them, improving resolution refers to converting a low-resolution image into a high-resolution image. Image magnification refers to making the image larger in size. Filling in missing information refers to supplementing the missing parts of the image in certain applications. In the embodiment of the present application, the intermediate texture image is mainly amplified by upsampling. Common upsampling methods include nearest neighbor interpolation, bilinear interpolation, and other more complex interpolation methods, such as bicubic interpolation. Among them, nearest neighbor interpolation is a method of interpolating based on the closest pixel value. Bilinear interpolation is a method of calculating the value of a new pixel by linear interpolation of surrounding pixels.

[0103] In some embodiments, in order to save storage space, the position information corresponding to the pixel points in the image can be scaled. Accordingly, the computer device scales the coordinate values ​​corresponding to the pixel points in the UV texture image to the target range. For any pixel point, when the coordinate value of the pixel point after scaling is greater than 0, the computer device fits the pixel value of the pixel point to the vertex corresponding to the pixel point in the target three-dimensional white model. Optionally, the target range is 0.5-1, that is, only when the coordinate value of the pixel point is greater than 0 (the part that is not black in the UV texture image), the pixel point will be applied to the three-dimensional model, and the other parts will be ignored.

[0104] For example, using 3D software, first extract the UV Position value. Then, determine if the UVPosition value is greater than 0. For any portion greater than 0, scale it to the range of 0.5 to 1. In the model's material or texture settings, only the UV texture map for the portion greater than 0 after scaling is applied. This approach offers the flexibility of controlling the texture's range on the model, enabling more precise and specific effects. For example, different textures can be applied to different parts of the model or under specific conditions. Alternatively, the UV Position value can be scaled to the range of 0.5 to 1 using the following steps: Step 1: Set uv- = float2(0.5, 0.5): Move the center point of the UV to the center. Step 2: Add half of the UV back to the original value: UV+ = float2(0.5, 0.5). It should be noted that if the UVPosition value falls outside the range of 0.5 to 1 during the scaling operation, you can adjust the scaling operation by checking the scaling algorithm or parameters to ensure that the value is correctly constrained to the required range. Alternatively, you can perform data filtering to ignore values ​​outside the range and process only valid data. The embodiments of the present application are not limited to this.

[0105] It should be noted that in order to make the texture image generation scheme provided in this application easier to understand, an example is given. Figure 8 , Figure 8 FIG. 1 is a flowchart of another method for generating a texture image according to an embodiment of the present application. Figure 8 As shown, random noise noise(T) serves as the latent feature Latent_T, also known as a random noise image. The attributes of the random noise image Latent_T are [b, 4, 64, 64]. b represents the batch to which Latent_T belongs, and the image size is 4*64*64. The denoising network inputs not only the random noise image but also the control map features extracted by IP_Adapter, also known as texture control features, and UV geometric features. After the first denoising pass, the denoising network outputs Latent_t-1, which is then used as the input for the next denoising pass. This cycle repeats until the number t returns to zero. Finally, by upsampling Latent_0, the UV texture image is obtained, namely UV_Texture = VAE_Dec(Latent_0).

[0106] It should be noted that the texture image generation scheme provided in the embodiments of this application can be widely applied to various scenarios requiring 3D model creation. By modifying the texture control information, various 3D models with the same shape but different textures can be generated, while maintaining the same 3D white model. Similarly, by replacing the 3D white model, 3D models with the same texture but different shapes can be generated, thus enabling the creation of a variety of unique 3D models. This increases the innovation and diversity of designs and allows for the rapid generation of multiple 3D models to help designers validate their ideas. This scheme can be applied in architectural design scenarios to generate models of various building structures and exteriors; industrial design scenarios to create diverse product shapes and styles; game development scenarios to quickly and easily build diverse game characters, scenes, and props; film and television special effects scenarios to create unique virtual objects and scenes; artistic creation scenarios to achieve 3D presentation of creative imagination; virtual reality scenarios to provide a rich set of model resources for virtual worlds; product display scenarios to display 3D product models in a variety of ways; educational scenarios to help students understand geometric shapes and design concepts; scientific visualization scenarios to present 3D representations of complex scientific concepts; and advertising and marketing scenarios to create engaging 3D advertising elements. Interior design scenes are used to generate models of various interior layouts and decorations. Digital art exhibition scenes are used to display diverse digital artworks.

[0107] The following describes an example of the application of the solution provided in the embodiment of the present application in a game development scenario.

[0108] The 3D modeling process in a game development scenario usually includes the following steps (1)-(11).

[0109] (1) Conceptual design.

[0110] The concept design step usually determines the theme, style, and atmosphere of the game environment.

[0111] In the solution provided in the embodiments of this application, a first interactive page can be provided, which is used to provide multiple theme options, multiple style options, and multiple atmosphere options. The multiple theme options include fantasy, science fiction, mythology, war, and medieval. The multiple style options include cartoon, ink painting, Japanese style, cyberpunk, and dark. The multiple atmosphere options include mystery, horror, romance, and sadness. Designers can use this first interactive page to quickly complete the concept design process and set the tone for the game scene.

[0112] (2) Collect reference materials.

[0113] In the solution provided in the embodiment of the present application, a second interactive page can also be provided, which is used to display reference materials that match the theme, style, and atmosphere selected in the first interactive page. The reference materials include pictures, videos, etc., which can be used as references for modeling.

[0114] (3) Establish a basic model.

[0115] Users can use 3D modeling software to create basic geometric bodies in the scene, such as cubes, cylinders, etc. to create basic models.

[0116] (4) Refine the model.

[0117] It should be noted that after creating the basic model in step (3), the user can add details and adjust the basic model to make it more realistic and artistic. Alternatively, the user can directly select an existing model or modify an existing model. Accordingly, in the solution provided in the embodiment of the present application, a third interactive page can also be provided, which is used to select a 3D white model from the model library or modify an existing 3D white model to create a new 3D white model.

[0118] (5) Construct scene layout.

[0119] Users can place the created 3D white models into the virtual scene to determine their position and scale in the scene.

[0120] (6) Create texture.

[0121] In this step, a suitable texture image can be generated for the three-dimensional white model by the texture image generation method provided in the embodiments of the present application to increase the sense of reality. Correspondingly, in the scheme provided in the embodiments of the present application, a fourth interactive page can also be provided, which is used to select a three-dimensional white model to be processed, that is, a game object white model for which a texture is to be created. The fourth interactive page is also used to input texture control information, which is used to describe the style and color of the texture and the like. Optionally, the fourth interactive page also displays a plurality of style options and a color palette for selecting different styles and colors. After the user submits the game object white model and the texture control information, first, the game object white model is subjected to UV unfolding, and the UV spatial information of the game object white model can be obtained. As described in step 301 above, details are not repeated here. Then, the UV position map and the UV normal map in the UV spatial information are spliced in the channel dimension to obtain the UV geometric features of the game object white model. As described in step 302 above, details are not repeated here. Then, the texture control features of the game object white model are obtained by feature extraction on the texture control information. Then, the UV geometric features, the texture control features and the random noise image are processed by the denoising network to obtain an intermediate texture image. As described in step 304 above, details are not repeated here. Finally, the intermediate texture image is up-sampled according to the size of the game object white model, and a matching texture image can be obtained. As described in step 305 above, details are not repeated here. The scheme generates a two-dimensional texture image of a game object model in an end-to-end manner, and has high generation efficiency. Moreover, for the same game object white model, only the texture control information needs to be changed, and different texture images of the game object white model can be generated, thereby obtaining a variety of game object models, greatly improving the diversity of the texture image and reducing the production cost of the game object model.

[0122] (7) Set material.

[0123] After the texture image is generated by the scheme provided in the embodiments of the present application, the properties of the texture in the texture image, such as color, reflectivity and the like, can be adjusted through the fifth interactive page.

[0124] (8) Add light.

[0125] After the texture is adjusted, the atmosphere and light and shadow effect of the scene can be created by setting the light.

[0126] (9) Optimize model.

[0127] After the texture image is fitted to the game object white model, a game object model is obtained, and then the game object can be optimized to reduce the number of faces and file size of the model and improve the game performance.

[0128] (10) Test and adjust.

[0129] View the scene in the game engine and make adjustments as needed.

[0130] (11) Export the model.

[0131] Finally, export the modeling results to a format supported by the game engine.

[0132] The following describes an example of an application of the solution provided in an embodiment of the present application in a product display scenario.

[0133] In product display scenarios, three-dimensional models serve multiple purposes, such as realistically presenting product forms, displaying products from multiple angles, displaying them in virtual environments, and reducing costs. Taking backpacks as an example, the same type of backpack can often have different color schemes, and users often want to customize their color schemes to meet their personalized needs. Accordingly, the solution provided in the embodiments of this application can provide a sixth interactive page for selecting the backpack style, namely the three-dimensional backpack white film. Once the style is determined, the solution provided in the embodiments of this application can provide a seventh interactive page for entering texture control information. After the user submits the three-dimensional backpack white film and texture control information, the solution provided in the embodiments of this application can generate a three-dimensional backpack model that meets the user's expectations and satisfies the texture control information entered by the user. The implementation method is similar to steps 301-305 above and will not be further described here. Through the above method, a three-dimensional backpack model that meets the user's needs can be generated in real time based on the user's needs, thereby meeting the user's personalized needs, improving human-computer interaction efficiency, and enhancing the user experience. Accordingly, manufacturers no longer need to design a large number of three-dimensional backpack models in advance, significantly saving labor costs.

[0134] The embodiment of the present application provides a method for generating a texture image, which introduces UV space information and texture control information. Since UV space information contains all the geometric information of a three-dimensional object, the texture control information can be used to directly control the generation result of the texture image in UV space. Compared with the existing solution of rendering images from different perspectives and then fusing them, this method solves the shortcomings of occlusion and multi-perspective inconsistency, and improves the accuracy of the generated texture image. Moreover, using UV space information as control information allows the generated UV texture image to learn geometric prior knowledge, such as symmetry, regional similarity, and regularity, so that it is more in line with human vision. In addition, the method uses an end-to-end approach to generate a two-dimensional texture image of a three-dimensional model, with high generation efficiency. Moreover, for the same three-dimensional white model, different texture images of the three-dimensional white model can be generated by simply changing the texture control information, thereby obtaining a rich and diverse three-dimensional model, greatly improving the diversity of texture images and reducing the production cost of the three-dimensional model.

[0135] Figure 9This is a block diagram of a texture image generation device provided according to an embodiment of the present application. Figure 9 The device includes: a two-dimensional information acquisition module 901, a denoising module 902 and an upsampling module 903.

[0136] The 2D information acquisition module 901 is used to perform UV unfolding on the target 3D white model to obtain UV spatial information of the target 3D white model. UV unfolding is used to unfold the 3D model surface onto a 2D plane. The UV spatial information is used to indicate the position and normal of the vertices on the target 3D white model in UV space.

[0137] Denoising module 902 is used to perform multiple denoising operations on the random noise image based on a denoising network using UV spatial information and texture control information as reference information to obtain an intermediate texture image. The texture control information is used to describe the style and color of the texture, and the denoising network is used to generate an image that meets the reference information.

[0138] The upsampling module 903 is configured to upsample the intermediate texture image to obtain a UV texture image of the target 3D white model, where pixels in the UV texture image correspond to vertices on the target 3D white model.

[0139] In some embodiments, the UV space information includes a UV position map and a UV normal map, wherein the UV position map is used to indicate positions of vertices on the target 3D white model in the UV space, and the UV normal map is used to indicate normal directions of vertices on the target 3D white model in the UV space.

[0140] The denoising module 902 includes:

[0141] The first feature extraction unit is used to splice the UV position map and the UV normal map in the channel dimension to obtain UV geometric features;

[0142] A second feature extraction unit is used to extract features from the texture control information to obtain texture control features;

[0143] The denoising unit is used to input UV geometric features, texture control features and random noise images into the denoising network to obtain an intermediate texture image.

[0144] In some embodiments, the denoising network includes a first subnetwork and a second subnetwork, the first subnetwork is a stable diffusion SD large model, and the second subnetwork includes an encoding part of the SD large model and a plurality of zero convolutional layers;

[0145] The denoising unit is used to input the random noise image and UV geometric features into the second subnetwork, and the second subnetwork performs multiple denoising on the random noise image to obtain multiple first intermediate features; input the texture control features and the random noise image into the first subnetwork, and the first subnetwork performs multiple denoising on the random noise image to obtain multiple second intermediate features; decode the multiple first intermediate features and second intermediate features through the first subnetwork to obtain an intermediate texture image; wherein, for any encoding layer in the second subnetwork, the first intermediate feature output by the encoding layer is the input of the decoding layer corresponding to the encoding layer in the first subnetwork.

[0146] In some embodiments, the denoising unit is further configured to initialize the convolution weights of the three bottom channels in the first convolutional layer of the second sub-network to zero.

[0147] In some embodiments, the denoising network performs multiple denoising on the random noise image N times, where N is a positive integer; the denoising unit is used to process the UV geometric features, texture control features and random noise image through the denoising network for the first denoising process to obtain a first noise image; for the i-th denoising process, the denoising network processes the UV geometric features, texture control features and the second noise image to obtain a third noise image, where the second noise image is the noise image obtained after the i-1-th denoising process, where i is a positive integer greater than 1 and less than N; for the N-th denoising process, the denoising network processes the UV geometric features, texture control features and the fourth noise image to obtain an intermediate texture image, where the fourth noise image is the noise image obtained after the N-1-th denoising process.

[0148] In some instances, the second feature extraction unit is used to perform feature extraction on text information to obtain texture control features when the texture control information is text; or to perform feature extraction on the image to obtain texture control features when the texture control information is an image; or to perform feature extraction on the text and the image separately when the texture control information includes text and images, and splice the extracted text features and image features to obtain texture control features.

[0149] In some embodiments, the apparatus further comprises:

[0150] The image processing module is used to scale the coordinate values ​​corresponding to the pixel points in the UV texture image to the target range; for any pixel point, when the coordinate value of the pixel point after scaling is greater than 0, the pixel point is mapped to the vertex corresponding to the pixel point in the target 3D white model.

[0151] The embodiment of the present application provides a texture image generation device, by introducing UV space information and texture control information, since the UV space information contains all geometric information of a three-dimensional object, the generation result of the texture image can be directly controlled on the UV space through the texture control information, so that for the same three-dimensional white model, only the texture control information needs to be changed, and different texture images of the three-dimensional white model can be generated, the generation efficiency and diversity of the texture image are greatly improved. Compared with the existing scheme of fusing images of different perspectives by rendering, the shortcomings of occlusion, multi-perspective inconsistency and the like are solved, and the accuracy of the generated texture image is improved.

[0152] In the embodiment of the present application, the computer device can be a terminal or a server, when the computer device is a terminal, the terminal is taken as an execution subject to implement the technical solution provided by the embodiment of the present application; when the computer device is a server, the server is taken as an execution subject to implement the technical solution provided by the embodiment of the present application; or, the technical solution provided by the present application is implemented through the interaction between the terminal and the server, and the embodiment of the present application does not limit this.

[0153] Figure 10 is a structural block diagram of a terminal 1000 according to the embodiment of the present application.

[0154] Generally, the terminal 1000 includes a processor 1001 and a memory 1002.

[0155] The processor 1001 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1001 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 1001 can also include a main processor and a coprocessor, the main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1001 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 1001 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0156] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one computer program, which is executed by the processor 1001 to implement the texture image generation method provided in the method embodiment of the present application.

[0157] In some embodiments, terminal 1000 may optionally include a peripheral device interface 1003 and at least one peripheral device. Processor 1001, memory 1002, and peripheral device interface 1003 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1003 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 1004, a display screen 1005, a camera assembly 1006, an audio circuit 1007, and a power supply 1008.

[0158] The peripheral device interface 1003 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 1001 and the memory 1002. In some embodiments, the processor 1001, the memory 1002, and the peripheral device interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1001, the memory 1002, and the peripheral device interface 1003 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0159] The RF circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1004 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. In some embodiments, the RF circuit 1004 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 1004 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1004 may also include circuitry related to Near Field Communication (NFC), although this application does not limit this.

[0160] The display screen 1005 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1005 is a touch screen display, the display screen 1005 also has the ability to collect touch signals on the surface or above the surface of the display screen 1005. The touch signal can be input as a control signal to the processor 1001 for processing. At this time, the display screen 1005 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen 1005, which is set on the front panel of the terminal 1000; in other embodiments, there can be at least two display screens 1005, which are respectively set on different surfaces of the terminal 1000 or in a folding design; in other embodiments, the display screen 1005 can be a flexible display screen, which is set on the curved surface or folding surface of the terminal 1000. Even more, the display screen 1005 can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0161] The camera assembly 1006 is used to capture images or videos. In some embodiments, the camera assembly 1006 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0162] The audio circuit 1007 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 1001 for processing, or input into the RF circuit 1004 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the terminal 1000. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 1001 or the RF circuit 1004 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1007 may also include a headphone jack.

[0163] Power supply 1008 is used to power various components in terminal 1000. Power supply 1008 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1008 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is charged via a wired line, while a wireless rechargeable battery is charged via a wireless coil. The rechargeable battery can also support fast charging technology.

[0164] In some embodiments, the terminal 1000 further includes one or more sensors 1009 . The one or more sensors 1009 include, but are not limited to, an acceleration sensor 1010 , a gyroscope sensor 1011 , a pressure sensor 1012 , an optical sensor 1013 , and a proximity sensor 1014 .

[0165] The accelerometer 1010 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the terminal 1000. For example, the accelerometer 1010 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 1001 can control the display screen 1005 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 1010. The accelerometer 1010 can also be used to collect game or user motion data.

[0166] The gyroscope sensor 1011 can detect the body orientation and rotation angle of the terminal 1000. The gyroscope sensor 1011 can cooperate with the acceleration sensor 1010 to collect the user's 3D movements on the terminal 1000. Based on the data collected by the gyroscope sensor 1011, the processor 1011 can implement the following functions: motion sensing (such as changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0167] The pressure sensor 1012 can be provided on the side frame of the terminal 1000 and / or below the display screen 1005. When the pressure sensor 1012 is provided on the side frame of the terminal 1000, it can detect the user's gripping signal of the terminal 1000, and the processor 1001 can perform left-hand or right-hand recognition or shortcut operations based on the gripping signal collected by the pressure sensor 1012. When the pressure sensor 1012 is provided below the display screen 1005, the processor 1001 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 1005. Operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0168] Optical sensor 1013 is used to detect ambient light intensity. In one embodiment, processor 1001 can control the display brightness of display screen 1005 based on the ambient light intensity detected by optical sensor 1013. Specifically, when the ambient light intensity is high, the display brightness of display screen 1005 is increased; when the ambient light intensity is low, the display brightness of display screen 1005 is decreased. In another embodiment, processor 1001 can also dynamically adjust the shooting parameters of camera assembly 1006 based on the ambient light intensity detected by optical sensor 1013.

[0169] Proximity sensor 1014, also known as a distance sensor, is typically located on the front panel of terminal 1000. Proximity sensor 1014 is used to detect the distance between the user and the front of terminal 1000. In one embodiment, when proximity sensor 1014 detects that the distance between the user and the front of terminal 1000 is gradually decreasing, processor 1001 controls display screen 1005 to switch from the screen-on state to the screen-off state. When proximity sensor 1004 detects that the distance between the user and the front of terminal 1000 is gradually increasing, processor 1001 controls display screen 1005 to switch from the screen-off state to the screen-on state.

[0170] Those skilled in the art will understand that Figure 10 The structure shown in the figure does not constitute a limitation on the terminal 1000, and the terminal 1000 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0171] Figure 11 This is a structural diagram of a server provided in accordance with an embodiment of the present application. The server 1100 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1101 and one or more memories 1102, wherein the memory 1102 stores at least one computer program, and the at least one computer program is loaded and executed by the processor 1101 to implement the generation method of model interpretation information provided by the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0172] An embodiment of the present application further provides a computer-readable storage medium, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the texture image generation method of any of the above implementations.

[0173] An embodiment of the present application also provides a computer program product, which includes at least one program segment, and the at least one program segment is stored in a computer-readable storage medium. The processor of the computer device reads the at least one program segment from the computer-readable storage medium, and the processor executes the at least one program segment, so that the computer device executes the texture image generation method of any of the above-mentioned implementation methods.

[0174] All the optional technical solutions can be combined to form optional embodiments of the application, and will not be repeated here. The above are only optional embodiments of the application, and are not used to limit the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for generating a texture image, characterized in that: The method comprises: Performing UV unfolding on the target 3D white model to obtain UV space information of the target 3D white model, wherein the UV unfolding is used to unfold the surface of the 3D model onto a 2D plane, and the UV space information is used to indicate the position and normal of the vertices on the target 3D white model in the UV space; Using the UV spatial information and the texture control information as reference information, the random noise image is denoised multiple times based on a denoising network to obtain an intermediate texture image, wherein the texture control information is used to describe the style and color of the texture, and the denoising network is used to generate an image that meets the reference information; The intermediate texture image is up-sampled to obtain a UV texture image of the target three-dimensional white model, where pixels in the UV texture image correspond to vertices on the target three-dimensional white model.

2. The method according to claim 1, characterized in that The UV space information includes a UV position map and a UV normal map, wherein the UV position map is used to indicate positions of vertices on the target 3D white model in the UV space, and the UV normal map is used to indicate normal directions of vertices on the target 3D white model in the UV space; The method of performing multiple denoising on the random noise image based on a denoising network using the UV space information and the texture control information as reference information to obtain an intermediate texture image includes: Splicing the UV position map and the UV normal map in the channel dimension to obtain UV geometric features; Performing feature extraction on the texture control information to obtain texture control features; The UV geometric features, the texture control features, and the random noise image are input into the denoising network to obtain the intermediate texture image.

3. The method according to claim 2, characterized in that The denoising network includes a first subnetwork and a second subnetwork, the first subnetwork is a stable diffusion SD large model, and the second subnetwork includes the encoding part of the SD large model and multiple zero convolution layers; The step of inputting the UV geometric features, the texture control features, and the random noise image into the denoising network to obtain the intermediate texture image comprises: Inputting the random noise image and the UV geometric features into the second sub-network, and performing multiple denoising operations on the random noise image by the second sub-network to obtain multiple first intermediate features; Inputting the texture control feature and the random noise image into the first sub-network, and performing multiple denoising operations on the random noise image by the first sub-network to obtain multiple second intermediate features; Decoding the plurality of first intermediate features and the second intermediate features through the first sub-network to obtain the intermediate texture image; Among them, for any encoding layer in the second sub-network, the first intermediate feature of the output of the encoding layer is the input of the decoding layer corresponding to the encoding layer in the first sub-network.

4. The method according to claim 3, characterized in that The method further comprises: Initialize the convolution weights of the three bottom channels in the first convolutional layer of the second sub-network to zero.

5. The method according to claim 2, characterized in that The denoising network performs multiple denoising operations on the random noise image for a number of times N, where N is a positive integer; The step of inputting the UV geometric features, the texture control features, and the random noise image into the denoising network to obtain the intermediate texture image comprises: For the first denoising process, the UV geometric features, the texture control features, and the random noise image are processed by the denoising network to obtain a first noise image; For the i-th denoising process, the UV geometric features, the texture control features, and the second noise image are processed by the denoising network to obtain a third noise image, where the second noise image is the noise image obtained after the i-1-th denoising process, where i is a positive integer greater than 1 and less than N; For the Nth denoising process, the UV geometric features, the texture control features and the fourth noise image are processed by the denoising network to obtain the intermediate texture image, and the fourth noise image is the noise image obtained after the N-1th denoising process is completed.

6. The method according to claim 2, characterized in that The extracting features of the texture control information to obtain texture control features includes: In the case where the texture control information is text, performing feature extraction on the text information to obtain the texture control feature; or In the case where the texture control information is an image, performing feature extraction on the image to obtain the texture control feature; or In the case that the texture control information includes text and image, feature extraction is performed on the text and image respectively, and the extracted text features and image features are spliced ​​to obtain the texture control features.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Scaling the coordinate values ​​of the pixel points in the UV texture image to a target range; For any pixel point, when the coordinate value of the pixel point after scaling is greater than 0, the pixel point is mapped to the vertex corresponding to the pixel point in the target three-dimensional white model.

8. A texture image generation device, characterized in that: The device comprises: a two-dimensional information acquisition module, configured to perform UV unfolding on a target three-dimensional white model to obtain UV spatial information of the target three-dimensional white model, wherein the UV unfolding is used to unfold the surface of the three-dimensional model onto a two-dimensional plane, and the UV spatial information is used to indicate the position and normal of the vertices on the target three-dimensional white model in UV space; a denoising module, configured to perform multiple denoising operations on the random noise image based on a denoising network using the UV spatial information and the texture control information as reference information to obtain an intermediate texture image, wherein the texture control information is used to describe the style and color of the texture, and the denoising network is used to generate an image that satisfies the reference information; An upsampling module is configured to upsample the intermediate texture image to obtain a UV texture image of the target three-dimensional white model, wherein the pixels in the UV texture image correspond to vertices on the target three-dimensional white model.

9. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executes the texture image generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store at least one computer program, and the at least one computer program is used to execute the texture image generation method according to any one of claims 1 to 7.

11. A computer program product, comprising at least one program segment, wherein the at least one program segment is stored in a computer-readable storage medium, wherein a processor of a computer device reads the at least one program segment from the computer-readable storage medium, and the processor executes the at least one program segment, so that the computer device executes the texture image generation method according to any one of claims 1 to 7.

Images