Texture generation method and apparatus for three-dimensional model, and generation method and apparatus for three-dimensional model

Abstract

The embodiments of the present disclosure relate to a texture generation method and apparatus for a three-dimensional model, and a generation method and apparatus for a three-dimensional model. The texture generation method for a three-dimensional model comprises: acquiring three-dimensional model representation data of a texture to be generated; generating a plurality of local surface maps of a target three-dimensional model on the basis of the three-dimensional model representation data; acquiring features corresponding to the local surface maps, so as to generate, on the basis of the features, texture maps corresponding to the local surface maps; and obtaining a surface texture of the three-dimensional model on the basis of the texture maps respectively corresponding to the plurality of local surface maps.

Description

Three-dimensional model texture generation method, three-dimensional model generation method and device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 31, 2023, with application number 202310952854.9 and invention name “Texture generation method for three-dimensional model, method and device for generating three-dimensional model”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present disclosure relates to the field of computer technology, and in particular to a texture generation method for a three-dimensional model, and a generation method and device for a three-dimensional model. Background Art

[0004] In virtual scenes such as VR (Virtual Reality), three-dimensional models are often involved, and the presentation effect of the three-dimensional models is crucial.

[0005] Summary of the Invention

[0006] The present disclosure provides a texture generation method for a three-dimensional model, a generation method and a device for a three-dimensional model.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for generating texture of a three-dimensional model, comprising: obtaining three-dimensional model representation data of a texture to be generated; generating multiple surface local areas of a target three-dimensional model based on the three-dimensional model representation data; obtaining features corresponding to the surface local areas to generate a texture map corresponding to the surface local areas based on the features; and obtaining the surface texture of the target three-dimensional model based on the texture maps corresponding to each of the multiple surface local areas.

[0008] In a second aspect, an embodiment of the present disclosure also provides a method for generating a three-dimensional model, comprising: generating three-dimensional model representation data of a texture to be generated based on received model information; obtaining a surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained using the texture generation method for a three-dimensional model provided in the first aspect; and generating a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

[0009] In a third aspect, an embodiment of the present disclosure also provides a texture generation device for a three-dimensional model, comprising: a model data acquisition module for acquiring three-dimensional model representation data of a texture to be generated; a local area generation module for generating multiple surface local areas of a target three-dimensional model based on the three-dimensional model representation data; a texture map generation module for acquiring features corresponding to the surface local areas, so as to generate a texture map corresponding to the surface local areas based on the features; and a surface texture generation module for obtaining the surface texture of the target three-dimensional model based on the texture maps corresponding to each of the multiple surface local areas.

[0010] In a fourth aspect, an embodiment of the present disclosure also provides a device for generating a three-dimensional model, comprising: a model data generation module for generating three-dimensional model representation data of a texture to be generated based on received model information; a surface texture acquisition module for acquiring a surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained using the texture generation method for the three-dimensional model described in the first aspect; and a target model generation module for generating a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

[0011] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor for reading the executable instructions from the memory and executing the instructions to implement a texture generation method for a three-dimensional model or a three-dimensional model generation method as provided in an embodiment of the present disclosure.

[0012] In a sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the texture generation method of a three-dimensional model or the generation method of a three-dimensional model as provided in an embodiment of the present disclosure.

[0013] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0015] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] FIG1 is a schematic diagram of a model texture provided by an embodiment of the present disclosure;

[0017] FIG2 is a schematic diagram of a flow chart of a texture generation method for a three-dimensional model provided by an embodiment of the present disclosure;

[0018] FIG3 is a schematic diagram of a flow chart of a method for generating a three-dimensional model provided by an embodiment of the present disclosure;

[0019] FIG4 is a schematic diagram of the structure of a three-dimensional model generation network provided by an embodiment of the present disclosure;

[0020] FIG5 is a schematic diagram of the structure of a three-dimensional model generation network provided by an embodiment of the present disclosure;

[0021] FIG6 is a schematic structural diagram of a texture generating device for a three-dimensional model provided by an embodiment of the present disclosure;

[0022] FIG7 is a schematic structural diagram of a three-dimensional model generation device provided by an embodiment of the present disclosure;

[0023] FIG8 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0026] After research, the inventors discovered that the texture generation method adopted by the related technology in the process of generating a three-dimensional model is not good. Specifically, the related technology needs to process each point on the surface of the three-dimensional model separately to determine the color corresponding to the point, so as to finally obtain the surface texture of the three-dimensional model. However, the texture effect obtained in this way is not good, and it is easy to have problems such as color stratification. For example, referring to a model texture schematic diagram shown in Figure 1, the texture of the vehicle in Figure 1 has obvious stratification and unevenness. Therefore, the color rationality and local continuity of the surface texture of the three-dimensional model generated by the related technology are not good. In order to improve at least one of the above problems, the embodiments of the present disclosure provide a texture generation method for a three-dimensional model, a generation method and a device for a three-dimensional model, which are explained in detail below.

[0027] The texture generation method of the three-dimensional model provided by the embodiment of the present disclosure can generate multiple surface local areas based on the three-dimensional model representation data of the texture to be generated, and obtain the features corresponding to the surface local areas, so as to generate the texture map corresponding to the surface local areas based on the features, and finally obtain the surface texture of the three-dimensional model based on the texture maps corresponding to each of the multiple surface local areas. The above-mentioned method of directly using the features of the multiple surface local areas of the three-dimensional model to generate the corresponding texture map is not only more efficient, but also can better ensure the local continuity of the surface texture of the three-dimensional model, and effectively ensure the texture generation effect of the three-dimensional model. Furthermore, the above-mentioned three-dimensional model generation method provided by the embodiment of the present disclosure first generates the three-dimensional model representation data of the texture to be generated, and uses the above-mentioned three-dimensional model texture generation method to generate a surface texture with better effect, thereby ensuring the presentation effect of the target three-dimensional model finally obtained, which can better ensure the user's perception.

[0028] Figure 2 is a flow chart of a method for generating textures for a three-dimensional model provided by an embodiment of the present disclosure. This method can be performed by a texture generation device for a three-dimensional model, wherein the device can be implemented using software and / or hardware and can generally be integrated into an electronic device. As shown in Figure 2, the method mainly includes the following steps S202 to S208:

[0029] Step S202: Acquire three-dimensional model representation data of the texture to be generated.

[0030] The three-dimensional model representation data can be implicit representation data of the three-dimensional model, or it can be displayed representation data of the three-dimensional model, or it can be implicit-display hybrid representation data of the three-dimensional model. Among them, the displayed representation data can intuitively present the model's external structure, and can also be understood as representing the three-dimensional model through an explicit 3D structure. For example, for a certain spatial point, it can be clearly and intuitively known whether the spatial point is inside or outside the model; while the implicit representation data only contains the structural information of the model, and can also be understood as representing the three-dimensional model through an implicit 3D structure, it cannot intuitively present the model's external structure, and it is necessary to use a network to predict whether the spatial point is inside or outside the model; implicit-display hybrid representation data can combine the advantages of display representation data and implicit representation data. For example, the data can be presented based on three orthogonal feature planes, which can be called tri-plane data; compared with display representation data, implicit-display hybrid representation data requires less data and can effectively reduce the amount of computation. Compared with implicit representation data, implicit-display hybrid representation data can, to a certain extent, perform display expression of the three-dimensional model, which helps to more accurately determine the model structure. In practical applications, the type of three-dimensional model representation data can be flexibly selected and is not restricted here.

[0031] In practical applications, one can first generate texture-free 3D model representation data based on the model information, or directly obtain existing texture-free 3D model representation data, without any limitation here. Furthermore, the disclosed embodiments do not restrict the model morphology corresponding to the 3D model representation data; the data can be, for example, objects, animals, people, buildings, and the like.

[0032] Step S204 : generating a plurality of surface local regions of the target three-dimensional model based on the three-dimensional model representation data.

[0033] In practical applications, a target 3D model can be determined based on the 3D model representation data, and the surface of the target 3D model can be divided into multiple blocks to obtain multiple surface local regions. In some implementation examples, the surface local regions can be represented by a set of points or directly in the form of an image, that is, the surface local regions can be in the form of a surface local map.

[0034] In some implementation examples, the combination of multiple surface local regions covers the entire surface of the target 3D model, thereby ensuring that the entire surface texture of the target 3D model can be completely obtained. In addition, some of the multiple surface local regions may also have overlapping areas, which is not limited here.

[0035] In practical applications, different surface local regions can have the same shape and size, such as rectangles or triangles of a specified size. Alternatively, multiple surface local regions can have different shapes and / or sizes. Because 3D models are three-dimensional structures, surfaces can often be irregular. Therefore, it is more flexible to generate surface local regions of different shapes, or to generate surface local regions of the same shape but different sizes, based on actual conditions.

[0036] Step S206 : Acquire features corresponding to the local area of ​​the surface, and generate a texture map corresponding to the local area of ​​the surface based on the features.

[0037] In practical applications, the embodiments of the present disclosure do not restrict the method of acquiring features of local surface areas. The method of generating corresponding texture maps based on the features corresponding to local surface areas is not only more efficient, but also fully considers the information of multiple points in the local surface area when generating the texture map. Compared with the method of directly generating corresponding texture colors for each point separately in the related art, the above-mentioned method provided by the embodiments of the present disclosure processes the entire local surface area, and when determining the texture information of a certain point, the information of surrounding points will also be considered, thereby effectively ensuring the rationality of the color of each point and the local continuity of the model surface texture, and the obtained texture map is more reliable.

[0038] Exemplarily, the texture maps corresponding to the respective local areas of the surface can be determined by a preset neural network based on the features corresponding to the local areas of the surface. In some specific examples, the feature map can be converted into an RGB image by a neural network to obtain the texture map corresponding to the local area of ​​the surface based on the RGB image. For example, the RGB image can be directly used as the texture map, or the RGB image can be fine-tuned as required, and the fine-tuned RGB image can be used as the texture map. In addition, the embodiments of the present disclosure do not limit the neural network. For example, the neural network can be a Unet network, etc., and any network that can convert the feature map into a three-channel RGB image is acceptable.

[0039] Step S208: Obtain the surface texture of the target 3D model based on the texture maps corresponding to the plurality of surface local regions. Specifically, the texture maps corresponding to the plurality of surface local regions are combined based on the positions of the surface local regions corresponding to the 3D model to obtain the entire surface texture of the 3D model.

[0040] The above method of directly using the feature maps of multiple local surface areas of the three-dimensional model to generate corresponding texture maps requires less computational effort, is not only more efficient, but can also better ensure the local continuity of the surface texture of the three-dimensional model, and effectively ensure the texture generation effect of the three-dimensional model.

[0041] The present disclosure provides an implementation example of obtaining features corresponding to a local area of ​​a surface, which can be performed with reference to the following steps A and B:

[0042] In step A, a plurality of sub-units in a local area of ​​the surface are determined, and unit features of the sub-units are obtained.

[0043] The sub-units of the local surface area can be flexibly set according to actual needs. For example, each point in the local surface area can be regarded as a sub-unit, and multiple point clusters in the local surface area can be regarded as a sub-unit. For example, each unit area in the local surface area divided according to a preset method can be regarded as a sub-unit, and the size of the unit area can be flexibly set according to needs.

[0044] In some specific implementation examples, the three-dimensional model characterization data is implemented based on three orthogonal feature planes, which are perpendicular to each other and can also be called tri-planes; for example, each feature plane is 256*256*32. In actual applications, when obtaining the unit features of a sub-unit, the sub-unit can be first projected onto each feature plane respectively; then, based on the projection results of the sub-unit on each feature plane, the unit features of the sub-unit are determined. It can be understood that, as mentioned above, since the three-dimensional model characterization data implemented by the three orthogonal feature planes is a mixture of explicit advantages and implicit advantages, the method of directly projecting the sub-unit onto each feature plane to determine the corresponding unit features, on the basis of being able to effectively guarantee the feature extraction effect, requires a lower amount of computation and can also more efficiently obtain the unit features of the sub-unit.

[0045] Step B: determining the features corresponding to the local area of ​​the surface based on the unit features of the multiple sub-units.

[0046] For example, the set of unit features of the subunits in a local surface area can be directly used as the feature corresponding to the local surface area; or the unit features of the subunits in the local surface area can be combined and spliced ​​to obtain the feature corresponding to the local surface area, which can also be represented in the form of a feature map. In this way, the corresponding features can be extracted for each local surface area, and the features corresponding to each local surface area can be further input into the neural network to obtain the texture map corresponding to each local surface area.

[0047] The above method mainly determines the surface texture of the model based on the textures of multiple local surface areas on the model. Compared with the method of directly determining the texture color of each point on the model in related technologies, it can not only improve the efficiency of texture generation, but also directly determine the corresponding features based on the local surface area and then determine the corresponding texture map. The information of multiple points within the local area can be comprehensively considered. The feature information of multiple points within the local area can affect each other, which can effectively ensure the rationality of the color of the model surface and the local continuity of the model surface texture, avoiding the problems of color stratification and discontinuity in related technologies, and the final texture map is more reliable.

[0048] Based on the above, the present disclosure also provides a method for generating a three-dimensional model. Figure 3 is a flow chart of a method for generating a three-dimensional model provided by the present disclosure. This method can be performed by a three-dimensional model generation device, wherein the device can be implemented using software and / or hardware and can generally be integrated into an electronic device. As shown in Figure 3, the method mainly includes the following steps S302 to S306:

[0049] Step S302 : generating three-dimensional model representation data of the texture to be generated based on the received model information.

[0050] In practical applications, the model information can be noise containing the model information, model description text, or a two-dimensional image corresponding to the model, etc. The embodiments of the present disclosure do not limit the form of expression of the model information. In addition, the embodiments of the present disclosure do not limit the specific implementation method of generating the three-dimensional model representation data of the texture to be generated based on the model information. The above-mentioned three-dimensional model representation data can be represented by implicit representation data, explicit representation data, or implicit-explicit hybrid representation data of the aforementioned three-dimensional model, which is not limited here.

[0051] In some implementation examples, the three-dimensional model representation data corresponding to the model information is implemented based on three orthogonal feature planes. In this way, the generation efficiency of the three-dimensional model representation data is higher, the required amount of calculation is smaller, and to a certain extent, the reliability of the model structure can be better guaranteed. In some specific implementation examples, the three-dimensional model representation data can be directly represented by three orthogonal feature planes. In other specific implementation examples, the three-dimensional model representation data can be the model geometry data finally obtained by the three orthogonal feature planes, which can intuitively present the three-dimensional structure of the model. Exemplarily, the model geometry data can be a 3D grid model, which can also be called a 3D mesh model.

[0052] In some specific implementation examples, based on the received model information, a preset 3D GAN (Generative adversarial network) can be used to generate three-dimensional model representation data of the texture to be generated, and with the help of the powerful expression ability of the GAN network, more accurate and realistic three-dimensional model representation data can be generated.

[0053] Step S304 , obtaining the surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained by using the texture generation method of the three-dimensional model, and the details can be referred to the above related content, which will not be repeated here.

[0054] In some specific implementation examples, a preset 2D GAN network can be used to obtain the surface texture of the initial three-dimensional model, that is, to generate the model texture in 2D space. Compared with the method of directly generating the model texture in 3D space in related technologies, the required amount of computation is smaller, and the method of obtaining the model texture based on the local surface area in 2D space can also better ensure local continuity.

[0055] Step S306 : generating a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

[0056] The above-mentioned three-dimensional model generation method provided in the embodiment of the present disclosure can use the above-mentioned three-dimensional model texture generation method to generate a surface texture with better effect, thereby ensuring the presentation effect of the final target three-dimensional model and better ensuring the user's viewing experience.

[0057] In some implementation examples, the step of generating three-dimensional model representation data of the texture to be generated based on the received model information includes: generating the three-dimensional model representation data of the texture to be generated based on the received model information using a preset generation network. The embodiments of this disclosure do not limit the specific structure of the generation network; any method that can generate three-dimensional model representation data is acceptable. For example, the generation network is trained according to the following steps a to c:

[0058] Step a: Obtain a model information sample and a base image of the target perspective corresponding to the 3D model representation data of the model information sample; the base image includes one or more of a depth map, a normal map, and a mask map. In practical applications, the model's stereoscopic structure can be determined based on the 3D model representation data corresponding to the model information sample. It is understood that the model's stereoscopic structure will correspond to different base images at different perspectives. In practical applications, the target perspective can be flexibly set, and the base image of the model's stereoscopic structure at the target perspective can be obtained as the base image of the target perspective corresponding to the 3D model representation data.

[0059] Step b: obtaining the three-dimensional model representation data of the texture to be generated output by the preset initial network for the model information sample, and obtaining the basic image of the target perspective corresponding to the three-dimensional model representation data output by the initial network.

[0060] In some implementation examples, the three-dimensional model representation data is implemented based on three orthogonal feature planes. At this time, a neural rendering algorithm can be used based on the three orthogonal feature planes to obtain the basic image under the target perspective corresponding to the three-dimensional model representation data output by the initial network. The required amount of computation is low, and the basic image can be generated more efficiently.

[0061] Step c: adjusting the parameters of the initial network according to the difference between the basic image corresponding to the 3D model sample and the basic image corresponding to the initial network to obtain the generated network.

[0062] It can be understood that if the difference between the three-dimensional model representation data of the model information sample and the three-dimensional model representation data output by the initial network is directly compared, the required amount of data is large and higher computing resources are consumed. The embodiment of the present disclosure requires less computing power by comparing the basic images of the two (one or more of the depth map, normal map, and mask map). Moreover, since the basic image corresponds to the three-dimensional model representation data, comparing the difference between the basic images is equivalent to indirectly comparing the difference between the three-dimensional model representation data, thereby achieving better network training effects. Adjusting the parameters of the initial network with the goal of reducing the difference between the basic images will also reduce the difference between the three-dimensional model representation data accordingly, so that the initial network can gradually output three-dimensional model representation data that is close to the three-dimensional model representation data of the model information sample, and finally obtain a trained generation network.

[0063] To facilitate understanding, in some specific implementation examples, the three-dimensional model generation process can be divided into two stages. The first stage is used to generate an initial three-dimensional model without texture, wherein the initial three-dimensional model can be a stereoscopic model obtained based on the aforementioned three-dimensional model representation data, such as, the initial three-dimensional model can be a 3D mesh model. The second stage is used to generate the surface texture of the initial three-dimensional model, and finally obtain the required target three-dimensional model.

[0064] For ease of understanding, based on the above, we can first refer to the structural diagram of a three-dimensional model generation network shown in Figure 4, which illustrates that the generation network of the three-dimensional model includes a one-stage network and a two-stage network. Exemplarily, the one-stage network includes a 3D GAN network, and the two-stage network includes a 2D GAN network. The one-stage network is used to generate an initial three-dimensional model without texture based on the received model information, and the two-stage network is used to generate the surface texture of the initial three-dimensional model based on the output of the one-stage network, thereby obtaining a target three-dimensional model. In specific implementation, the two-stage network can generate a series of surface local maps (corresponding to the aforementioned surface local areas, which can be called patches) based on the model structure output by the one-stage network, and then obtain the texture map corresponding to the surface local map.

[0065] On the basis of the above, the embodiment of the present disclosure also provides a structural schematic diagram of a three-dimensional model generation network as shown in Figure 5. Specifically, after the model information is input into the 3D GAN network, three planes (that is, three orthogonal feature planes, which can be called tri-planes) are obtained. The spatial points in the three planes can obtain an initial three-dimensional model after passing through a multi-level cascade network, wherein the initial three-dimensional model can be obtained based on the three planes. Based on the surface of the initial three-dimensional model, multiple surface local maps (patches) can be obtained. Based on the three planes, the 2DGAN network can be used to obtain the feature map (feature patch) corresponding to the surface local map. The feature map corresponding to the surface local map is passed through the Unet network to obtain the texture map corresponding to the surface local map. Through the initial three-dimensional model and the texture map, the target three-dimensional model can be obtained. Among them, the 3D GAN network and the multi-level cascade network can be regarded as belonging to the one-stage network, and the 2D GAN network and the Unet network can be regarded as belonging to the two-stage network. Figure 5 is only an exemplary illustration. In actual applications, other network structures can also be used, which are not limited here.

[0066] For ease of understanding, the disclosed embodiment also provides a training method for the generative network of the above-mentioned three-dimensional model. Specifically, the generative network of the three-dimensional model to be trained can be made to generate an initial three-dimensional model of the texture to be generated and the surface texture of the initial three-dimensional model based on the model sample information of the three-dimensional model sample, thereby obtaining a target three-dimensional model corresponding to the model sample information. A first loss is determined according to the difference between the basic image (one or more of the depth map, normal map, and mask map) corresponding to the initial three-dimensional model and the basic image of the three-dimensional model sample, and a second loss is determined according to the difference between the rendered image of the target three-dimensional model and the rendered image of the three-dimensional model sample, and the parameters of the generative network of the three-dimensional model are adjusted according to the first loss and the second loss. It can be understood that the above-mentioned basic image and rendered image are directly related to the model structure. Compared with the method of adjusting the network parameters directly based on the model structure difference, the method of adjusting the network parameters through image difference requires less computation and is more convenient and quick.

[0067] In practical applications, in order to improve the efficiency of network training, the network training process can also be divided into two stages. In the first training stage, the parameters of the first-stage network in the generative network of the three-dimensional model are iteratively optimized based on the above-mentioned first loss until the first loss converges to a preset threshold range. At this time, the first-stage network can output an initial network model that meets expectations, and then fix the parameters of the first-stage network, and then based on the above-mentioned second loss, iteratively optimize the parameters of the second-stage network in the generative network of the three-dimensional model until the second loss converges to a preset threshold range. In this way, the generative network of the three-dimensional model can be obtained more efficiently and reliably. In addition, the embodiment of the present disclosure does not limit the first loss and the second loss. For example, the first loss and the second loss can be GAN losses.

[0068] In summary, through the three-dimensional model generation method provided by the embodiment of the present disclosure, using the above-mentioned three-dimensional model generation network, a more reliable initial network model can be generated in the first stage, and a surface texture with better effect can be generated in the second stage, thereby effectively ensuring the presentation effect of the final target three-dimensional model and better ensuring the user's viewing experience.

[0069] Corresponding to the aforementioned texture generation method for a three-dimensional model, FIG6 is a schematic structural diagram of a texture generation device for a three-dimensional model provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. As shown in FIG6 , the device includes:

[0070] Model data acquisition module 602, used to obtain three-dimensional model representation data of the texture to be generated;

[0071] A local region generating module 604 is configured to generate a plurality of surface local regions of a target three-dimensional model based on the three-dimensional model representation data;

[0072] A texture map generating module 606 is configured to obtain features corresponding to a local area of ​​the surface, and generate a texture map corresponding to the local area of ​​the surface based on the features;

[0073] The surface texture generation module 608 is configured to obtain the surface texture of the target three-dimensional model based on the texture maps corresponding to the plurality of surface local regions.

[0074] The above-mentioned device for directly using the features of multiple local surface areas of a three-dimensional model to generate corresponding texture maps can better ensure the local continuity of the surface texture of the three-dimensional model and effectively ensure the texture generation effect of the three-dimensional model.

[0075] In some embodiments, the texture map generation module 606 is specifically used to: determine multiple sub-units in the local area of ​​the surface and obtain unit features of the sub-units; and determine features corresponding to the local area of ​​the surface based on the unit features of the multiple sub-units.

[0076] In some embodiments, the three-dimensional model representation data is implemented based on three orthogonal feature planes, and the texture map generation module 606 is specifically used to: project the sub-unit onto each of the feature planes respectively; and determine the unit features of the sub-unit based on the projection results of the sub-unit on each of the feature planes.

[0077] In some embodiments, the texture map generation module 606 is specifically configured to determine, based on features corresponding to the plurality of surface local regions, texture maps corresponding to each of the plurality of surface local regions through a preset neural network.

[0078] In some embodiments, the combination of the multiple surface local areas covers the entire surface of the three-dimensional model.

[0079] In some embodiments, different surface local regions have the same shape and size; or, among the plurality of surface local regions, there are surface local regions with different shapes and / or sizes.

[0080] The texture generation device for a three-dimensional model provided in the embodiments of the present disclosure can execute the texture generation method for a three-dimensional model provided in any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.

[0081] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0082] Corresponding to the aforementioned method for generating a three-dimensional model, FIG7 is a schematic structural diagram of a three-dimensional model generation device provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. As shown in FIG7 , it includes:

[0083] A model data generation module 702 is configured to generate three-dimensional model representation data of a texture to be generated based on the received model information;

[0084] A surface texture acquisition module 704 is configured to acquire a surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained by using any of the above-mentioned three-dimensional model texture generation methods;

[0085] The target model generation module 706 is configured to generate a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

[0086] The above-mentioned three-dimensional model generation device provided by the embodiment of the present disclosure can use the above-mentioned three-dimensional model texture generation method to generate a surface texture with better effect, thereby ensuring the presentation effect of the final target three-dimensional model and better ensuring the user's viewing experience.

[0087] In some embodiments, the three-dimensional model representation data corresponding to the model information is implemented based on three orthogonal feature planes.

[0088] In some embodiments, the model data generation module 702 is specifically configured to generate three-dimensional model representation data of the texture to be generated based on the received model information using a preset generation network.

[0089] In some embodiments, the device also includes a training module for training the generation network according to the following steps: obtaining a model information sample and a basic image of the target perspective corresponding to the three-dimensional model representation data of the model information sample; the basic image includes one or more of a depth map, a normal map, and a mask map; obtaining the three-dimensional model representation data of the texture to be generated output by a preset initial network for the model information sample, and obtaining the basic image of the target perspective corresponding to the three-dimensional model representation data output by the initial network; adjusting the parameters of the initial network according to the difference between the basic image corresponding to the three-dimensional model sample and the basic image corresponding to the initial network to obtain the generation network.

[0090] In some embodiments, the training module is specifically used to obtain, based on the three orthogonal feature planes, a basic image under the target perspective corresponding to the three-dimensional model representation data output by the initial network using a neural rendering algorithm.

[0091] In some embodiments, the model data generation module 702 is specifically configured to: generate three-dimensional model representation data of the texture to be generated based on the received model information using a preset 3D GAN network;

[0092] In some embodiments, the surface texture acquisition module 704 is specifically configured to: use a preset 2D GAN network to acquire the surface texture corresponding to the three-dimensional model representation data.

[0093] The texture device for a three-dimensional model provided in the embodiments of the present disclosure can execute the texture method for a three-dimensional model provided in any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.

[0094] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0095] An embodiment of the present disclosure provides an electronic device, which includes: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of a texture generation method for a three-dimensional model or the steps of a three-dimensional model generation method.

[0096] Reference is now made to FIG8 , which illustrates a schematic diagram of the structure of an electronic device 800 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device illustrated in FIG8 is merely an example and should not limit the functionality or scope of use of embodiments of the present disclosure.

[0097] As shown in Figure 8, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0098] Typically, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although FIG8 shows the electronic device 800 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0099] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0100] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to perform the image processing method provided by the embodiments of the present disclosure. The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present disclosure, the programming languages ​​including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0101] In addition, the embodiments of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor is caused to execute the method provided by the embodiments of the present disclosure.

[0102] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0103] The embodiments of the present disclosure further provide a computer program product, including a computer program / instruction, which implements the method in the embodiments of the present disclosure when executed by a processor.

[0104] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0105] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0106] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0107] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0109] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A texture generation method for a three-dimensional model, comprising: Acquire three-dimensional model representation data of the texture to be generated; generating a plurality of surface local regions of a target three-dimensional model based on the three-dimensional model representation data; Acquire features corresponding to the local area of ​​the surface to generate a texture map corresponding to the local area of ​​the surface based on the features; Based on the texture maps corresponding to the multiple surface local areas, the surface texture of the target three-dimensional model is obtained.

2. The method according to claim 1, wherein: The acquiring the feature corresponding to the local area of ​​the surface includes: Determine a plurality of sub-units in the local area of ​​the surface, and obtain unit characteristics of the sub-units; The feature corresponding to the local area of ​​the surface is determined according to the unit features of the plurality of sub-units.

3. The method according to claim 2, wherein: The three-dimensional model representation data is implemented based on three orthogonal feature planes, and the obtaining of the unit features of the subunits includes: The subunits are projected onto each of the characteristic planes respectively; and the unit characteristics of the subunits are determined based on the projection results of the subunits onto each of the characteristic planes.

4. The method according to claim 1, wherein: The step of generating a texture map corresponding to the local area of ​​the surface based on the feature map comprises: Based on the features corresponding to the plurality of local surface areas, a texture map corresponding to each of the plurality of local surface areas is determined through a preset neural network.

5. The method according to claim 1, wherein: The combination of the multiple surface local areas covers the entire surface of the three-dimensional model.

6. The method according to claim 1, wherein: The shapes and sizes of different surface local regions are the same; or, among the plurality of surface local regions, there are surface local regions with different shapes and / or sizes.

7. A method for generating a three-dimensional model, comprising: Based on the received model information, generating three-dimensional model representation data of the texture to be generated; Acquire the surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained by using the texture generation method of the three-dimensional model according to any one of claims 1 to 6; A target three-dimensional model is generated based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

8. The method according to claim 7, wherein: The three-dimensional model representation data corresponding to the model information is realized based on three orthogonal characteristic planes.

9. The method according to claim 8, wherein: The step of generating three-dimensional model representation data of a texture to be generated based on the received model information includes: Based on the received model information, a preset generation network is used to generate three-dimensional model representation data of the texture to be generated.

10. The method according to claim 9, wherein: The generative network is trained according to the following steps: Acquire a model information sample and a basic image of a target perspective corresponding to the three-dimensional model representation data of the model information sample; the basic image includes one or more of a depth map, a normal map, and a mask map; Acquire three-dimensional model representation data of the texture to be generated output by a preset initial network for the model information sample, and acquire a basic image of the target perspective corresponding to the three-dimensional model representation data output by the initial network; According to the difference between the basic image corresponding to the three-dimensional model sample and the basic image corresponding to the initial network, the parameters of the initial network are adjusted to obtain the generated network.

11. The method according to claim 10, wherein: The step of obtaining the basic image under the target perspective corresponding to the three-dimensional model representation data output by the initial network includes: Based on the three orthogonal feature planes, a neural rendering algorithm is used to obtain a basic image under the target perspective corresponding to the three-dimensional model representation data output by the initial network.

12. The method according to claim 7, wherein: The step of generating three-dimensional model representation data of a texture to be generated based on the received model information includes: Based on the received model information, a preset 3D GAN network is used to generate three-dimensional model representation data of the texture to be generated; The obtaining of the surface texture corresponding to the three-dimensional model representation data includes: The surface texture corresponding to the three-dimensional model representation data is obtained using a preset 2D GAN network.

13. A texture generation device for a three-dimensional model, comprising: A model data acquisition module, used to acquire three-dimensional model representation data of the texture to be generated; A local area generation module, used to generate a plurality of surface local areas of a target three-dimensional model based on the three-dimensional model representation data; A texture map generation module, used for acquiring features corresponding to the local area of ​​the surface, so as to generate a texture map corresponding to the local area of ​​the surface based on the feature map; The surface texture generation module is used to obtain the surface texture of the target three-dimensional model based on the texture maps corresponding to the multiple surface local areas.

14. A device for generating a three-dimensional model, comprising: A model data generation module, used to generate three-dimensional model representation data of the texture to be generated based on the received model information; A surface texture acquisition module, used to acquire the surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained by using the texture generation method of the three-dimensional model according to any one of claims 1 to 6; The target model generation module is used to generate a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

15. An electronic device, comprising: a storage device having a computer program stored thereon; A processing device, used to execute the computer program in the storage device to implement the steps of the texture generation method of the three-dimensional model described in any one of claims 1-6 or the steps of the generation method of the three-dimensional model described in any one of claims 7-12.

16. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the steps of the texture generation method for a three-dimensional model described in any one of claims 1 to 6 or the steps of the generation method for a three-dimensional model described in any one of claims 7 to 12.

Images