Texture generation method of three-dimensional model and generation method and device of three-dimensional model

Abstract

The embodiment of the invention relates to a texture generation method of a three-dimensional model and a generation method and device of the three-dimensional model. The texture generation method of the three-dimensional model comprises the steps of obtaining three-dimensional model representation data of a texture to be generated; generating a plurality of surface local maps of the target three-dimensional model based on the three-dimensional model representation data; obtaining features corresponding to the surface local graph, and generating a texture graph corresponding to the surface local graph based on the features; and obtaining the surface texture of the three-dimensional model based on the texture maps corresponding to the plurality of surface local maps. According to the embodiment of the invention, the local continuity of the surface texture of the three-dimensional model can be well guaranteed, the texture generation effect of the three-dimensional model is effectively guaranteed, the presentation effect of the finally obtained target three-dimensional model is further guaranteed, and the user impression can be well guaranteed.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a texture generation method for a three-dimensional model, and a generation method and device for a three-dimensional model. Background Art

[0002] In virtual scenes such as VR (Virtual Reality), three-dimensional models are usually involved, and the presentation effect of the three-dimensional model is crucial. The inventor has found through research that the texture generation method used in the relevant technology in the process of generating three-dimensional models also has poor texture effect, resulting in poor presentation effect of the three-dimensional model and poor user perception. Summary of the invention

[0003] In order to solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides a texture generation method for a three-dimensional model, a generation method and a device for a three-dimensional model.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for generating texture of a three-dimensional model, comprising: obtaining three-dimensional model representation data of a texture to be generated; generating multiple surface local areas of a target three-dimensional model based on the three-dimensional model representation data; obtaining features corresponding to the surface local areas to generate a texture map corresponding to the surface local areas based on the features; and obtaining the surface texture of the target three-dimensional model based on the texture maps corresponding to each of the multiple surface local areas.

[0005] In a second aspect, an embodiment of the present disclosure also provides a method for generating a three-dimensional model, comprising: generating three-dimensional model representation data of a texture to be generated based on received model information; obtaining a surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained using the texture generation method for a three-dimensional model provided in the first aspect; and generating a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

[0006] In a third aspect, an embodiment of the present disclosure also provides a texture generation device for a three-dimensional model, comprising: a model data acquisition module, used to acquire three-dimensional model representation data of a texture to be generated; a local area generation module, used to generate multiple surface local areas of a target three-dimensional model based on the three-dimensional model representation data; a texture map generation module, used to acquire features corresponding to the surface local areas, so as to generate a texture map corresponding to the surface local areas based on the features; and a surface texture generation module, used to obtain the surface texture of the target three-dimensional model based on the texture maps corresponding to each of the multiple surface local areas.

[0007] In a fourth aspect, an embodiment of the present disclosure also provides a three-dimensional model generation device, comprising: a model data generation module, used to generate three-dimensional model representation data of a texture to be generated based on received model information; a surface texture acquisition module, used to acquire a surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained using the three-dimensional model texture generation method described in the first aspect; and a target model generation module, used to generate a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

[0008] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a texture generation method for a three-dimensional model or a three-dimensional model generation method as provided in an embodiment of the present disclosure.

[0009] In a sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute a texture generation method for a three-dimensional model or a generation method for a three-dimensional model as provided in an embodiment of the present disclosure.

[0010] The texture generation method of the above-mentioned three-dimensional model provided in the embodiment of the present disclosure can generate multiple surface local areas based on the three-dimensional model representation data of the texture to be generated, and obtain the features corresponding to the surface local areas, so as to generate the texture map corresponding to the surface local areas based on the features, and finally obtain the surface texture of the three-dimensional model based on the texture maps corresponding to each of the multiple surface local areas. The above-mentioned method of directly using the features of multiple surface local areas of the three-dimensional model to generate the corresponding texture map is not only more efficient, but also can better guarantee the local continuity of the surface texture of the three-dimensional model, and effectively guarantee the texture generation effect of the three-dimensional model. Furthermore, the generation method of the above-mentioned three-dimensional model provided in the embodiment of the present disclosure first generates the three-dimensional model representation data of the texture to be generated, and uses the texture generation method of the above-mentioned three-dimensional model to generate a surface texture with better effect, thereby ensuring the presentation effect of the target three-dimensional model finally obtained, which can better guarantee the user's perception.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0014] Figure 1 A schematic diagram of a model texture provided by an embodiment of the present disclosure;

[0015] Figure 2 A schematic diagram of a flow chart of a texture generation method for a three-dimensional model provided in an embodiment of the present disclosure;

[0016] Figure 3 A schematic diagram of a process for generating a three-dimensional model provided in an embodiment of the present disclosure;

[0017] Figure 4 A schematic diagram of a structure of a three-dimensional model generation network provided in an embodiment of the present disclosure;

[0018] Figure 5 A schematic diagram of a structure of a three-dimensional model generation network provided in an embodiment of the present disclosure;

[0019] Figure 6 A schematic diagram of the structure of a texture generating device for a three-dimensional model provided in an embodiment of the present disclosure;

[0020] Figure 7 A schematic diagram of the structure of a three-dimensional model generation device provided in an embodiment of the present disclosure;

[0021] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0024] The inventor has found through research that the texture generation method used in the related art in the process of generating a 3D model is not good. Specifically, the related art needs to process each point on the surface of the 3D model separately to determine the color corresponding to the point, so as to finally obtain the surface texture of the 3D model. However, the texture effect obtained in this way is not good, and it is easy to have problems such as color stratification, such as, see Figure 1 A schematic diagram of a model texture is shown in Figure 1 The texture of the vehicle in the image is obviously layered and uneven. Therefore, the color rationality and local continuity of the surface texture of the three-dimensional model generated by the relevant technology are not good. In order to improve at least one of the above problems, the embodiment of the present disclosure provides a texture generation method of a three-dimensional model, a generation method and device of a three-dimensional model, which are described in detail below.

[0025] Figure 2 The present invention provides a flow chart of a method for generating texture of a three-dimensional model according to an embodiment of the present invention. The method can be executed by a device for generating texture of a three-dimensional model, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 2 As shown, the method mainly includes the following steps S202 to S208:

[0026] Step S202, obtaining three-dimensional model representation data of the texture to be generated.

[0027] The three-dimensional model representation data can be the implicit representation data of the three-dimensional model, the display representation data of the three-dimensional model, or the implicit-display hybrid representation data of the three-dimensional model. Among them, the display representation data can intuitively present the model shape structure, and can also be understood as the representation of the three-dimensional model through the explicit 3D structure. For example, for a certain spatial point, it can be clearly and intuitively known whether the spatial point is located inside or outside the model; while the implicit representation data only contains the structural information of the model, and can also be understood as the representation of the three-dimensional model through the implicit 3D structure, which cannot intuitively present the model shape structure, and needs to predict whether the spatial point is located inside or outside the model through the network; the implicit-display hybrid representation data can mix the advantages of display representation data and implicit representation data. For example, the data can be presented based on three orthogonal feature planes, which can be called tri-plane data; compared with the display representation data, the implicit-display hybrid representation data requires a smaller amount of data, which can effectively reduce the amount of calculation, and compared with the implicit representation data, the implicit-display hybrid representation data can display and express the three-dimensional model to a certain extent, which helps to more accurately determine the model structure. In practical applications, the type of three-dimensional model representation data can be flexibly selected, and there is no restriction here.

[0028] In practical applications, three-dimensional model representation data without texture can be first generated based on model information, or existing three-dimensional model representation data without texture can be directly obtained, which is not limited here. In addition, the embodiment of the present disclosure does not limit the model morphology type corresponding to the three-dimensional model representation data, such as objects, animals, people, buildings, etc.

[0029] Step S204: generating a plurality of surface local regions of the target three-dimensional model based on the three-dimensional model representation data.

[0030] In practical applications, the target 3D model can be determined based on the 3D model representation data, and the surface of the target 3D model can be divided into multiple blocks to obtain multiple surface local areas. In some implementation examples, the surface local area can be represented by a set of points or directly in the form of an image, that is, the surface local area can be in the form of a surface local map.

[0031] In some implementation examples, the combination of multiple surface local areas covers the entire surface of the target three-dimensional model, thereby ensuring that the entire surface texture of the target three-dimensional model can be completely obtained. In addition, some of the multiple surface local areas may also have overlapping areas, which is not limited here.

[0032] In practical applications, the shapes and sizes of different surface local areas can be the same, such as rectangles or triangles of specified sizes; or, among multiple surface local areas, there can be surface local areas of different shapes and / or sizes. Since the three-dimensional model is a three-dimensional structure, the surface may be irregular in many cases, so surface local areas of different shapes, or surface local areas of the same shape but different sizes can be generated according to actual conditions. This method is more flexible.

[0033] Step S206, acquiring features corresponding to the local area of ​​the surface, so as to generate a texture map corresponding to the local area of ​​the surface based on the features.

[0034] In practical applications, the embodiments of the present disclosure do not restrict the method for acquiring features of local surface areas. The method for generating corresponding texture maps based on features corresponding to local surface areas is not only more efficient, but also fully considers the information of multiple points in the local surface area when generating the texture map. Compared with the method of directly generating corresponding texture colors for each point separately in the related art, the above method provided by the embodiments of the present disclosure processes the entire local surface area, and when determining the texture information of a certain point, the information of surrounding points will also be considered, thereby effectively ensuring the rationality of the color of each point and the local continuity of the model surface texture, and the resulting texture map is more reliable.

[0035] Exemplarily, the texture maps corresponding to the respective local areas of the multiple surfaces can be determined by a preset neural network based on the features corresponding to the local areas of the multiple surfaces. In some specific examples, the feature map can be converted into an RGB image by a neural network to obtain the texture map corresponding to the local area of ​​the surface based on the RGB image. For example, the RGB image can be directly used as the texture map, or the RGB image can be fine-tuned as required, and the fine-tuned RGB image can be used as the texture map. In addition, the embodiments of the present disclosure do not limit the neural network. For example, the neural network can be a Unet network, etc., and any network that can convert the feature map into a three-channel RGB image is acceptable.

[0036] Step S208, based on the texture maps corresponding to the plurality of surface local regions, the surface texture of the target 3D model is obtained. Specifically, the texture maps corresponding to the plurality of surface local regions are combined based on the positions of the surface local regions corresponding to the 3D model to obtain the entire surface texture of the 3D model.

[0037] The above method of directly using the feature maps of multiple local surface areas of the three-dimensional model to generate corresponding texture maps requires less computational effort, is not only more efficient, but can also better ensure the local continuity of the surface texture of the three-dimensional model, and effectively ensure the texture generation effect of the three-dimensional model.

[0038] The present disclosure provides an implementation example of obtaining features corresponding to a local area of ​​a surface, which can be performed with reference to the following steps A and B:

[0039] Step A, determining a plurality of sub-units in a local area of ​​the surface, and obtaining unit features of the sub-units.

[0040] The sub-units of the local surface area can be flexibly set according to actual needs. For example, each point in the local surface area can be regarded as a sub-unit, and multiple point clusters in the local surface area can be regarded as a sub-unit. Also, each unit area in the local surface area divided in a preset manner can be regarded as a sub-unit, and the size of the unit area can be flexibly set according to needs.

[0041] In some specific implementation examples, the three-dimensional model characterization data is implemented based on three orthogonal feature planes, which are perpendicular to each other and can also be called tri-planes; illustratively, each feature plane is 256*256*32. In practical applications, when obtaining the unit features of a sub-unit, the sub-unit can be first projected onto each feature plane respectively; then, based on the projection results of the sub-unit on each feature plane, the unit features of the sub-unit are determined. It can be understood that, as mentioned above, since the three-dimensional model characterization data implemented by three orthogonal feature planes is a mixture of explicit advantages and implicit advantages, the method of directly projecting the sub-unit onto each feature plane to determine the corresponding unit features, on the basis of being able to effectively guarantee the feature extraction effect, requires a lower amount of computation and can also more efficiently obtain the unit features of the sub-unit.

[0042] Step B: determining the features corresponding to the local area of ​​the surface according to the unit features of the plurality of sub-units.

[0043] For example, the set of unit features of the sub-units on the local area of ​​the surface can be directly used as the feature corresponding to the local area of ​​the surface; the unit features of the sub-units on the local area of ​​the surface can also be combined and spliced ​​to obtain the feature corresponding to the local area of ​​the surface, and the feature can also be represented in the form of a feature map. In the above manner, the corresponding features can be extracted for each local area of ​​the surface, and the features corresponding to each local area of ​​the surface are further input into the neural network to obtain the texture map corresponding to each local area of ​​the surface.

[0044] The above method mainly determines the surface texture of the model based on the textures of multiple local surface areas on the model. Compared with the method of directly determining the texture color of each point on the model in the related technology, it can not only improve the texture generation efficiency, but also directly determine the corresponding features based on the local surface area and then determine the corresponding texture map. The information of multiple points within the local area can be comprehensively considered. The feature information of multiple points within the local area can influence each other, which can effectively ensure the rationality of the color of the model surface and the local continuity of the model surface texture, avoid the problems of color stratification and discontinuity in the related technology, and the final texture map is more reliable.

[0045] Based on the above, the present disclosure also provides a method for generating a three-dimensional model. Figure 3 The present invention provides a flow chart of a method for generating a three-dimensional model, which can be executed by a device for generating a three-dimensional model, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 3 As shown, the method mainly includes the following steps S302 to S306:

[0046] Step S302: Generate three-dimensional model representation data of the texture to be generated based on the received model information.

[0047] In practical applications, the model information may be noise containing the model information, or may be a model description text, or may be a two-dimensional image corresponding to the model, etc. The embodiments of the present disclosure do not limit the form of expression of the model information. In addition, the embodiments of the present disclosure do not limit the specific implementation method of generating the three-dimensional model representation data of the texture to be generated based on the model information. Among them, the above-mentioned three-dimensional model representation data can be represented by implicit representation data, explicit representation data, or implicit-explicit mixed representation data of the aforementioned three-dimensional model, which is not limited here.

[0048] In some implementation examples, the three-dimensional model representation data corresponding to the model information is implemented based on three orthogonal feature planes. In this way, the generation efficiency of the three-dimensional model representation data is higher, the required amount of calculation is smaller, and the reliability of the model structure can be better guaranteed to a certain extent. In some specific implementation examples, the three-dimensional model representation data can be directly represented by three orthogonal feature planes. In other specific implementation examples, the three-dimensional model representation data can be the model geometry data finally obtained by the three orthogonal feature planes, which can intuitively present the three-dimensional structure of the model. Exemplarily, the model geometry data can be a 3D mesh model, also known as a 3D mesh model.

[0049] In some specific implementation examples, based on the received model information, a preset 3DGAN (Generative adversarial network) can be used to generate three-dimensional model representation data of the texture to be generated, and more accurate and realistic three-dimensional model representation data can be generated with the help of the powerful expression ability of the GAN network.

[0050] Step S304, obtaining the surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained by using the texture generation method of the three-dimensional model mentioned above, and the details can be referred to the above related contents, which will not be repeated here.

[0051] In some specific implementation examples, a preset 2DGAN network can be used to obtain the surface texture of the initial three-dimensional model, that is, to generate the model texture in 2D space. Compared with the method of directly generating the model texture in 3D space in the related art, the required amount of calculation is smaller, and the method of obtaining the model texture based on the local surface area in 2D space can also better ensure local continuity.

[0052] Step S306: Generate a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

[0053] The above-mentioned three-dimensional model generation method provided in the embodiment of the present disclosure can use the above-mentioned three-dimensional model texture generation method to generate a surface texture with better effect, thereby ensuring the presentation effect of the final target three-dimensional model and better ensuring the user's viewing experience.

[0054] In some implementation examples, the step of generating three-dimensional model representation data of the texture to be generated based on the received model information includes: generating three-dimensional model representation data of the texture to be generated using a preset generation network based on the received model information. The disclosed embodiment does not limit the specific structure of the generation network, and any method that can generate three-dimensional model representation data is acceptable. For example, the generation network is trained according to the following steps a to c:

[0055] Step a, obtaining a model information sample and a basic image of a target perspective corresponding to the three-dimensional model representation data of the model information sample; the basic image includes one or more of a depth map, a normal map, and a mask map. In practical applications, the three-dimensional structure of the model can be determined based on the three-dimensional model representation data corresponding to the model information sample. It can be understood that the basic images corresponding to the three-dimensional structure of the model at different perspectives are different. In practical applications, the above-mentioned target perspective can be flexibly set, and the basic image of the three-dimensional structure of the model at the target perspective can be obtained as the basic image of the target perspective corresponding to the three-dimensional model representation data.

[0056] Step b: obtaining the three-dimensional model representation data of the texture to be generated output by the preset initial network for the model information sample, and obtaining the basic image of the target perspective corresponding to the three-dimensional model representation data output by the initial network.

[0057] In some implementation examples, the three-dimensional model representation data is implemented based on three orthogonal feature planes. At this time, a neural rendering algorithm can be used based on the three orthogonal feature planes to obtain the basic image under the target perspective corresponding to the three-dimensional model representation data output by the initial network. The required amount of computation is low, and the basic image can be generated more efficiently.

[0058] Step c: adjusting the parameters of the initial network according to the difference between the basic image corresponding to the three-dimensional model sample and the basic image corresponding to the initial network to obtain the generated network.

[0059] It can be understood that if the difference between the three-dimensional model representation data of the model information sample and the three-dimensional model representation data output by the initial network is directly compared, the required amount of data is large and higher computing resources are needed. The embodiment of the present disclosure requires less computing power by comparing the basic images of the two (one or more of the depth map, normal map, and mask map). Moreover, since the basic image corresponds to the three-dimensional model representation data, comparing the difference between the basic images is equivalent to indirectly comparing the difference between the three-dimensional model representation data, thereby achieving better network training effects. Adjusting the parameters of the initial network with the goal of reducing the difference between the basic images will also reduce the difference between the three-dimensional model representation data accordingly, so that the initial network can gradually output three-dimensional model representation data that is close to the three-dimensional model representation data of the model information sample, and finally a trained generated network is obtained.

[0060] To facilitate understanding, in some specific implementation examples, the three-dimensional model generation process can be divided into two stages. The first stage is used to generate an initial three-dimensional model without texture, wherein the initial three-dimensional model can be a stereoscopic model obtained based on the aforementioned three-dimensional model representation data, such as, the initial three-dimensional model can be a 3D mesh model, and the second stage is used to generate the surface texture of the initial three-dimensional model, and finally obtain the desired target three-dimensional model.

[0061] For ease of understanding, based on the above, we can first refer to Figure 4 The schematic diagram of the structure of a generation network of a three-dimensional model shown in the figure illustrates that the generation network of the three-dimensional model includes a one-stage network and a two-stage network. Exemplarily, the one-stage network includes a 3DGAN network, and the two-stage network includes a 2DGAN network. The one-stage network is used to generate an initial three-dimensional model without texture based on the received model information, and the two-stage network is used to generate the surface texture of the initial three-dimensional model based on the output of the one-stage network, thereby obtaining a target three-dimensional model. In specific implementation, the two-stage network can generate a series of surface local maps (corresponding to the aforementioned surface local areas, which can be called patches) based on the model structure output by the one-stage network, and then obtain the texture map corresponding to the surface local map.

[0062] On the basis of the foregoing, the present disclosure also provides the following embodiments: Figure 5A schematic diagram of the structure of a three-dimensional model generation network is shown. Specifically, after the model information is input into the 3DGAN network, three planes (that is, three orthogonal feature planes, which can be called tri-planes) are obtained. After the spatial points in the three planes pass through a multi-level cascade network, an initial three-dimensional model can be obtained, wherein the initial three-dimensional model can be obtained based on the three planes. Based on the surface segmentation of the initial three-dimensional model, multiple surface local maps (patches) can be obtained. Based on the three planes, the feature map (feature patch) corresponding to the surface local map can be obtained by using the 2DGAN network. The feature map corresponding to the surface local map can be passed through the Unet network to obtain the texture map corresponding to the surface local map. Through the initial three-dimensional model and the texture map, the target three-dimensional model can be obtained. Among them, the 3D GAN network and the multi-level cascade network can be regarded as belonging to the one-stage network, and the 2D GAN network and the Unet network can be regarded as belonging to the two-stage network. Figure 5 This is only an example description, and other network structures may be used in actual applications and are not limited here.

[0063] For ease of understanding, the disclosed embodiment also provides a training method for the generative network of the above-mentioned three-dimensional model. Specifically, the generative network of the three-dimensional model to be trained can generate an initial three-dimensional model of the texture to be generated and the surface texture of the initial three-dimensional model based on the model sample information of the three-dimensional model sample, thereby obtaining a target three-dimensional model corresponding to the model sample information. The first loss is determined according to the difference between the basic image (one or more of the depth map, normal map, and mask map) corresponding to the initial three-dimensional model and the basic image of the three-dimensional model sample, and the second loss is determined according to the difference between the rendered image of the target three-dimensional model and the rendered image of the three-dimensional model sample, and the parameters of the generative network of the three-dimensional model are adjusted according to the first loss and the second loss. It can be understood that the above-mentioned basic image and rendered image are directly related to the model structure. Compared with the method of adjusting the network parameters directly based on the model structure difference, the method of adjusting the network parameters through image difference requires less computation and is more convenient and quick.

[0064] In practical applications, in order to improve the efficiency of network training, the network training process can also be divided into two stages. In the first training stage, the parameters of the first-stage network in the generative network of the three-dimensional model are iteratively optimized mainly based on the above-mentioned first loss, until the first loss converges to a preset threshold range. At this time, the first-stage network can output an initial network model that meets expectations, and then fix the parameters of the first-stage network, and then based on the above-mentioned second loss, iteratively optimize the parameters of the second-stage network in the generative network of the three-dimensional model until the second loss converges to a preset threshold range. In the above manner, the generative network of the three-dimensional model can be obtained more efficiently and reliably. In addition, the embodiments of the present disclosure do not limit the first loss and the second loss. Exemplarily, the first loss and the second loss can be GAN losses.

[0065] In summary, through the three-dimensional model generation method provided by the embodiment of the present invention, using the above-mentioned three-dimensional model generation network, a more reliable initial network model can be generated in the first stage, and a surface texture with better effect can be generated in the second stage, thereby effectively ensuring the presentation effect of the final target three-dimensional model and better ensuring the user's viewing experience.

[0066] Corresponding to the texture generation method of the aforementioned three-dimensional model, Figure 6 The present invention provides a structural diagram of a texture generation device for a three-dimensional model. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device, such as Figure 6 As shown, including:

[0067] Model data acquisition module 602, used to acquire three-dimensional model representation data of the texture to be generated;

[0068] A local region generating module 604 is used to generate a plurality of surface local regions of a target three-dimensional model based on the three-dimensional model representation data;

[0069] The texture map generation module 606 is used to obtain features corresponding to the local area of ​​the surface, so as to generate a texture map corresponding to the local area of ​​the surface based on the features;

[0070] The surface texture generation module 608 is used to obtain the surface texture of the target three-dimensional model based on the texture maps corresponding to the multiple surface local areas.

[0071] The above-mentioned device for directly using the features of multiple local surface areas of the three-dimensional model to generate the corresponding texture map can better ensure the local continuity of the surface texture of the three-dimensional model and effectively ensure the texture generation effect of the three-dimensional model.

[0072] In some embodiments, the texture map generation module 606 is specifically used to: determine multiple sub-units in the local area of ​​the surface and obtain unit features of the sub-units; and determine features corresponding to the local area of ​​the surface based on the unit features of the multiple sub-units.

[0073] In some embodiments, the three-dimensional model representation data is implemented based on three orthogonal feature planes, and the texture map generation module 606 is specifically used to: project the sub-unit to each of the feature planes respectively; and determine the unit characteristics of the sub-unit based on the projection results of the sub-unit on each of the feature planes.

[0074] In some implementations, the texture map generation module 606 is specifically used to: determine the texture maps corresponding to each of the plurality of local surface regions through a preset neural network based on the features corresponding to the plurality of local surface regions.

[0075] In some embodiments, the combination of the multiple surface local areas covers the entire surface of the three-dimensional model.

[0076] In some embodiments, different surface local regions have the same shape and size; or, among the plurality of surface local regions, there are surface local regions with different shapes and / or sizes.

[0077] The texture generation device for a three-dimensional model provided in the embodiments of the present disclosure can execute the texture generation method for a three-dimensional model provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0078] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0079] Corresponding to the aforementioned method of generating a three-dimensional model, Figure 7 A schematic diagram of the structure of a three-dimensional model generation device provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated in an electronic device, such as Figure 7 As shown, including:

[0080] A model data generation module 702, for generating three-dimensional model representation data of a texture to be generated based on the received model information;

[0081] The surface texture acquisition module 704 is used to acquire the surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained by using any of the above-mentioned three-dimensional model texture generation methods;

[0082] The target model generation module 706 is used to generate a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

[0083] The above-mentioned three-dimensional model generation device provided by the embodiment of the present disclosure can adopt the above-mentioned three-dimensional model texture generation method to generate a surface texture with better effect, thereby ensuring the presentation effect of the final target three-dimensional model and better ensuring the user's viewing experience.

[0084] In some implementations, the three-dimensional model representation data corresponding to the model information is implemented based on three orthogonal feature planes.

[0085] In some implementations, the model data generation module 702 is specifically configured to: generate three-dimensional model representation data of the texture to be generated based on the received model information using a preset generation network.

[0086] In some embodiments, the device also includes a training module for training the generation network according to the following steps: obtaining a model information sample and a basic image of a target perspective corresponding to the three-dimensional model representation data of the model information sample; the basic image includes one or more of a depth map, a normal map, and a mask map; obtaining the three-dimensional model representation data of the texture to be generated output by a preset initial network for the model information sample, and obtaining a basic image of the target perspective corresponding to the three-dimensional model representation data output by the initial network; adjusting the parameters of the initial network according to the difference between the basic image corresponding to the three-dimensional model sample and the basic image corresponding to the initial network to obtain the generation network.

[0087] In some embodiments, the training module is specifically used to obtain, based on the three orthogonal feature planes, a basic image under the target perspective corresponding to the three-dimensional model representation data output by the initial network using a neural rendering algorithm.

[0088] In some implementations, the model data generation module 702 is specifically used to: generate three-dimensional model representation data of the texture to be generated using a preset 3DGAN network based on the received model information;

[0089] In some implementations, the surface texture acquisition module 704 is specifically used to: use a preset 2DGAN network to acquire the surface texture corresponding to the three-dimensional model representation data.

[0090] The texture device for a three-dimensional model provided in the embodiments of the present disclosure can execute the texture method for a three-dimensional model provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0092] An embodiment of the present disclosure provides an electronic device, which includes: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of a texture generation method for a three-dimensional model or the steps of a generation method for a three-dimensional model.

[0093] Reference below Figure 8 , which shows a schematic diagram of the structure of an electronic device 800 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0094] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0095] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0096] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0097] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to perform the image processing method provided by the embodiments of the present disclosure. The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present disclosure, the programming languages ​​including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0098] In addition, the embodiments of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the method provided by the embodiments of the present disclosure.

[0099] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0100] The embodiments of the present disclosure also provide a computer program product, including a computer program / instruction, which implements the method in the embodiments of the present disclosure when executed by a processor.

[0101] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0102] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0103] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0104] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0105] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0106] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A texture generation method for a three-dimensional model, characterized in that: include: Acquire three-dimensional model representation data of the texture to be generated; generating a plurality of surface local regions of a target three-dimensional model based on the three-dimensional model representation data; Acquire features corresponding to the local area of ​​the surface to generate a texture map corresponding to the local area of ​​the surface based on the features; Based on the texture maps corresponding to the multiple surface local areas, the surface texture of the target three-dimensional model is obtained.

2. The method according to claim 1, characterized in that The acquiring the feature corresponding to the local area of ​​the surface includes: Determine a plurality of sub-units in the local area of ​​the surface, and obtain unit characteristics of the sub-units; The feature corresponding to the local area of ​​the surface is determined according to the unit features of the plurality of sub-units.

3. The method according to claim 2, characterized in that The three-dimensional model representation data is implemented based on three orthogonal feature planes, and the obtaining of the unit features of the subunits includes: The subunits are projected onto each of the characteristic planes respectively; and the unit characteristics of the subunits are determined based on the projection results of the subunits onto each of the characteristic planes.

4. The method according to claim 1, characterized in that: The step of generating a texture map corresponding to the local area of ​​the surface based on the feature map comprises: Based on the features corresponding to the plurality of local surface areas, a texture map corresponding to each of the plurality of local surface areas is determined through a preset neural network.

5. The method according to claim 1, characterized in that The combination of the multiple surface local areas covers the entire surface of the three-dimensional model.

6. The method according to claim 1, characterized in that The shapes and sizes of different surface local regions are the same; or, among the plurality of surface local regions, there are surface local regions with different shapes and / or sizes.

7. A method for generating a three-dimensional model, characterized in that: include: Based on the received model information, generating three-dimensional model representation data of the texture to be generated; Acquire the surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained by using the texture generation method of the three-dimensional model according to any one of claims 1 to 6; A target three-dimensional model is generated based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

8. The method according to claim 7, characterized in that The three-dimensional model representation data corresponding to the model information is realized based on three orthogonal characteristic planes.

9. The method according to claim 8, characterized in that The step of generating three-dimensional model representation data of a texture to be generated based on the received model information includes: Based on the received model information, a preset generation network is used to generate three-dimensional model representation data of the texture to be generated.

10. The method according to claim 9, characterized in that The generative network is trained according to the following steps: Acquire a model information sample and a basic image of a target perspective corresponding to the three-dimensional model representation data of the model information sample; the basic image includes one or more of a depth map, a normal map, and a mask map; Acquire three-dimensional model representation data of the texture to be generated output by a preset initial network for the model information sample, and acquire a basic image of the target perspective corresponding to the three-dimensional model representation data output by the initial network; According to the difference between the basic image corresponding to the three-dimensional model sample and the basic image corresponding to the initial network, the parameters of the initial network are adjusted to obtain the generated network.

11. The method according to claim 10, characterized in that The step of obtaining the basic image under the target perspective corresponding to the three-dimensional model representation data output by the initial network includes: Based on the three orthogonal feature planes, a neural rendering algorithm is used to obtain a basic image under the target perspective corresponding to the three-dimensional model representation data output by the initial network.

12. The method according to claim 7, characterized in that The step of generating three-dimensional model representation data of a texture to be generated based on the received model information includes: Based on the received model information, a preset 3DGAN network is used to generate three-dimensional model representation data of the texture to be generated; The obtaining of the surface texture corresponding to the three-dimensional model representation data includes: The surface texture corresponding to the three-dimensional model representation data is obtained using a preset 2DGAN network.

13. A texture generation device for a three-dimensional model, characterized in that: include: A model data acquisition module, used to acquire three-dimensional model representation data of the texture to be generated; A local area generation module, used to generate a plurality of surface local areas of a target three-dimensional model based on the three-dimensional model representation data; A texture map generation module, used for acquiring features corresponding to the local area of ​​the surface, so as to generate a texture map corresponding to the local area of ​​the surface based on the feature map; The surface texture generation module is used to obtain the surface texture of the target three-dimensional model based on the texture maps corresponding to the multiple surface local areas.

14. A three-dimensional model generation device, characterized in that: include: A model data generation module, used to generate three-dimensional model representation data of the texture to be generated based on the received model information; A surface texture acquisition module, used to acquire the surface texture corresponding to the three-dimensional model representation data; wherein the surface texture is obtained by using the texture generation method of the three-dimensional model according to any one of claims 1 to 6; The target model generation module is used to generate a target three-dimensional model based on the three-dimensional model representation data and the surface texture corresponding to the three-dimensional model representation data.

15. An electronic device, characterized in that: The electronic device comprises: a storage device having a computer program stored thereon; A processing device, used to execute the computer program in the storage device to implement the steps of the texture generation method of the three-dimensional model described in any one of claims 1-6 or the steps of the generation method of the three-dimensional model described in any one of claims 7-12.

16. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the steps of the texture generation method of the three-dimensional model described in any one of claims 1 to 6 or the steps of the generation method of the three-dimensional model described in any one of claims 7 to 12.

Images