Material map generation method, model training method, device, equipment and medium
By splitting the material map generation task into two subtasks: background map and texture map, and using a generative model trained with label samples, the problem of insufficient fineness of material maps in the existing technology is solved, and high-precision materials are achieved Map generation meets the needs of realistic scenes.
Patent Information
- Application Number
- CN202311408797.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-26
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-10-26
AI Technical Summary
The 3D model material map generated by the existing material generation tools is poor in detail and cannot meet the application requirements of realistic scenes.
Split the material map generation task into two subtasks: background map and texture map. The background map generation model and texture map generation model trained on multiple labeled samples are used to generate high-precision target material maps through fusion.
It improves the precision of material maps, meets the application requirements of realistic scenes, reduces manual drawing time, and improves production efficiency.
Smart Images

Figure CN117495995B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to computer technology, and in particular to a material map generation method, model training method, device, equipment and medium. Background Art
[0002] With the continuous development of the entertainment industry, movies, TV series, cartoons, games and other entertainment activities occupy more and more of people's leisure time, and the demand for related three-dimensional (3D) content and 3D assets is also increasing.
[0003] The production of material maps for 3D models has always been an important process in the 3D model production process, which directly affects the interaction between the model and light during the final rendering. At present, the material maps of 3D models are mainly generated through material generation tools. However, due to the poor precision of the material maps generated by material generation tools, they often cannot meet the application requirements of realistic scenes. Summary of the invention
[0004] The present application provides a material map generation method, model training method, device, equipment and medium to solve the problem that the material maps generated by existing material generation tools have poor fineness and cannot meet the application requirements of realistic scenes.
[0005] In a first aspect, the present application provides a method for generating a material map, comprising:
[0006] Acquire material description information of the 3D white model, wherein the material description information includes background color information and texture information;
[0007] Inputting the background color information into a background color map generation model, and obtaining a background color map output by the background color map generation model, wherein the background color map generation model is trained based on a plurality of labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map;
[0008] Inputting the texture information into a texture map generation model to obtain a texture map output by the texture map generation model, wherein the texture map generation model is trained based on a plurality of labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map;
[0009] The base color map and the texture map are merged to generate a target material map of the 3D white model.
[0010] In a possible implementation, the fusing the base color map and the texture map to generate a target material map for the 3D white model includes:
[0011] The base color map is used as a bottom layer map, the texture map is used as an upper layer map, the base color map and the texture map are merged to generate the target material map.
[0012] In a possible implementation, the base color map is used as a bottom layer map, the texture map is used as an upper layer map, and the base color map and the texture map are merged to generate the target material map, including:
[0013] The target material map is generated by formula (1): f1(A1, B1) = α × A1 + (1-α) × B1; wherein f1(A1, B1) is the target material map, A1 is the bottom layer map, B1 is the top layer map, and α is the transparency;
[0014] or,
[0015] According to formula (2): Generate the target material map; wherein f2(A2, B2) is the target material map, A2 is the normalized bottom layer map, and B2 is the normalized upper layer map.
[0016] In a possible implementation, the material description information of the 3D white mold further includes damage information. After acquiring the material description information of the 3D white mold, the method further includes:
[0017] Inputting the damage information into a damage map generation model to obtain a damage map output by the damage map generation model, wherein the damage map generation model is trained based on a plurality of labeled sample damage maps, and the label of each sample damage map is used to characterize the damage information of the sample damage map;
[0018] Accordingly, the base color map and the texture map are merged to generate the target material map of the 3D white model, including:
[0019] The base color map, the texture map and the damage map are merged to generate a target material map of the 3D white model.
[0020] In a possible implementation, the material description information of the 3D white model further includes dirt information. After acquiring the material description information of the 3D white model, the method further includes:
[0021] Inputting the dirt information into a dirt map generation model to obtain a dirt map output by the dirt map generation model, wherein the dirt map generation model is trained based on a plurality of labeled sample dirt maps, and the label of each sample dirt map is used to characterize the dirt information of the sample dirt map;
[0022] Accordingly, the base color map, the texture map and the damage map are merged to generate a target material map of the 3D white model, including:
[0023] The base color map, the texture map, the damage map and the dirt map are merged to generate a target material map of the 3D white model.
[0024] In a possible implementation, the fusing the base color map, the texture map, the damage map, and the dirt map to generate a target material map for the 3D white model includes:
[0025] Using the base color map as a bottom layer map and the texture map as an upper layer map, the base color map and the texture map are merged to generate a first material map;
[0026] Using the first texture map as a bottom texture and the damaged texture map as an upper texture, the first texture map and the damaged texture map are merged to generate a second texture map;
[0027] The second texture map is used as a bottom layer texture, the dirt map is used as an upper layer texture, and the second texture map and the dirt map are merged to generate the target texture map.
[0028] In a second aspect, the present application provides a model training method, comprising:
[0029] Acquire a background color training set, wherein the background color training set includes a plurality of labeled sample background color maps, wherein the label of each sample background color map is used to characterize the background color information of the sample background color map;
[0030] Model training is performed according to the background color training set to obtain a background color map generation model, where the background color map generation model is used to generate a background color map of the 3D white model according to the background color information of the 3D white model.
[0031] In a third aspect, the present application provides a model training method, comprising:
[0032] Acquire a texture training set, wherein the texture training set includes a plurality of labeled sample texture maps, wherein the label of each sample texture map is used to characterize texture information of the sample texture map;
[0033] Model training is performed according to the texture training set to obtain a texture map generation model, where the texture map generation model is used to generate a texture map of the 3D white model according to the texture information of the 3D white model.
[0034] In a fourth aspect, the present application provides a model training method, comprising:
[0035] Acquire a damage training set, wherein the damage training set includes sample damage maps with labels, and the label of each sample damage map is used to represent damage information of the sample damage map;
[0036] Model training is performed according to the damage training set to obtain a damage map generation model, where the damage map generation model is used to generate a damage map of the 3D white model according to damage information of the 3D white model.
[0037] In a fifth aspect, the present application provides a model training method, comprising:
[0038] Obtain a dirt training set, wherein the dirt training set includes sample dirt maps with labels, and the label of each sample dirt map is used to characterize dirt information of the sample dirt map;
[0039] Model training is performed according to the dirt training set to obtain a dirt map generation model, where the dirt map generation model is used to generate a dirt map of the 3D white model according to dirt information of the 3D white model.
[0040] In a sixth aspect, the present application provides a device for generating a material map, comprising:
[0041] An acquisition module, used to acquire material description information of the 3D white model, wherein the material description information includes background color information and texture information;
[0042] An input module, used to input the background color information into a background color map generation model, and obtain a background color map output by the background color map generation model, wherein the background color map generation model is trained based on a plurality of labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map;
[0043] The input module is further used to input the texture information into a texture map generation model to obtain a texture map output by the texture map generation model, wherein the texture map generation model is trained based on a plurality of labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map;
[0044] A fusion module is used to fuse the base color map and the texture map to generate a target material map of the 3D white model.
[0045] In a possible implementation, the fusion module is specifically used to:
[0046] The base color map is used as a bottom layer map, the texture map is used as an upper layer map, the base color map and the texture map are merged to generate the target material map.
[0047] In a possible implementation, the fusion module is specifically configured to:
[0048] The target material map is generated by formula (1): f1(A1, B1) = α × A1 + (1-α) × B1; wherein f1(A1, B1) is the target material map, A1 is the bottom layer map, B1 is the top layer map, and α is the transparency;
[0049] or,
[0050] According to formula (2): Generate the target material map; wherein f2(A2, B2) is the target material map, A2 is the normalized bottom layer map, and B2 is the normalized upper layer map.
[0051] In a possible implementation, the material description information of the 3D white model further includes damage information. After acquiring the material description information of the 3D white model, the input module is further configured to:
[0052] Inputting the damage information into a damage map generation model to obtain a damage map output by the damage map generation model, wherein the damage map generation model is trained based on a plurality of labeled sample damage maps, and the label of each sample damage map is used to characterize the damage information of the sample damage map;
[0053] Accordingly, the fusion module is specifically used for:
[0054] The base color map, the texture map and the damage map are merged to generate a target material map of the 3D white model.
[0055] In a possible implementation, the material description information of the 3D white model further includes dirt information. After acquiring the material description information of the 3D white model, the input module is further configured to:
[0056] Inputting the dirt information into a dirt map generation model to obtain a dirt map output by the dirt map generation model, wherein the dirt map generation model is trained based on a plurality of labeled sample dirt maps, and the label of each sample dirt map is used to characterize the dirt information of the sample dirt map;
[0057] Accordingly, the fusion module is specifically used for:
[0058] The base color map, the texture map, the damage map and the dirt map are merged to generate a target material map of the 3D white model.
[0059] In a possible implementation, the fusion module is specifically configured to:
[0060] Using the base color map as a bottom layer map and the texture map as an upper layer map, the base color map and the texture map are merged to generate a first material map;
[0061] Using the first texture map as a bottom texture and the damaged texture map as an upper texture, the first texture map and the damaged texture map are merged to generate a second texture map;
[0062] The second texture map is used as a bottom layer texture, the dirt map is used as an upper layer texture, and the second texture map and the dirt map are merged to generate the target texture map.
[0063] In a seventh aspect, the present application provides a model training device, comprising:
[0064] A first training module, a second training module, a third training module and a fourth training module;
[0065] The first training module is used to execute the method described in the second aspect;
[0066] The second training module is used to execute the method described in the third aspect;
[0067] The third training module is used to execute the method described in the fourth aspect;
[0068] The fourth training module is used to execute the method described in the fifth aspect.
[0069] In an eighth aspect, the present application provides an electronic device, including:
[0070] at least one processor; and
[0071] a memory communicatively coupled to the at least one processor;
[0072] In which, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the methods shown in the first aspect, the second aspect, the third aspect, the fourth aspect, the fifth aspect and each possible implementation method.
[0073] In a ninth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the method shown in the first aspect, the second aspect, the third aspect, the fourth aspect, the fifth aspect and each possible implementation method is implemented.
[0074] The material map generation method, model training method, device, equipment and medium provided by the present application, in the material map generation method, by obtaining the material description information of the 3D white model, the material description information includes background color information and texture information, the background color information is input into the background color map generation model, the background color map output by the background color map generation model is obtained, and the texture information is input into the texture map generation model, and the texture map output by the texture map generation model is obtained. Finally, the background color map and the texture map are fused to generate the target material map of the 3D white model. Among them, the background color map generation model is obtained based on the training of multiple labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map, and the texture map generation model is obtained based on the training of multiple labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map. In this technical solution, the task of generating material maps is split into two subtasks of generating base color maps and generating texture maps. Since the parameters of the base color map generation model and the texture map generation model are fewer and more controllable, it is possible to generate base color maps and texture maps with higher precision, so that the texture map obtained by fusing the base color map and the texture map is more refined, meeting the application requirements of realistic scenes. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0076] Figure 1 A schematic diagram of a 3D modeling scene;
[0077] Figure 2 A schematic diagram of a process flow of a first embodiment of a method for generating a material map provided in an embodiment of the present application;
[0078] Figure 3 A schematic diagram of a flow chart of a second embodiment of a method for generating a material map provided in an embodiment of the present application;
[0079] Figure 4 A schematic diagram of a 3D model with a material map provided in an embodiment of the present application;
[0080] Figure 5 A flowchart of the first embodiment of the model training method provided in the embodiment of the present application;
[0081] Figure 6 A flowchart of the second embodiment of the model training method provided in the embodiment of the present application;
[0082] Figure 7 A flowchart of the third embodiment of the model training method provided in the embodiment of the present application;
[0083] Figure 8 A flowchart of a fourth embodiment of the model training method provided in the embodiment of the present application;
[0084] Fig. 9 A schematic diagram of the structure of a device for generating a material map provided in an embodiment of the present application;
[0085] Fig.10 A schematic diagram of the structure of a first embodiment of a model training device provided in an embodiment of the present application;
[0086] Fig.11 A schematic diagram of the structure of a second embodiment of a model training device provided in an embodiment of the present application;
[0087] Fig.12 A schematic diagram of the structure of a third embodiment of a model training device provided in an embodiment of the present application;
[0088] Fig.13 A schematic diagram of the structure of a fourth embodiment of a model training device provided in an embodiment of the present application;
[0089] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0090] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0091] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0092] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0093] In order to clearly describe the technical solutions of the embodiments of the present application, the terms involved in the present application are first explained.
[0094] Physically Based Rendering (PBR): A modern rendering technology that simulates the behavior of light based on physical laws to achieve more realistic rendering effects.
[0095] Texture Map: A two-dimensional image used to simulate the surface details of an object. It may contain color information, lighting information, reflectivity information, etc., and is used to render the appearance of an object. The material mentioned in this application may refer to a combination of material and texture. Exemplarily, the material may be any one of wood, plastic, leather, etc. The texture may be a visual representation of various properties of an object, such as color, texture, transparency, reflectivity, roughness, etc. The material and texture of the same material map are the same. Two material maps with different materials and / or textures are two different material maps.
[0096] Color Map: A map that contains the color information of the surface of an object. It defines the colors of different parts of the object and is used to color the object.
[0097] Normal Map: A map used to simulate the surface details of an object. It simulates the concave and convex surface of an object by changing the normal direction of each pixel, thereby increasing the details and realism of the object.
[0098] Roughness Map: A map used to simulate the roughness of an object's surface. It defines the smoothness of different areas on the surface of an object and is used to simulate the reflected light effects of different materials.
[0099] White model: Model without matching material map.
[0100] Stable Diffusion (SD): A deep learning text-to-image generation model. It is mainly used to generate detailed images based on text descriptions, and can also be applied to other tasks such as infill rendering, outfill rendering, and image-to-image translation under the guidance of prompt words.
[0101] Next, the application scenarios involved in this application are explained.
[0102] With the continuous development of 3D technology, more and more Internet applications are presented to users in 3D, including online video, e-reading, online games, virtual communities, e-commerce, and distance education. Even for the tourism industry, scenic spots, sculptures, and antiques can be presented to users in 3D. 3D technology is also widely used in virtual shooting and other fields, such as providing virtual scenes for virtual shooting. The virtual scenes are rendered to the screen and can be used as shooting backgrounds. After being shot together with the real scenes and actors in front of the screen, a virtual and real fusion shooting picture can be obtained, which reduces the workload of post-production and improves shooting efficiency and effect. Compared with 2D models, 3D models can bring users a more realistic, immersive, and shocking experience.
[0103] For example, Figure 1 is a schematic diagram of a 3D modeling scene. Figure 1 As shown, the electronic device can respond to the modeling operation triggered by the user and build a 3D white model through the 3D modeling software. Then, the electronic device can assign a material map to the 3D white model through the modeling software so that the 3D model has visual expressions such as color, texture, roughness, etc. corresponding to the material map.
[0104] The production of material maps for 3D models has always been an important step in the 3D model production process, which directly affects the interaction between the 3D model and light during the final rendering.
[0105] At present, the material mapping of 3D models is mainly achieved by manual drawing by modelers. However, manual production takes a lot of time. The production time of material mapping for film-level 3D models may be equal to or even longer than that of white models, and the production efficiency is low.
[0106] Taking the construction of 3D ancient costume scenes as an example, there are a lot of ancient building-related model construction work in 3D ancient costume scenes, such as many repeated tiles, rafters, pillars, beams and other components. If each independent 3D model is asked to be manually drawn by the modeler, then each building of a specific style will consume a lot of time in making material maps.
[0107] In order to solve the above technical problems, the prior art also proposes a method for generating a material map of a 3D model based on a material generation tool. The existing material generation tool can generate a material map corresponding to the 3D white model according to input description information related to the 3D white model material.
[0108] However, existing material generation tools involve a large number of complex and adjustable parameters, which results in poor controllability of the material map generation process. The generated material maps have low precision and realism, and cannot meet the application requirements of realistic scenes.
[0109] Based on the above technical problems, the technical concept of the present application is as follows: Since the existing material generation tools involve multiple parameters, the material generation tools are too complicated and cannot generalize data well. When studying related fields, the inventors found that the parameters in the material generation tools can be divided into four categories, namely parameters related to the background color, parameters related to the texture, parameters related to the damage, and parameters related to the dirt. If the task for generating the material map is split into 4 sub-map generation tasks according to the type of parameters, the background color map, texture map, damage map and dirt map of the 3D white model are generated by these 4 sub-map generation tasks, and then the material map of the 3D white model only needs to be merged by these 4 maps. Since the parameters of each sub-map generation task are simple and the effect is controllable, the fineness of the output map of each sub-map generation task can be effectively guaranteed, thereby ensuring the fineness of the material map and meeting the application requirements of realistic scenes.
[0110] It should be understood that the present application does not limit the application scenarios of the method for generating a texture map. For example, the method for generating a texture map can be used in modeling services as part of the modeling process. Figure 1 The scene shown. Alternatively, the texture map can also be used to render the 2D model framework to obtain a 2D model. Alternatively, the above-mentioned method for generating the texture map can also be used in any scenario other than the modeling service where the texture map can be used. For example, the method for generating the texture map can be applied to provide a material map service. The material map service can output a material map that matches the material of the target image provided by the user.
[0111] The technical solution of the present application is described in detail below through specific embodiments.
[0112] It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0113] Figure 2 This is a flow chart of a method for generating a material map according to an embodiment of the present application. Figure 2 As shown, the method for generating the material map may include the following steps:
[0114] S21. Obtain material description information of the 3D white model.
[0115] The execution subject of the embodiment of the present application is an electronic device, which can be a terminal device, such as a mobile phone, a desktop computer, a notebook, etc., or a server. In practical applications, whether the electronic device is a terminal device or a server can be determined according to actual conditions, and the embodiment of the present application does not specifically limit this.
[0116] In this step, the material description information includes background color information and texture information. For example, the background color information can be white, black, purple and other related colors. The texture information can be a specific texture, such as wood grain, marble grain, pattern, etc., and can also be the name of the 3D white model, such as wood, pottery, stone, granite, etc.
[0117] In a possible implementation, the material description information of the 3D white model input by the user may be received through an application programming interface (Application Programming Interface, API) or a graphical user interface (Graphical User Interface, GUI).
[0118] In another possible implementation manner, the material description information of the 3D white model may be obtained from a server or a database storing the material description information of the 3D white model.
[0119] In another possible implementation, the material description information of the 3D white mold may also be data pre-stored in a local storage space of the electronic device, and the electronic device may directly obtain the material description information of the 3D white mold from the local storage space.
[0120] Furthermore, the material description information of the 3D white mold can be directly obtained through the above three methods, or the initial material description information of the 3D white mold can be first obtained and further parsed to obtain the material description information of the 3D white mold. It should be understood that the method of obtaining the initial material description information of the 3D white mold can refer to the above three methods of obtaining the material description information of the 3D white mold, which will not be repeated here.
[0121] Exemplarily, the initial material description information of the 3D white model includes the material subclass of the 3D white model, the humidity of the environment, the newness, the use traces, etc. After the initial material description information of the 3D white model is obtained, the initial material description information of the 3D white model can be parsed to extract the background color information related to the background color and the texture information related to the texture from the initial material description information of the 3D white model, thereby obtaining the material description information of the 3D white model.
[0122] S22, inputting the background color information into the background color map generation model, and obtaining the background color map output by the background color map generation model.
[0123] In this step, the base color map generation model is trained based on multiple labeled sample base color maps, and the label of each sample base color map is used to characterize the base color information of the sample base color map. The base color map generation model contains a small number of model parameters related to the base color, and the effect is more controllable, and a more detailed base color map can be generated according to the base color information.
[0124] The base color map also includes a corresponding sub-roughness map, a sub-color map, and a sub-normal map.
[0125] S23, inputting texture information into a texture map generation model, and obtaining a texture map output by the texture map generation model.
[0126] In this step, the texture map generation model is trained based on multiple labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map. The texture map generation model contains a small number of model parameters related to texture, and the effect is more controllable, and a more detailed texture map can be generated according to the texture information.
[0127] The texture map also includes a corresponding sub-roughness map, a sub-color map, and a sub-normal map.
[0128] It should be understood that there is no sequential execution order for S22 and S23. That is, S22 may be executed first and then S23, or S23 may be executed first and then S22, or S22 and S23 may be executed simultaneously.
[0129] S24, the base color map and the texture map are merged to generate a target material map of the 3D white model.
[0130] In this step, after obtaining the base color map and the texture map, they need to be merged based on the actual situation of the object to obtain the target material map of the 3D white model.
[0131] In real life, the texture of an object is usually on top of the base color. Therefore, when fusing the base color map and the texture map, the base color map needs to be used as the bottom layer map and the texture map as the top layer map, so as to fuse the base color map and the texture map to generate the target material map. In this way, the generated target material map will be closer to the real life scene and improve the realism of the target material map.
[0132] In one possible manner, the target material map can be generated by formula (1): f1(A1, B1) = α×A1+(1-α)×B1; wherein f1(A1, B1) is the target material map, A1 is the bottom layer map, B1 is the top layer map, and α is the transparency.
[0133] In this method, the transparency can be determined according to the material description information of the 3D white model, can be preset by the user, or can be set in response to the user's setting operation after the base color map and the texture map are generated. By adjusting the transparency of the base color map and the texture map, the generated target material map is further made to be more in line with the actual situation.
[0134] In another possible way, by formula (2): Generate the target material map; where f2(A2,B2) is the target material map, A2 is the normalized bottom layer map, and B2 is the normalized top layer map
[0135] It should be understood that in actual applications, the base color map and the texture map can be fused by any of the above methods, or by using both of the above methods at the same time to obtain the target material map corresponding to the two methods for the user to choose. The specific method used to fuse the base color map and the texture map can be pre-set or determined according to the material description information of the 3D white model, and this is not specifically limited.
[0136] Furthermore, when fusing the base color map and the texture map, the sub-roughness maps, sub-color maps and sub-normal maps of the two can be fused separately to obtain the fused target roughness map, target color map and target normal map. The target roughness map, target color map and target normal map are the target material maps of the 3D white model.
[0137] Taking the sub-roughness map as an example, the sub-roughness map of the base color map can be used as the bottom layer map, and the sub-roughness map of the texture map can be used as the top layer map. The sub-roughness map of the base color map and the sub-roughness map of the texture map are merged to generate a target roughness map.
[0138] It should be understood that the fusion process of the sub-color map and the sub-normal map can refer to the fusion process of the sub-roughness map mentioned above, which will not be repeated here.
[0139] It should be understood that the sub-roughness map of the base color map and the sub-roughness map of the texture map can be merged by the above formula (1) or formula (2) to obtain the target roughness map. The specific principle and implementation process can refer to the relevant content of formula (1) or formula (2), which will not be repeated here.
[0140] Among them, the target material map is the entire set of basic PBR material maps for the 3D white model.
[0141] The method for generating a material map provided in an embodiment of the present application obtains material description information of a 3D white model, wherein the material description information includes background color information and texture information, inputs the background color information into a background color map generation model, obtains the background color map output by the background color map generation model, and inputs the texture information into a texture map generation model, obtains the texture map output by the texture map generation model. Finally, the background color map and the texture map are fused to generate a target material map of the 3D white model. The background color map generation model is obtained by training based on a plurality of labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map. The texture map generation model is obtained by training based on a plurality of labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map. In this technical solution, the task of generating material maps is split into two subtasks of generating base color maps and generating texture maps. Since the parameters of the base color map generation model and the texture map generation model are fewer and more controllable, it is possible to generate base color maps and texture maps with higher precision, so that the texture map obtained by fusing the base color map and the texture map is more refined, meeting the application requirements of realistic scenes.
[0142] Based on the above embodiments, in real life, objects may be damaged during use. Therefore, the material description information of the 3D white model may also include damage information. When the material description information of the 3D white model includes damage information, after obtaining the material description information of the 3D white model, the damage information may be input into a damage map generation model to obtain a damage map output by the damage map generation model. The damage map generation model is obtained by training based on multiple labeled sample damage maps, and the label of each sample damage map is used to characterize the damage information of the sample damage map.
[0143] It should be understood that the base color map and texture map Figure 1 Similarly, the damage map also includes the corresponding sub-roughness map, sub-color map and sub-normal map.
[0144] In this way, after the damage map is generated, the base color map, texture map and damage map can be fused to generate the target material map of the 3D white model.
[0145] Specifically, since the damage of an object is generally generated during use, the damage map should be above the base color map and the texture map. The base color map can be used as the bottom layer map and the texture map as the top layer map, so as to fuse the base color map and the texture map to generate a first material map. Afterwards, the first material map is used as the bottom layer map and the damage image is used as the top layer map, so as to fuse the first material map and the damage map, thereby obtaining the target material map of the 3D white model.
[0146] It should be understood that the above-mentioned method of fusing the bottom layer map and the upper layer map can refer to formula (1) and formula (2) in S24 and will not be repeated here.
[0147] In the base color map, texture map and damage map Figure 3 In the process of fusing the three, the sub-roughness map, sub-color map and sub-normal map of the three can be fused respectively, so as to obtain the fused target roughness map, target color map and target normal map, and the target roughness map, target color map and target normal map are the target material map of the 3D white model. The specific fusion process can refer to the relevant content in S24, which will not be repeated here.
[0148] The damage map generation model is used to obtain the damage map of the 3D white model, thereby generating a texture map with a damage effect, making the texture map closer to life and meeting the requirements of realistic scenes.
[0149] Based on any of the above embodiments, in real life, objects may still be dirty during use. Therefore, the material description information of the 3D white model also includes dirt information. After obtaining the material description information of the 3D white model, the dirt information may be input into a dirt map generation model to obtain a dirt map output by the dirt map generation model. The dirt map generation model is obtained by training based on a plurality of labeled sample dirt maps, and the label of each sample dirt map is used to characterize the dirt information of the sample dirt map.
[0150] With base color map, texture map and damage map Figure 1 Similarly, a dirt map also includes a corresponding sub-roughness map, sub-color map, and sub-normal map.
[0151] In this way, after generating the dirt map, the base color map, texture map and dirt map can be fused to generate a target material map of the 3D white model, so that the generated target material map has a dirty effect and is more in line with real life.
[0152] Specifically, since dirt on an object is generally generated during use, the dirt map should be above the base color map and the texture map. The base color map can be used as the bottom layer map and the texture map as the top layer map, thereby fusing the base color map and the texture map to generate a first material map. Afterwards, the first material map is used as the bottom layer map and the dirt image is used as the top layer map, thereby fusing the first material map and the dirt map to obtain the target material map of the 3D white model.
[0153] It should be understood that the above-mentioned method of fusing the bottom layer map and the upper layer map can refer to formula (1) and formula (2) in S24 and will not be repeated here.
[0154] In the base color map, texture map and dirt map Figure 3 In the process of fusing the three, the sub-roughness map, sub-color map and sub-normal map of the three can be fused respectively, so as to obtain the fused target roughness map, target color map and target normal map, and the target roughness map, target color map and target normal map are the target material map of the 3D white model. The specific fusion process can refer to the relevant content in S24, which will not be repeated here.
[0155] Optionally, in actual applications, since dirt and damage are caused by users using items, they usually exist on the surface of objects at the same time. Therefore, the base color map, texture map, damage map and dirt map can be merged to generate a target material map of the 3D white model, so that the target material map is closer to the actual life situation and the use traces of the object are more realistic.
[0156] Specifically, the base color map is used as the bottom layer map, the texture map is used as the top layer map, the base color map and the texture map are merged to generate a first material map; the first material map is used as the bottom layer map, the damage map is used as the top layer map, the first material map and the damage map are merged to generate a second material map; the second material map is used as the bottom layer map, the dirt map is used as the top layer map, the second material map and the dirt map are merged to generate a target material map.
[0157] It should be understood that the above-mentioned method of fusing the bottom layer map and the upper layer map can refer to formula (1) and formula (2) in S24 and will not be repeated here.
[0158] When applying base color maps, texture maps, damage maps, and dirt maps Figure 4 In the process of fusing the four, the sub-roughness maps, sub-color maps and sub-normal maps of the four can be fused respectively, so as to obtain the fused target roughness map, target color map and target normal map, and the target roughness map, target color map and target normal map are the target material maps of the 3D white model. The specific fusion process can refer to the relevant content in S24, which will not be repeated here.
[0159] It should be understood that the model you want to use can be flexibly selected according to the actual situation, and the control of the target texture mapping effect can be achieved by fusing the maps output by different models, which is highly flexible. Moreover, there is no execution order between multiple models, and they can run in parallel, thus effectively improving the efficiency of texture mapping generation.
[0160] Optionally, the target material map generated based on any of the above embodiments is a 2D texture map of the 3D white model. Further, the target material map can also be mapped back to the 3D white model.
[0161] Based on the method for generating a material map provided in any of the above embodiments, a specific example is used to explain it.
[0162] Figure 3 This is a flow chart of a second embodiment of a method for generating a material map provided in an embodiment of the present application. Figure 3 As shown, the method for generating the material map may include the following steps:
[0163] Step 1: Build a 3D white mold of known materials.
[0164] Step 2: Get the material description information of the 3D white model.
[0165] The material description information of the 3D white model includes background color information, texture information, damage information and dirt information.
[0166] In a possible implementation, the electronic device may display multiple options to the user through a browser or an application, and the multiple options may be material sub-category, environmental humidity, newness, and use traces. The user may set each option according to the texture requirements of the 3D white model. The electronic device obtains the material description information of the 3D white model in response to the user's setting operation of the multiple options.
[0167] Among them, the options related to the background color are: material subcategory, environmental humidity and condition; the options related to the texture are: material subcategory; the options related to damage are: condition and signs of use; the options related to dirt are: environmental humidity and condition.
[0168] Step 3: Input the background color information into the background color map generation model to obtain the background color map output by the background color map generation model; input the texture information into the texture map generation model to obtain the texture map output by the texture map generation model; input the damage information into the damage map generation model to obtain the damage map output by the damage map generation model; input the dirt information into the dirt map generation model to obtain the dirt map output by the dirt map generation model.
[0169] Step 4: Merge the base color map, texture map, damage map and dirt map to generate the target material map of the 3D white model.
[0170] Step 5: Post-process the target material map and assign the processed target material map to the 3D white model.
[0171] Exemplarily, the post-processing may include determining a projection method, adjusting the textures of the target material maps at different viewing angles to be consistent, and removing texture seams of the target material maps at different viewing angles.
[0172] For example, Figure 4 This is a schematic diagram of a 3D model with a material map provided in an embodiment of the present application. Figure 4 As shown, the target material map includes a target roughness map, a target color map and a target normal map, and the target material map can be generated based on the relevant content in any of the above embodiments. The target material map is assigned to the 3D white model, and then rendered to obtain a 3D model with the material map.
[0173] The material map generation method provided in any of the above embodiments can effectively improve the fineness of the texture map, reduce the time spent on manually drawing the texture map, and improve the production efficiency and scene building efficiency.
[0174] Before using the above-mentioned background color map generation model, texture map generation model, damage map generation model, and dirt map generation model, it is necessary to perform model training on them to optimize their model parameters. The model training process is described in detail below in conjunction with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.
[0175] In specific implementation, the execution subject of the model training method can also be an electronic device with processing capabilities such as a terminal device or a server. It should be understood that the electronic device that executes the material map generation method and the electronic device that executes the model training method can be the same device or different devices.
[0176] Figure 5 The flowchart of the first embodiment of the model training method provided in the embodiment of the present application is as follows. Figure 5 As shown, the material model training method may include the following steps:
[0177] S51. Obtain background color training set.
[0178] In this step, the background color training set includes a plurality of labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map.
[0179] The sample background color map may be obtained from a server or database storing the sample background color map, and in response to the user's labeling operation on the sample background color map, the label of the sample background color map is determined, thereby generating a labeled sample background color map. The labeled sample background color map may also be obtained from a server or database storing the sample background color map. It should be understood that the labeled sample background color map may also be stored in the user's local storage space, and the electronic device may directly obtain the labeled sample background color map from the local storage space.
[0180] It should be understood that this embodiment does not limit the number of sample background color maps included in the background color training set, which can be determined according to actual conditions.
[0181] S52, performing model training according to the background color training set, and obtaining a background color map generation model.
[0182] In this step, the image generation model is trained using the background color training set to fine-tune the model parameters in the image generation model, thereby obtaining a background color map generation model that can generate a background color map of a 3D white model based on the background color information of the 3D white model.
[0183] Optionally, internal parameters of the base color map generation model include base color ratio, hue, and saturation.
[0184] Figure 6 The flowchart of the second embodiment of the model training method provided in the embodiment of the present application is as follows. Figure 6 As shown, the material model training method may include the following steps:
[0185] S61. Obtain a texture training set.
[0186] In this step, the texture training set includes a plurality of labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map.
[0187] The sample texture map can be obtained from a server or database storing the sample texture map, and in response to the user's labeling operation on the sample texture map, the label of the sample texture map is determined, thereby generating a labeled sample texture map. The labeled sample texture map can also be obtained from a server or database storing the sample texture map. It should be understood that the labeled sample texture map can also be stored in the user's local storage space, and the electronic device can directly obtain the labeled sample texture map from the local storage space.
[0188] It should be understood that this embodiment does not limit the number of sample texture maps included in the texture training set, which can be determined according to actual conditions.
[0189] S62: Perform model training according to the texture training set to obtain a texture map generation model.
[0190] In this step, the image generation model is trained using a texture training set to fine-tune the model parameters in the image generation model, thereby obtaining a texture map generation model that can generate a texture map of a 3D white model based on the texture information of the 3D white model.
[0191] Optionally, the internal parameters of the texture map generation model include different sub-class texture variation parameters.
[0192] Figure 7 This is a flow chart of the third embodiment of the model training method provided in the present application. Figure 7 As shown, the material model training method may include the following steps:
[0193] S71. Obtain a damaged training set.
[0194] In this step, the damage training set includes labeled sample damage maps, and the label of each sample damage map is used to characterize the damage information of the sample damage map.
[0195] The sample damaged image may be obtained from a server or database storing the sample damaged image, and in response to the user's labeling operation on the sample damaged image, the label of the sample damaged image is determined, thereby generating a labeled sample damaged image. The labeled sample damaged image may also be obtained from a server or database storing the sample damaged image. It should be understood that the labeled sample damaged image may also be stored in the user's local storage space, and the electronic device may directly obtain the labeled sample damaged image from the local storage space.
[0196] It should be understood that this embodiment does not limit the number of sample damage maps included in the damage training set, which can be determined according to actual conditions.
[0197] S72. Perform model training according to the damage training set to obtain a damage map generation model.
[0198] In this step, the image generation model is trained using a damage training set to fine-tune the model parameters in the image generation model, thereby obtaining a damage map generation model that can generate a damage map of a 3D white model based on damage information of the 3D white model.
[0199] Optionally, internal parameters of the damage map generation model include damage type, degree and location.
[0200] Figure 8 This is a flow chart of the fourth embodiment of the model training method provided in the present application. Figure 8 As shown, the material model training method may include the following steps:
[0201] S81. Obtain a dirty training set.
[0202] In this step, the training set includes labeled sample maps, and the label of each sample map is used to characterize the information of the sample map.
[0203] The sample dirt map may be obtained from a server or database storing the sample dirt map, and in response to the user's labeling operation on the sample dirt map, the label of the sample dirt map is determined, thereby generating a labeled sample dirt map. The labeled sample dirt map may also be obtained from a server or database storing the sample dirt map. It should be understood that the labeled sample dirt map may also be stored in the user's local storage space, and the electronic device may directly obtain the labeled sample dirt map from the local storage space.
[0204] It should be understood that this embodiment does not limit the number of sample dirt maps included in the dirt training set, which can be determined according to actual conditions.
[0205] S82, performing model training according to the dirty trace training set to obtain a texture generation model.
[0206] In this step, the image generation model is trained using the dirt training set to fine-tune the model parameters in the image generation model, thereby obtaining a dirt map generation model that can generate a 3D white model map based on the information of the 3D white model.
[0207] Optionally, internal parameters of the dirt map generation model include dirt type, degree and location.
[0208] It should be understood that in the embodiments of the above-mentioned model training method, the image generation models may be the same model or different models.
[0209] It should be understood that the execution subject of the above-mentioned model training method embodiment can be the same electronic device or different electronic devices.
[0210] In the above embodiment, starting from the four directions of background color, texture, damage, and dirt, training sets are respectively obtained in each direction, so that model training is performed based on the training sets, thereby obtaining a background color map generation model, a texture map generation model, a damage map generation model, and a dirt map generation model, laying a foundation for the subsequent generation of texture maps based on the above four models.
[0211] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0212] Fig. 9 This is a schematic diagram of the structure of a device for generating a texture map provided in an embodiment of the present application. Fig. 9 As shown, the material map generating device 90 includes:
[0213] The acquisition module 91 is used to acquire material description information of the 3D white model, where the material description information includes background color information and texture information.
[0214] The input module 92 is used to input the background color information into the background color map generation model to obtain the background color map output by the background color map generation model. The background color map generation model is trained based on multiple labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map.
[0215] The input module 92 is also used to input texture information into the texture map generation model to obtain the texture map output by the texture map generation model. The texture map generation model is trained based on multiple labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map.
[0216] The fusion module 93 is used to fuse the base color map and the texture map to generate a target material map of the 3D white model.
[0217] In a possible implementation, the fusion module 93 is specifically configured to:
[0218] Use the base color map as the bottom layer map and the texture map as the top layer map, merge the base color map and the texture map to generate the target material map.
[0219] In a possible implementation, the fusion module 93 is specifically configured to:
[0220] The target material map is generated by formula (1): f1(A1, B1) = α × A1 + (1-α) × B1; where f1(A1, B1) is the target material map, A1 is the bottom layer map, B1 is the top layer map, and α is the transparency;
[0221] or,
[0222] According to formula (2): Generate the target material map; where f2(A2, B2) is the target material map, A2 is the normalized bottom layer map, and B2 is the normalized top layer map.
[0223] In a possible implementation, the material description information of the 3D white model also includes damage information. After obtaining the material description information of the 3D white model, the input module 92 is further used to:
[0224] The damage information is input into a damage map generation model to obtain a damage map output by the damage map generation model. The damage map generation model is trained based on multiple labeled sample damage maps, and the label of each sample damage map is used to characterize the damage information of the sample damage map.
[0225] Accordingly, the fusion module 93 is specifically used for:
[0226] The base color map, texture map and damage map are merged to generate the target material map of the 3D white model.
[0227] In a possible implementation, the material description information of the 3D white model also includes dirt information. After obtaining the material description information of the 3D white model, the input module 92 is further used to:
[0228] The dirt information is input into a dirt map generation model to obtain a dirt map output by the dirt map generation model. The dirt map generation model is trained based on a plurality of labeled sample dirt maps. The label of each sample dirt map is used to characterize the dirt information of the sample dirt map.
[0229] Accordingly, the fusion module 93 is specifically used for:
[0230] The base color map, texture map, damage map and dirt map are merged to generate the target material map of the 3D white model.
[0231] In a possible implementation, the fusion module 93 is specifically configured to:
[0232] The base color map is used as the bottom layer map, the texture map is used as the top layer map, the base color map and the texture map are merged to generate a first material map.
[0233] The first material map is used as the bottom layer map, the damaged map is used as the top layer map, and the first material map and the damaged map are merged to generate a second material map.
[0234] The second material map is used as the bottom layer map, the dirt map is used as the top layer map, the second material map and the dirt map are merged to generate a target material map.
[0235] The device for generating a texture map provided in the embodiment of the present application can be used to execute the method for generating a texture map in any of the above embodiments. The implementation principle and technical effect are similar and will not be described in detail here.
[0236] Fig.10 This is a schematic diagram of the structure of the first embodiment of the model training device provided in the present application. Fig.10 As shown, the model training device 100 includes a first training module 101.
[0237] The first training module 101 is used for:
[0238] A background color training set is obtained, where the background color training set includes a plurality of labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map.
[0239] The model is trained according to the background color training set to obtain a background color map generation model, where the background color map generation model is used to generate a background color map of the 3D white model according to the background color information of the 3D white model.
[0240] The model training device provided in the embodiment of the present application can be used to execute the model training method on the background color map generation model side in any of the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.
[0241] Fig.11 This is a schematic diagram of the structure of the second embodiment of the model training device provided in the present application. Fig.11 As shown, the model training device 110 includes a second training module 111.
[0242] The second training module 111 is used for:
[0243] A texture training set is obtained, where the texture training set includes a plurality of labeled sample texture maps, and the label of each sample texture map is used to characterize texture information of the sample texture map.
[0244] Model training is performed according to the texture training set to obtain a texture map generation model, where the texture map generation model is used to generate a texture map of the 3D white model according to the texture information of the 3D white model.
[0245] The model training device provided in the embodiment of the present application can be used to execute the model training method on the texture map generation model side in any of the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.
[0246] Fig.12 This is a schematic diagram of the structure of the third embodiment of the model training device provided in the embodiment of the present application. Fig.12 As shown, Fig.12 As shown, the model training device 120 includes a third training module 121.
[0247] The third training module 121 is used for:
[0248] A damage training set is obtained, where the damage training set includes sample damage maps with labels, and the label of each sample damage map is used to characterize the damage information of the sample damage map.
[0249] The model is trained according to the damage training set to obtain a damage map generation model, and the damage map generation model is used to generate a damage map of the 3D white model according to the damage information of the 3D white model.
[0250] The model training device provided in the embodiment of the present application can be used to execute the model training method on the damaged color map generation model side in any of the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.
[0251] Fig.13 This is a schematic diagram of the structure of the fourth embodiment of the model training device provided in the present application. Fig.13 As shown, Fig.13 As shown, Fig.13 As shown, the model training device 130 includes a fourth training module 131 .
[0252] A dirt training set is obtained, where the dirt training set includes sample dirt maps with labels, and the label of each sample dirt map is used to characterize the dirt information of the sample dirt map.
[0253] Model training is performed according to the dirt training set to obtain a dirt map generation model, where the dirt map generation model is used to generate a dirt map of the 3D white model according to the dirt information of the 3D white model.
[0254] The model training device provided in the embodiment of the present application can be used to execute the model training method on the dirt map generation model side in any of the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.
[0255] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.
[0256] Fig.14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.14 As shown, the electronic device of this embodiment may include:
[0257] at least one processor 141; and
[0258] a memory 142 communicatively coupled to the at least one processor;
[0259] The memory 142 stores instructions that can be executed by the at least one processor 141, and the instructions are executed by the at least one processor 141 to enable the electronic device to execute the method described in any of the above embodiments.
[0260] Optionally, the memory 142 may be independent or integrated with the processor 141 .
[0261] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the aforementioned embodiments and will not be described in detail here.
[0262] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in any of the above embodiments is implemented.
[0263] An embodiment of the present application further provides a computer program product, including a computer program, which implements the method described in any of the above embodiments when executed by a processor.
[0264] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.
[0265] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application.
[0266] It should be understood that the above-mentioned processor can be a processing unit (Central Processing Unit, referred to as CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, referred to as DSP), application-specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.
[0267] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.
[0268] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.
[0269] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0270] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0271] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0272] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A method for generating a material map, It is characterized in that include: Acquire material description information of the 3D white model, wherein the material description information includes background color information and texture information; the material description information is obtained by parsing the initial material description information, wherein the initial material description information includes the material subclass, the humidity of the environment, and the newness and oldness of the 3D white model; Inputting the background color information into a background color map generation model, and obtaining a background color map output by the background color map generation model, wherein the background color map generation model is trained based on a plurality of labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map; the background color map includes a corresponding first sub-roughness map, a first sub-color map, and a first sub-normal map; Inputting the texture information into a texture map generation model to obtain a texture map output by the texture map generation model, wherein the texture map generation model is trained based on a plurality of labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map; the texture map includes a second sub-roughness map, a second sub-color map, and a second sub-normal map; Using the base color map as the bottom layer map and the texture map as the top layer map, the base color map and the texture map are merged to generate a target material map; The step of fusing the base color map and the texture map to generate a target material map includes: Taking the first sub-roughness map as the bottom layer map and the second sub-roughness map as the top layer map, the first sub-roughness map and the second sub-roughness map are merged to obtain a target roughness map; Using the first sub-color map as a bottom layer map and the second sub-color map as an upper layer map, and fusing the first sub-color map and the second sub-color map to obtain a target color map; Using the first sub-normal map as a bottom layer map and the second sub-normal map as an upper layer map, and fusing the first sub-normal map and the second sub-normal map to obtain a target normal map; Among them, the fusion of the bottom layer map and the upper layer map can be achieved by the following formula: Among them, f 2 (A 2 ,B 2 ) is the fused map, A 2 is the normalized bottom layer map, B 2 It is the normalized upper layer map.
2. The method according to claim 1, It is characterized in that The material description information of the 3D white mold also includes damage information. After obtaining the material description information of the 3D white mold, the method further includes: Inputting the damage information into a damage map generation model to obtain a damage map output by the damage map generation model, wherein the damage map generation model is trained based on a plurality of labeled sample damage maps, and the label of each sample damage map is used to characterize the damage information of the sample damage map; Accordingly, the base color map and the texture map are merged to generate a target material map, including: The base color map, the texture map and the damage map are merged to generate a target material map of the 3D white model.
3. The method according to claim 2, It is characterized in that The material description information of the 3D white model also includes dirt information. After obtaining the material description information of the 3D white model, the method further includes: Inputting the dirt information into a dirt map generation model to obtain a dirt map output by the dirt map generation model, wherein the dirt map generation model is trained based on a plurality of labeled sample dirt maps, and the label of each sample dirt map is used to characterize the dirt information of the sample dirt map; Accordingly, the base color map, the texture map and the damage map are merged to generate a target material map of the 3D white model, including: The base color map, the texture map, the damage map and the dirt map are merged to generate a target material map of the 3D white model.
4. The method according to claim 3, It is characterized in that The step of fusing the base color map, the texture map, the damage map and the dirt map to generate a target material map of the 3D white model includes: Using the base color map as a bottom layer map and the texture map as an upper layer map, the base color map and the texture map are merged to generate a first material map; Using the first texture map as a bottom texture and the damaged texture map as an upper texture, the first texture map and the damaged texture map are merged to generate a second texture map; The second texture map is used as a bottom layer texture, the dirt map is used as an upper layer texture, and the second texture map and the dirt map are merged to generate the target texture map.
5. A model training method, It is characterized in that include: Acquire a background color training set, wherein the background color training set includes a plurality of labeled sample background color maps, wherein the label of each sample background color map is used to characterize the background color information of the sample background color map; Model training is performed according to the background color training set to obtain a background color map generation model, wherein the background color map generation model is used to generate a background color map of the 3D white model according to the background color information of the 3D white model; the background color map generation model is used to implement the material map generation method described in any one of claims 1 to 4.
6. A model training method, It is characterized in that include: Acquire a texture training set, wherein the texture training set includes a plurality of labeled sample texture maps, wherein the label of each sample texture map is used to characterize texture information of the sample texture map; Model training is performed according to the texture training set to obtain a texture map generation model, wherein the texture map generation model is used to generate a texture map of the 3D white model according to the texture information of the 3D white model; the texture map generation model is used to implement the material map generation method described in any one of claims 1 to 4.
7. A model training method, It is characterized in that include: Acquire a damage training set, wherein the damage training set includes sample damage maps with labels, and the label of each sample damage map is used to represent damage information of the sample damage map; Model training is performed according to the damage training set to obtain a damage map generation model, wherein the damage map generation model is used to generate a damage map of the 3D white model according to the damage information of the 3D white model; the damage map generation model is used to implement the material map generation method described in any one of claims 1-4.
8. A model training method, It is characterized in that include: Obtain a dirt training set, wherein the dirt training set includes sample dirt maps with labels, and the label of each sample dirt map is used to characterize dirt information of the sample dirt map; Model training is performed according to the dirt training set to obtain a dirt map generation model, wherein the dirt map generation model is used to generate a dirt map of the 3D white model according to the dirt information of the 3D white model; the dirt map generation model is used to implement the material map generation method described in any one of claims 1 to 4.
9. A device for generating a material map, It is characterized in that include: An acquisition module is used to acquire material description information of the 3D white model, wherein the material description information includes background color information and texture information; the material description information is obtained by parsing the initial material description information, wherein the initial material description information includes the material subclass of the 3D white model, the humidity of the environment, the newness and oldness, and the use traces; An input module is used to input the background color information into a background color map generation model to obtain a background color map output by the background color map generation model, wherein the background color map generation model is obtained by training based on a plurality of labeled sample background color maps, and the label of each sample background color map is used to characterize the background color information of the sample background color map; the background color map includes a corresponding first sub-roughness map, a first sub-color map, and a first sub-normal map; The input module is further used to input the texture information into a texture map generation model to obtain a texture map output by the texture map generation model, wherein the texture map generation model is obtained by training based on a plurality of labeled sample texture maps, and the label of each sample texture map is used to characterize the texture information of the sample texture map; the texture map includes a second sub-roughness map, a second sub-color map, and a second sub-normal map; A fusion module, used to use the base color map as a bottom layer map and the texture map as an upper layer map, and fuse the base color map and the texture map to generate a target material map of the 3D white model; The fusion module is specifically used to: use the first sub-roughness map as the bottom layer map, use the second sub-roughness map as the top layer map, and fuse the first sub-roughness map and the second sub-roughness map to obtain a target roughness map; use the first sub-color map as the bottom layer map, and the second sub-color map as the top layer map, and fuse the first sub-color map and the second sub-color map to obtain a target color map; use the first sub-normal map as the bottom layer map, and the second sub-normal map as the top layer map, and fuse the first sub-normal map and the second sub-normal map to obtain a target normal map; Among them, the fusion of the bottom layer map and the upper layer map can be achieved by the following formula: Generate the target material map; where f 2 (A 2 ,B 2 ) is the target material map, A 2 is the normalized bottom layer map, B 2 It is the normalized upper layer map.
10. A model training device, It is characterized in that include: A first training module, a second training module, a third training module and a fourth training module; The first training module is used to execute the method according to claim 5; The second training module is used to perform the method according to claim 6; The third training module is used to perform the method according to claim 7; The fourth training module is used to execute the method of claim 8.
11. An electronic device, It is characterized in that include: at least one processor; as well as a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the method described in any one of claims 1 to 8.
12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method according to any one of claims 1 to 8 is implemented.
