Map generation method and device, medium, equipment and computer program product

Abstract

The invention discloses a map generation method and device, a medium, equipment and a computer program product, and the method comprises the steps: receiving a target image for generating a map, and obtaining a three-dimensional model corresponding to the target image; rendering the three-dimensional model based on a plurality of preset rendering angles to obtain a rendering normal view of the three-dimensional model at each rendering angle; generating a normal feature and a physical rendering feature corresponding to the three-dimensional model at each rendering angle based on the rendering normal view, the target image and an image generation model; the vertexes of the three-dimensional model are updated based on a target normal view corresponding to the normal features, an updated target three-dimensional model is obtained, and the vertexes corresponding to the target normal view are the same as the vertexes corresponding to the target three-dimensional model; and generating a target UV map corresponding to the target three-dimensional model based on the target three-dimensional model and a target view corresponding to the physical rendering feature.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a map generation method, apparatus, medium, device and computer program product. Background Art

[0002] UGC (User Generated Content) products usually need to support players in configuring and creating characters based on their needs, but high-quality art production often requires high technical requirements, which makes it difficult for players to configure. Summary of the invention

[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides a method for generating a map, the method comprising: Receiving a target image for generating a map, and obtaining a three-dimensional model corresponding to the target image; Rendering the three-dimensional model based on a plurality of preset rendering angles to obtain a rendering normal view of the three-dimensional model at each of the rendering angles; Generate normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles based on the rendered normal view, the target image, and the image generation model; updating the vertices of the three-dimensional model based on the target normal view corresponding to the normal feature to obtain an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model; A target UV map corresponding to the target three-dimensional model is generated based on the target three-dimensional model and the target view corresponding to the physical rendering feature.

[0005] In a second aspect, the present disclosure provides a map generation device, the device comprising: A processing module, used for receiving a target image for generating a map, and obtaining a three-dimensional model corresponding to the target image; A rendering module, used to render the three-dimensional model based on a plurality of preset rendering angles, and obtain a rendering normal view of the three-dimensional model at each of the rendering angles; A first generating module, configured to generate normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles based on the rendered normal view, the target image and the image generating model; an updating module, configured to update the vertices of the three-dimensional model based on a target normal view corresponding to the normal feature to obtain an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model; The second generating module is used to generate a target UV map corresponding to the target three-dimensional model based on the target three-dimensional model and the target view corresponding to the physical rendering feature.

[0006] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when the computer program is executed by a processing device.

[0007] In a fourth aspect, the present disclosure provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0008] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0009] Through the above technical scheme, a three-dimensional model corresponding to the target image provided by the user and a UV map for the surface of the three-dimensional model can be generated, and the vertices of the three-dimensional model can be updated to ensure the matching degree between the target normal view and the target three-dimensional model, thereby effectively simplifying the processing process of generating UV maps from the target view corresponding to the physical rendering features obtained based on the image generation model. At the same time, the degree of fit between the generated UV map and the target three-dimensional model can be ensured, and the refinement and usability of the generated map can be improved, thereby supporting users to trigger the generation of new characters other than standardized characters during the interaction process, improving the diversity of character configurations in UGC products, and at the same time reducing the technical requirements for players and improving user experience and interactive activity.

[0010] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 It is a flowchart of a method for generating a texture according to an embodiment of the present disclosure.

[0012] Figure 2 It is a schematic diagram of a method for generating a texture based on an embodiment of the present disclosure.

[0013] Figure 3 It is a block diagram of a sticker generating device provided according to an embodiment of the present disclosure.

[0014] Figure 4 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0017] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0018] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0020] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0021] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0022] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0023] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0024] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0025] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0026] Figure 1 As shown, it is a flowchart of a method for generating a map according to an embodiment of the present disclosure, as shown in Figure 1 As shown, the method may include: In step 11, a target image for generating a map is received, and a three-dimensional model corresponding to the target image is obtained.

[0027] Among them, this embodiment can support users to upload target images to automatically generate corresponding maps. As an example, the target image can contain a target object. As an example, the target object can be a character object to be added by the user, and the image of the target object can be displayed in the target image.

[0028] In this step, a three-dimensional model corresponding to the target object can be generated based on the target image, wherein the three-dimensional model can be represented by a 3D mesh. For example, when a user wants to add a character to the character library, he can upload an image of the character. In this step, the corresponding three-dimensional model can be generated first, and then a texture of the surface of the three-dimensional model, such as a skin texture, can be further generated based on the three-dimensional model, so that a personalized new character can be constructed. As an example, the three-dimensional model corresponding to the image can be obtained based on the technology of generating a 3D mesh based on a single image using a sparse view large reconstruction model (LRM, Large ReconstructionModel) architecture in the art, which is not limited here.

[0029] In step 12, the three-dimensional model is rendered based on a plurality of preset rendering angles to obtain a rendering normal view of the three-dimensional model at each rendering angle.

[0030] Among them, the rendering angle can be set based on the actual application scenario, for example, 6 rendering angles such as 0, 45, 90, 180, -90, and -45 can be set. Among them, 0 is used to represent the front side, 180 can be used to represent the back side, the direction from 0 to the right is represented as a positive value, and the direction from 0 to the left is represented as a negative value. As an example, the three-dimensional model can be rendered based on the PBR (Physically Based Rendering) workflow of Metallic-Roughness. The PBR workflow can include three sets of data: Albedo (reflectivity), Metallic & Roughness (hereinafter abbreviated as MR), and Normal. Therefore, the three-dimensional model can be rendered through the PBR workflow to obtain a rendered normal view. Among them, the rendering process of the PBR workflow can be implemented based on the general rendering technology in this field, which is not limited here.

[0031] In step 13, based on the rendered normal view, the target image and the image generation model, normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles are generated.

[0032] The image generation model may be implemented based on a diffusion model, such as a Diffusion model, etc. For example, the rendered normal view, the target image, and Gaussian noise may be input into the image generation model, so that the Gaussian noise may be gradually denoised based on the features of the rendered normal view and the target image and the image generation model to obtain physical rendering features.

[0033] As an example, the physical rendering feature includes at least one of a metallic feature (Metallic), a roughness feature (Roughness), and a reflectivity feature (Albedo). Among them, the metallic feature is used to indicate whether the model surface tends to be a metal conductor in the real world. The roughness feature is a single-channel map distinguished by black and white colors, which can express the smoothness of the model surface. The closer to white, the rougher the surface, and vice versa. The reflectivity feature (Albedo) is used for the color map of the rendering effect.

[0034] In step 14, the vertices of the three-dimensional model are updated based on the target normal view corresponding to the normal feature to obtain an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model.

[0035] Among them, the normal feature can be decoded based on the decoder corresponding to the normal feature to obtain the target normal view. In this step, the vertices corresponding to the generated target normal view and the vertices corresponding to the three-dimensional model are usually inconsistent. Then, the surface vertices of the three-dimensional model can be further optimized and updated through the target normal view based on the differentiable rendering technology, so that the vertices corresponding to the target normal view and the vertices corresponding to the three-dimensional model are the same, so as to map the view corresponding to the generated physical rendering feature to the surface of the three-dimensional model.

[0036] In step 15, a target UV map corresponding to the target three-dimensional model is generated based on the target three-dimensional model and the target view corresponding to the physical rendering feature.

[0037] As an example, the physical rendering feature can be decoded based on the decoder corresponding to the physical rendering feature to obtain Albedo images and MR images, respectively. In the disclosed embodiment, the vertex information corresponding to the normal feature and the physical rendering feature is consistent. After the vertices of the three-dimensional model are updated in step 14, the vertices of the target three-dimensional model and the vertices corresponding to the normal feature are consistent. Therefore, in this embodiment, the vertices in the image corresponding to the target three-dimensional model and the physical rendering feature are consistent at each rendering angle. In the step, the above-mentioned decoded image can be mapped to the surface of the target three-dimensional model through the back-projection technology to obtain the UV map corresponding to Albedo and MR, that is, the target UV map. Based on the target three-dimensional model and the target UV map, the three-dimensional character and its surface map corresponding to the target image can be obtained.

[0038] Through the above technical scheme, a three-dimensional model corresponding to the target image provided by the user and a UV map for the surface of the three-dimensional model can be generated, and the vertices of the three-dimensional model can be updated to ensure the matching degree between the target normal view and the target three-dimensional model, thereby effectively simplifying the processing process of generating UV maps from the target view corresponding to the physical rendering features obtained based on the image generation model. At the same time, the degree of fit between the generated UV map and the target three-dimensional model can be ensured, and the refinement and usability of the generated map can be improved, thereby supporting users to trigger the generation of new characters other than standardized characters during the interaction process, improving the diversity of character configurations in UGC products, and at the same time reducing the technical requirements for players and improving user experience and interactive activity.

[0039] The following first introduces the PBR feature processing model, which includes an encoder and a decoder. The encoder is used to encode the physical rendering image to obtain the physical rendering features corresponding to the image, and the decoder is used to decode the physical rendering features to obtain the corresponding physical rendering image.

[0040] Among them, the PBR feature processing model can be implemented based on the VAE (Variational Auto-Encoder) model. As an example, the training three-dimensional model can be rendered based on the PBR rendering technology to obtain the physical rendering image corresponding to the training three-dimensional model, such as Albedo image, MR image, Normal image, etc. Afterwards, the pre-trained VAE model can be fine-tuned by the Albedo image, MR image, and Normal image respectively to obtain the Albedo VAE model, MR VAE model, and Normal VAE model. Among them, the method of fine-tuning the VAE model can be implemented based on the training method of the VAE model in the field, and the present disclosure is not limited to this. Therefore, in the above step 14, the normal feature can be decoded based on the decoder of the Normal VAE model, and the physical rendering feature can be decoded based on the decoder of the MR VAE model and the Albedo VAE model to obtain the corresponding images respectively.

[0041] In a possible embodiment, the generating normal features and physical rendering features corresponding to the three-dimensional model at each rendering angle based on the rendered normal view, the target image, and the image generation model may include: Feature extraction is performed on the rendered normal view and the target image to obtain rendered normal features corresponding to the rendered normal view and image features corresponding to the target image.

[0042] As an example, the encoder of the above-mentioned Normal VAE model can be used to extract features from the rendered normal view to obtain rendered normal features, and the target image can be extracted based on the pre-trained VAE model to obtain latent space features of the target image. As an example, the latent space features can be used as image features. As another example, feature extraction can be performed on the target image based on the CLIP (Contrastive Language-Image Pre-Training) model. The CLIP model is a multimodal pre-training model based on contrastive learning, which can be pre-trained with a large amount of paired data of images and texts to learn the alignment relationship between images and texts. In the present disclosure, feature extraction and encoding can be performed on the target image based on the encoder in the pre-trained CLIP model to obtain encoded features, and the latent space features and the encoded features can be further fused to obtain the image features.

[0043] Afterwards, the rendering normal features, the image features and the preset noise image features are input into the image generation model to perform denoising on the noise image features to obtain the normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles.

[0044] Among them, the preset noise image feature can be a Gaussian noise feature, wherein the preset noise image feature can include a noise image feature corresponding to each output feature at each rendering angle. For example, the rendering angle includes 6 angles, and the output feature includes 3 features such as Albedo feature, MR feature, and Normal feature. Then, the preset noise image feature includes 18 (6*3) noise image features.

[0045] As an example, the predicted noise corresponding to each noise image feature can be determined based on the rendering normal feature, the image feature and the preset noise image feature, and the corresponding predicted noise can be subtracted from the noise image feature to implement a denoising process for the noise image feature. Through the denoising process at multiple time steps, the output features corresponding to each noise image feature are finally obtained, that is, 18 features are output, corresponding to each feature at each rendering angle.

[0046] Therefore, through the above technical scheme, the normal features and physical rendering features corresponding to the three-dimensional model at each rendering angle can be obtained through the image generation model, and the features at multiple rendering angles are calculated and processed synchronously in the process, which can ensure the consistency between the features at multiple rendering angles.

[0047] In a possible embodiment, inputting the rendered normal feature, the image feature, and the preset noise image feature into the image generation model to perform denoising on the noise image feature to obtain the normal feature and physical rendering feature corresponding to the three-dimensional model at each rendering angle may include: During the denoising process at each time step, the normal feature, the image feature and the noise image feature corresponding to the current time step are input into the image generation model to obtain the predicted noise corresponding to each noise image feature. When the current time step is the first time step, the noise image feature corresponding to the current time step is the preset noise image feature. When the current time step is not the first time step, the noise image feature corresponding to the current time step is the noise image feature obtained after the denoising process at the previous time step. The normal feature, the image feature and each noise image feature corresponding to the current time step can be spliced ​​to obtain the input feature. The multiple input features corresponding to different features at different rendering angles are input into the image generation model as a batch, and the image generation model performs noise prediction.

[0048] Afterwards, based on the predicted noise corresponding to the noise image features at the target rendering angle, the predicted noise corresponding to the noise image features at other rendering angles except the target rendering angle is updated, and the updated noise is used as the new predicted noise corresponding to the noise image features at the other rendering angles.

[0049] Among them, the target rendering angle can be one of multiple rendering angles, which can be pre-configured based on the actual application scenario. For example, it can be a front rendering angle. In this step, the predicted noise corresponding to the noise image feature of the front rendering angle can be used as the basic predicted noise. For example, based on the basic predicted noise, an adain operation can be performed on the predicted noise corresponding to other rendering angles, that is, for the predicted noise at other angles, it can be updated by subtracting its own mean and its own variance, then multiplying it by the variance of the basic predicted noise and adding the mean of the basic predicted noise, so as to obtain the corresponding new predicted noise at the rendering angle.

[0050] De-noising the noise image features at the rendering angle based on the predicted noise corresponding to the rendering angle, to obtain the noise image features obtained after the de-noising at the current time step; According to the noise image features obtained after the denoising process in the last time step, the normal features and physical rendering features corresponding to the three-dimensional model at each rendering angle are determined.

[0051] Among them, the implementation method of denoising the noise image features after determining the predicted image can be based on the denoising method in the diffusion model in this field. After determining the noise image features obtained after the denoising process at the current time step, noise prediction and denoising can be continued at the next time step based on the noise image features obtained after the denoising process, thereby gradually removing interference noise from the noise image features to obtain output features.

[0052] As an example, the input noise image features may include the noise image features at 6 rendering angles corresponding to Normal, the noise image features at 6 rendering angles corresponding to MR, and the noise image features at 6 rendering angles corresponding to Albedo. After performing denoising processing at the above multiple time steps, the normal features and physical rendering features corresponding to each rendering angle after denoising processing can be obtained respectively.

[0053] Therefore, through the above technical solution, in the process of determining the predicted noise, the predicted noise corresponding to multiple rendering angles can be updated with respect to the predicted noise corresponding to the target rendering angle, thereby improving the consistency of the style and tone of the features generated at different rendering angles, and providing effective data support for the subsequent map generation under multiple perspectives.

[0054] In a possible embodiment, the image generation model may be determined in the following manner: The training three-dimensional model is obtained. As an example, a three-dimensional model corresponding to a character in a character library may be obtained as the training three-dimensional model.

[0055] The training 3D model is rendered based on the multiple rendering angles to obtain a training view of the training 3D model at each rendering angle, wherein the training view at least includes a reflectivity view and a normal view.

[0056] Based on the feature processing model, feature extraction is performed on each of the training views to obtain view features corresponding to the training views. Similarly, in this step, image rendering can be performed from multiple rendering angles based on the same rendering method described above, so that the training views corresponding to each rendering angle can be obtained, such as the Albedo view, MR view, and Normal view corresponding to the above 6 rendering angles, that is, each training 3D model can be rendered to obtain the 18 training views.

[0057] Furthermore, the feature processing model can be the PBR feature processing model described above. Then, feature extraction can be performed for the Albedo view through the Albedo VAE model, for the MR view through the MR VAE model, and for the Normal view through the Normal VAE model, thereby improving the accuracy of feature extraction for different views and improving the accuracy and effectiveness of the obtained view features.

[0058] Add target noise to the view feature to obtain a noisy view feature.

[0059] The target noise may be a preset Gaussian noise. The noise adding process may be a step-by-step process, such as starting from a real data distribution (i.e., the grid feature), and gradually converting the data distribution into a standard Gaussian distribution by adding Gaussian noise in each step, wherein the target noise in each step may be pre-configured and the target noise in each step may be different.

[0060] In this step, when adding target noise to the visual feature, the time step corresponding to the target noise can be identified at the same time, and the noisy view feature can be obtained by masking some details in the view feature by adding the target noise to the view feature.

[0061] The image generation model is obtained by generating training samples based on the view features of the normal view, the view features of the reflectivity view, the target noise and the noise-added view features, and training a preset model.

[0062] As an example, the view features of the normal view and the view features of the reflectivity view can be spliced ​​with each of the noise-added view features to obtain an input feature, and the multiple input features corresponding to the different features at different rendering angles are input as a batch into a preset model, and then the added noise is predicted based on the preset model. Furthermore, the loss of the model is determined based on the predicted noise and the added target noise, and then the parameters of the model are adjusted to obtain an image generation model. The loss of the model can be calculated based on the loss function of the DDPM (Denoising Diffusion Probabilistic Models) model, which is not limited in the present disclosure.

[0063] As another example, the generating training samples by using the view features of the normal view, the view features of the reflectivity view, the target noise, and the noise-added view features, training a preset model, and obtaining the image generation model may include: A first training sample is generated using the view feature of the reflectivity view, the target noise and the noise-added view feature, and the preset model is trained to obtain a trained basic model.

[0064] In this step, the view features of the reflectivity view and each of the noise-added view features can be spliced ​​to obtain an input feature, and the multiple input features corresponding to the different features at different rendering angles are input as a batch into a preset model to predict the added noise based on the preset model. Then, the loss of the model is determined based on the predicted noise and the added target noise, and then the parameters of the model are adjusted to obtain a basic model. In this training process, the view features at multiple rendering angles can be trained as a batch, so that the basic model can generate images with multi-view consistency.

[0065] Afterwards, a second training sample is generated using the view features of the normal view, the view features of the reflectivity view, the target noise, and the noise-added view features, and the basic model is fine-tuned to obtain the image generation model.

[0066] In order to improve the accuracy of the subsequently generated UV map, the disclosed solution can further allow the normal features of the three-dimensional model to be input as control quantities on the basis of the basic model. In this step, the view features of the normal view at multiple rendering angles and the view features of the reflectivity view can be spliced ​​with each of the noisy view features to obtain input features, and the multiple input features corresponding to the different features at different rendering angles are input as a batch into the basic model to fine-tune the basic model. In this training process, the added noise can be predicted based on the basic model, and the model loss can be determined based on the predicted noise and the target noise, so as to further adjust the parameters of the basic model based on the model loss to obtain the image generation model.

[0067] Among them, the calculation of the model loss and the implementation method of fine-tuning training in the above process can be determined based on the loss function and training method of the DDPM model in the art, and the present disclosure is not limited here.

[0068] Therefore, through the above technical solution, the first model can be trained based on the view features under different rendering angles as input, and fine-tuning training can be performed based on the first model in combination with the normal of the three-dimensional model as a control quantity, so that the trained image generation model can generate images with multi-perspective consistency while further improving the matching degree between the normal features generated by the image generation model and the three-dimensional model, thereby improving the accuracy of the subsequently generated UV map.

[0069] In actual application scenarios, features at the same height under different rendering angles are usually correlated, and the uniformity of features under multiple rendering angles needs to be ensured. In this embodiment, during the training process of noise prediction of the image generation model, attention calculation can be performed based on the row attention mechanism. For example, self-attention calculation can be performed on each line in the input feature, so that each position in the line and the attention weights between each position in the line can be learned during the attention calculation process to realize attention calculation, that is, the distribution of attention is calculated separately for each line, so that the data processing amount of attention calculation can be effectively reduced while ensuring the accuracy of the output results of the image generation model, thereby improving the processing efficiency of the image generation model.

[0070] In a possible embodiment, the attention mechanism of the image generation model is implemented by a row-by-row self-attention mechanism. During the calculation process of the row-by-row self-attention mechanism, a query vector is generated based on the features of each row, and a key vector and a value vector are generated based on the features of a target row corresponding to the row, wherein the target row corresponding to the row contains multiple rows of features with the row as the center row.

[0071] As an example, in the process of image reasoning based on the image generation model, in order to ensure the continuity of the generated image, the query vector Query can be generated based on the features of the current row, and the corresponding key vector Key and value vector Value can be generated based on the features of the current row and its adjacent rows. As an example, the range corresponding to the target row can be set. For example, the range can be set to 1, and the target row can include the current row, the previous row of the current row, and the next row of the current row.

[0072] Therefore, through the above technical solution, in the process of attention calculation, the current row can pay attention to the characteristics of the current row and its adjacent rows, thereby ensuring the continuity between adjacent rows during the image denoising process, thereby improving the continuity of the rendering features output by the image generation model and improving the accuracy of the subsequently generated UV map.

[0073] In a possible embodiment, updating the vertices of the three-dimensional model based on the target normal view corresponding to the normal feature to obtain an updated target three-dimensional model may include: Normal view difference information is determined based on the target normal view and the rendering normal view corresponding to each of the rendering angles.

[0074] As an example, for each rendering angle, the L2 loss can be calculated based on the corresponding target normal view and the rendering normal view at the rendering angle, and then the sum of the corresponding L2 losses at each rendering angle is used as the normal view difference information. As another example, weights can be pre-set based on the importance of different rendering angles, and the weighted sum of the corresponding L2 losses at each rendering angle can be used as the normal view difference information.

[0075] The vertices of the three-dimensional model are adjusted based on the normal view difference information to obtain an updated three-dimensional model.

[0076] Afterwards, the return gradient can be determined based on the determined normal view difference information, and the return gradient can be returned to each vertex of the three-dimensional model to adjust the vertex information and obtain an updated three-dimensional model. The gradient return adjustment method can be implemented based on the differentiable rendering technology in the field, which will not be elaborated here.

[0077] Rendering the updated three-dimensional model to obtain a rendered normal view corresponding to the updated three-dimensional model, and returning to the step of determining normal view difference information based on the target normal view and the rendered normal view corresponding to each rendering angle, until the target normal view and the rendered normal view corresponding to the updated three-dimensional model are the same; The three-dimensional model obtained after the last update is used as the target three-dimensional model.

[0078] Furthermore, the updated three-dimensional model can be rendered again based on the PBR workflow to obtain its corresponding rendered normal view, and the normal view difference information between the rendered normal view and the target normal view obtained based on the image generation model can be further determined, and the three-dimensional model can be adjusted again based on the normal view difference information, so as to ultimately make the target normal view and the rendered normal view corresponding to the updated three-dimensional model the same.

[0079] Among them, the normal view can be used to represent an image obtained by drawing a normal line at each point on the concave and convex surface of the three-dimensional model, and the direction of the normal line can be marked by the RGB color channel in the image. Then the target normal view is the same as the rendering normal view corresponding to the updated three-dimensional model, and it can be considered that the generated normal view is completely consistent with the surface points of the updated three-dimensional model, and the view corresponding to the physical rendering feature is corresponding to the target normal view, then the target view corresponding to the physical rendering feature can be directly mapped to the surface of the updated three-dimensional model through back projection, and the three-dimensional model obtained after the last update can be used as the target three-dimensional model.

[0080] Therefore, through the above technical scheme, the vertices of the three-dimensional model can be updated based on the target normal view obtained by the image generation model. On the one hand, the fineness of the three-dimensional model can be improved, and on the other hand, the matching degree between the target normal view and the target three-dimensional model can be guaranteed, which effectively simplifies the processing of generating UV maps based on the target view corresponding to the physical rendering features obtained by the image generation model. At the same time, it can also ensure the fit between the generated UV maps and the target three-dimensional model, improve the fineness and usability of the generated maps, and improve the user experience of users in content generation and the accuracy and diversity of the generated content.

[0081] Figure 2 As shown, it is a schematic diagram of a map generation method provided based on an embodiment of the present disclosure, wherein B1 is used to represent the target image, B2 is used to represent the three-dimensional model, B3 is used to represent the target three-dimensional model, B4 represents the UV map, A6 represents the Albedo feature under 6 rendering angles, MR6 represents the MR feature under 6 rendering angles, and N6 represents the Normal feature under 6 rendering angles.

[0082] Based on the same inventive concept, the present disclosure also provides a map generating device, such as Figure 3 As shown, the device 10 comprises: The processing module 100 is used to receive a target image for generating a map, and obtain a three-dimensional model corresponding to the target image; A rendering module 200, configured to render the three-dimensional model based on a plurality of preset rendering angles, and obtain a rendering normal view of the three-dimensional model at each of the rendering angles; A first generating module 300, configured to generate normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles based on the rendered normal view, the target image and the image generating model; An updating module 400, configured to update the vertices of the three-dimensional model based on a target normal view corresponding to the normal feature to obtain an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model; The second generating module 500 is used to generate a target UV map corresponding to the target three-dimensional model based on the target three-dimensional model and the target view corresponding to the physical rendering feature.

[0083] Optionally, the first generating module includes: A first extraction submodule, configured to perform feature extraction on the rendered normal view and the target image to obtain rendered normal features corresponding to the rendered normal view and image features corresponding to the target image; The first generation submodule is used to input the rendering normal feature, the image feature and the preset noise image feature into the image generation model to denoise the noise image feature and obtain the normal feature and physical rendering feature corresponding to the three-dimensional model at each rendering angle.

[0084] Optionally, the first generating submodule includes: A second generation submodule is used to input the normal feature, the image feature and the noise image feature corresponding to the current time step into the image generation model during the denoising process of each time step, to obtain the predicted noise corresponding to each of the noise image features, when the current time step is the first time step, the noise image feature corresponding to the current time step is the preset noise image feature, and when the current time step is not the first time step, the noise image feature corresponding to the current time step is the noise image feature obtained after the denoising process of the previous time step; A first updating submodule is configured to update the predicted noise corresponding to the noise image features at other rendering angles other than the target rendering angle based on the predicted noise corresponding to the noise image features at the target rendering angle, and use the updated noise as the new predicted noise corresponding to the noise image features at the other rendering angles; A first processing submodule is used to perform denoising on the noise image features at the rendering angle based on the predicted noise corresponding to the rendering angle, and obtain the noise image features obtained after the denoising at the current time step; The first determination submodule is used to determine the normal features and physical rendering features corresponding to the three-dimensional model at each rendering angle according to the noise image features obtained after the denoising process in the last time step.

[0085] Optionally, the physical rendering feature includes at least one of a metal feature, a roughness feature and a reflectivity feature.

[0086] Optionally, the image generation model is determined by a training module, and the training module includes: An acquisition submodule is used to acquire a training three-dimensional model; A first rendering submodule, configured to render the training 3D model based on the multiple rendering angles to obtain a training view of the training 3D model at each rendering angle, wherein the training view at least includes a reflectivity view and a normal view; A second extraction submodule, configured to extract features from each of the training views based on a feature processing model to obtain view features corresponding to the training views; An adding submodule, used for adding target noise to the view feature to obtain a noisy view feature; The first training submodule is used to generate training samples based on the view features of the normal view, the view features of the reflectivity view, the target noise and the noise-added view features, train a preset model, and obtain the image generation model.

[0087] Optionally, the first training submodule includes: A second training submodule is used to generate a first training sample based on the view feature of the reflectivity view, the target noise and the noise-added view feature, and train the preset model to obtain a trained basic model; The third training submodule is used to generate a second training sample based on the view features of the normal view, the view features of the reflectivity view, the target noise and the noise-added view features, and to perform fine-tuning training on the basic model to obtain the image generation model.

[0088] Optionally, the attention mechanism of the image generation model is implemented by a row-by-row self-attention mechanism. During the calculation process of the row-by-row self-attention mechanism, a query vector is generated based on the features of each row, and a key vector and a value vector are generated based on the features of a target row corresponding to the row, wherein the target row corresponding to the row contains multiple rows of features with the row as the center row.

[0089] Optionally, the update module includes: A second determining submodule, configured to determine normal view difference information based on the target normal view and the rendering normal view corresponding to each of the rendering angles; A second updating submodule, configured to adjust vertices of the three-dimensional model based on the normal view difference information to obtain an updated three-dimensional model; a second rendering submodule, configured to render the updated three-dimensional model, obtain a rendered normal view corresponding to the updated three-dimensional model, and trigger the second determining submodule to determine normal view difference information based on the target normal view and the rendered normal view corresponding to each of the rendering angles, until the target normal view and the rendered normal view corresponding to the updated three-dimensional model are the same; The third determination submodule is used to use the three-dimensional model obtained after the last update as the target three-dimensional model.

[0090] Reference below Figure 4, which shows a schematic diagram of the structure of an electronic device (such as a terminal device or a server) 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0091] like Figure 4 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0092] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0093] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0094] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0095] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0096] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0097] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: receives a target image for generating a map, and obtains a three-dimensional model corresponding to the target image; renders the three-dimensional model based on a plurality of preset rendering angles, and obtains a rendering normal view of the three-dimensional model at each rendering angle; generates normal features and physical rendering features corresponding to the three-dimensional model at each rendering angle based on the rendering normal view, the target image, and the image generation model; updates the vertices of the three-dimensional model based on the target normal view corresponding to the normal features, and obtains an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model; generates a target UV map corresponding to the target three-dimensional model based on the target three-dimensional model and the target view corresponding to the physical rendering features.

[0098] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0099] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0100] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, a processing module may also be described as a "module that receives a target image for generating a map and obtains a three-dimensional model corresponding to the target image."

[0101] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0102] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0103] According to one or more embodiments of the present disclosure, Example 1 provides a method for generating a map, wherein the method includes: Receiving a target image for generating a map, and obtaining a three-dimensional model corresponding to the target image; Rendering the three-dimensional model based on a plurality of preset rendering angles to obtain a rendering normal view of the three-dimensional model at each of the rendering angles; Generate normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles based on the rendered normal view, the target image, and the image generation model; updating the vertices of the three-dimensional model based on the target normal view corresponding to the normal feature to obtain an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model; A target UV map corresponding to the target three-dimensional model is generated based on the target three-dimensional model and the target view corresponding to the physical rendering feature.

[0104] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the step of generating normal features and physical rendering features corresponding to the three-dimensional model at each rendering angle based on the rendered normal view, the target image, and the image generation model includes: Performing feature extraction on the rendered normal view and the target image to obtain rendered normal features corresponding to the rendered normal view and image features corresponding to the target image; The rendering normal feature, the image feature and the preset noise image feature are input into the image generation model to perform denoising on the noise image feature to obtain the normal feature and physical rendering feature corresponding to the three-dimensional model at each rendering angle.

[0105] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein the step of inputting the rendering normal feature, the image feature, and the preset noise image feature into the image generation model to perform denoising on the noise image feature to obtain the normal feature and physical rendering feature corresponding to the three-dimensional model at each rendering angle includes: In the denoising process of each time step, the normal feature, the image feature and the noise image feature corresponding to the current time step are input into the image generation model to obtain the predicted noise corresponding to each of the noise image features, when the current time step is the first time step, the noise image feature corresponding to the current time step is the preset noise image feature, and when the current time step is not the first time step, the noise image feature corresponding to the current time step is the noise image feature obtained after the denoising process of the previous time step; Based on the predicted noise corresponding to the noise image feature at the target rendering angle, updating the predicted noise corresponding to the noise image feature at other rendering angles except the target rendering angle, and using the updated noise as the new predicted noise corresponding to the noise image feature at the other rendering angles; De-noising the noise image features at the rendering angle based on the predicted noise corresponding to the rendering angle, to obtain the noise image features obtained after the de-noising at the current time step; According to the noise image features obtained after the denoising process in the last time step, the normal features and physical rendering features corresponding to the three-dimensional model at each rendering angle are determined.

[0106] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 1, wherein the physical rendering feature includes at least one of a metal feature, a roughness feature, and a reflectivity feature.

[0107] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 1, wherein the image generation model is determined in the following manner: Obtain a training 3D model; Rendering the training 3D model based on the multiple rendering angles to obtain a training view of the training 3D model at each rendering angle, wherein the training view at least includes a reflectivity view and a normal view; Extracting features from each of the training views based on a feature processing model to obtain view features corresponding to the training views; Adding target noise to the view feature to obtain a noisy view feature; The image generation model is obtained by generating training samples based on the view features of the normal view, the view features of the reflectivity view, the target noise and the noise-added view features, and training a preset model.

[0108] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 5, wherein the generating a training sample based on the view feature of the normal view, the view feature of the reflectivity view, the target noise, and the noise-added view feature, training a preset model, and obtaining the image generation model includes: Generate a first training sample based on the view feature of the reflectivity view, the target noise and the noise-added view feature, train the preset model, and obtain a trained basic model; A second training sample is generated using the view features of the normal view, the view features of the reflectivity view, the target noise, and the noise-added view features, and fine-tuning training is performed on the basic model to obtain the image generation model.

[0109] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 1, wherein the attention mechanism of the image generation model is implemented by a row-by-row self-attention mechanism, and during the calculation process of the row-by-row self-attention mechanism, a query vector is generated based on the features of each row, and a key vector and a value vector are generated based on the features of a target row corresponding to the row, wherein the target row corresponding to the row contains multiple rows of features with the row as the center row.

[0110] According to one or more embodiments of the present disclosure, Example 8 provides the method of Example 1, wherein updating the vertices of the three-dimensional model based on the target normal view corresponding to the normal feature to obtain an updated target three-dimensional model includes: Determining normal view difference information based on the target normal view and the rendering normal view corresponding to each of the rendering angles; adjusting vertices of the three-dimensional model based on the normal view difference information to obtain an updated three-dimensional model; Rendering the updated three-dimensional model to obtain a rendered normal view corresponding to the updated three-dimensional model, and returning to the step of determining normal view difference information based on the target normal view and the rendered normal view corresponding to each rendering angle, until the target normal view and the rendered normal view corresponding to the updated three-dimensional model are the same; The three-dimensional model obtained after the last update is used as the target three-dimensional model.

[0111] According to one or more embodiments of the present disclosure, Example 9 provides a map generation device, the device comprising: A processing module, used for receiving a target image for generating a map, and obtaining a three-dimensional model corresponding to the target image; A rendering module, used to render the three-dimensional model based on a plurality of preset rendering angles, and obtain a rendering normal view of the three-dimensional model at each of the rendering angles; A first generating module, configured to generate normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles based on the rendered normal view, the target image and the image generating model; an updating module, configured to update the vertices of the three-dimensional model based on a target normal view corresponding to the normal feature to obtain an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model; The second generating module is used to generate a target UV map corresponding to the target three-dimensional model based on the target three-dimensional model and the target view corresponding to the physical rendering feature.

[0112] According to one or more embodiments of the present disclosure, Example 10 provides a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processing device, the steps of the method described in any one of Examples 1-8 are implemented.

[0113] According to one or more embodiments of the present disclosure, Example 11 provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of any one of the methods described in Examples 1-8.

[0114] According to one or more embodiments of the present disclosure, Example 12 provides a computer program product, including a computer program, which implements the steps of any one of the methods of Examples 1-8 when executed by a processor.

[0115] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0116] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0117] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A method for generating a texture, characterized in that: The method comprises: Receiving a target image for generating a map, and obtaining a three-dimensional model corresponding to the target image; Rendering the three-dimensional model based on a plurality of preset rendering angles to obtain a rendering normal view of the three-dimensional model at each of the rendering angles; Generate normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles based on the rendered normal view, the target image, and the image generation model; updating the vertices of the three-dimensional model based on the target normal view corresponding to the normal feature to obtain an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model; A target UV map corresponding to the target three-dimensional model is generated based on the target three-dimensional model and the target view corresponding to the physical rendering feature.

2. The method according to claim 1, characterized in that The generating of normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles based on the rendered normal view, the target image and the image generation model comprises: Performing feature extraction on the rendered normal view and the target image to obtain rendered normal features corresponding to the rendered normal view and image features corresponding to the target image; The rendering normal feature, the image feature and the preset noise image feature are input into the image generation model to perform denoising on the noise image feature to obtain the normal feature and physical rendering feature corresponding to the three-dimensional model at each rendering angle.

3. The method according to claim 2, characterized in that The step of inputting the rendered normal feature, the image feature and the preset noise image feature into the image generation model to perform denoising on the noise image feature to obtain the normal feature and physical rendering feature corresponding to the three-dimensional model at each rendering angle includes: In the denoising process of each time step, the normal feature, the image feature and the noise image feature corresponding to the current time step are input into the image generation model to obtain the predicted noise corresponding to each of the noise image features, when the current time step is the first time step, the noise image feature corresponding to the current time step is the preset noise image feature, and when the current time step is not the first time step, the noise image feature corresponding to the current time step is the noise image feature obtained after the denoising process of the previous time step; Based on the predicted noise corresponding to the noise image feature at the target rendering angle, updating the predicted noise corresponding to the noise image feature at other rendering angles except the target rendering angle, and using the updated noise as the new predicted noise corresponding to the noise image feature at the other rendering angles; De-noising the noise image features at the rendering angle based on the predicted noise corresponding to the rendering angle, to obtain the noise image features obtained after the de-noising at the current time step; According to the noise image features obtained after the denoising process in the last time step, the normal features and physical rendering features corresponding to the three-dimensional model at each rendering angle are determined.

4. The method according to claim 1, characterized in that: The physical rendering feature includes at least one of a metal feature, a roughness feature, and a reflectivity feature.

5. The method according to claim 1, characterized in that: The image generation model is determined by: Obtain a training 3D model; Rendering the training 3D model based on the multiple rendering angles to obtain a training view of the training 3D model at each rendering angle, wherein the training view at least includes a reflectivity view and a normal view; Extracting features from each of the training views based on a feature processing model to obtain view features corresponding to the training views; Adding target noise to the view feature to obtain a noisy view feature; The image generation model is obtained by generating training samples based on the view features of the normal view, the view features of the reflectivity view, the target noise and the noise-added view features, and training a preset model.

6. The method according to claim 5, characterized in that The generating of training samples by using the view features of the normal view, the view features of the reflectivity view, the target noise and the noise-added view features, training a preset model and obtaining the image generation model comprises: Generate a first training sample based on the view feature of the reflectivity view, the target noise and the noise-added view feature, train the preset model, and obtain a trained basic model; A second training sample is generated using the view features of the normal view, the view features of the reflectivity view, the target noise, and the noise-added view features, and fine-tuning training is performed on the basic model to obtain the image generation model.

7. The method according to claim 1, characterized in that The attention mechanism of the image generation model is implemented through a row-by-row self-attention mechanism. During the calculation process of the row-by-row self-attention mechanism, a query vector is generated based on the features of each row, and a key vector and a value vector are generated based on the features of a target row corresponding to the row, wherein the target row corresponding to the row contains multiple rows of features with the row as the center row.

8. The method according to claim 1, characterized in that The updating of the vertices of the three-dimensional model based on the target normal view corresponding to the normal feature to obtain an updated target three-dimensional model includes: Determining normal view difference information based on the target normal view and the rendering normal view corresponding to each of the rendering angles; adjusting vertices of the three-dimensional model based on the normal view difference information to obtain an updated three-dimensional model; Rendering the updated three-dimensional model to obtain a rendered normal view corresponding to the updated three-dimensional model, and returning to the step of determining normal view difference information based on the target normal view and the rendered normal view corresponding to each rendering angle, until the target normal view and the rendered normal view corresponding to the updated three-dimensional model are the same; The three-dimensional model obtained after the last update is used as the target three-dimensional model.

9. A map generating device, characterized in that: The device comprises: A processing module, used for receiving a target image for generating a map, and obtaining a three-dimensional model corresponding to the target image; A rendering module, used to render the three-dimensional model based on a plurality of preset rendering angles, and obtain a rendering normal view of the three-dimensional model at each of the rendering angles; A first generating module, configured to generate normal features and physical rendering features corresponding to the three-dimensional model at various rendering angles based on the rendered normal view, the target image and the image generating model; an updating module, configured to update the vertices of the three-dimensional model based on a target normal view corresponding to the normal feature to obtain an updated target three-dimensional model, wherein the vertices corresponding to the target normal view are the same as the vertices corresponding to the target three-dimensional model; The second generating module is used to generate a target UV map corresponding to the target three-dimensional model based on the target three-dimensional model and the target view corresponding to the physical rendering feature.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images