Methods, devices, electronic equipment, storage media, and program products for editing 3D model textures

Abstract

This disclosure relates to a method, apparatus, electronic device, storage medium, and program product for editing 3D model textures. The method includes: acquiring a target 3D model texture to be edited; acquiring target text description information for editing the target 3D model texture; and editing the target 3D model texture according to the target text description information to obtain an edited 3D model texture. This disclosure enables more flexible editing and more diverse editing effects by automatically editing 3D model textures based on input text description information. By adopting embodiments of this disclosure, users can be guided to edit 3D model textures through natural language descriptions. This text-based editing method is more in line with human intuitive expression habits, lowers the technical threshold, and allows more users to easily personalize 3D model textures.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method for editing 3D model textures, a device for editing 3D model textures, electronic devices, computer-readable storage media, and computer program products. Background Technology

[0002] Currently, editing 3D (3D) face textures primarily relies on 2D (2D) face images for attribute editing. For example, a 2D face image can first be input into a pre-trained face attribute editing model to obtain an edited face image. Then, target segmentation and portrait segmentation are performed on both the input 2D face image and the edited image to obtain a target domain mask and a target portrait mask, respectively, and their difference is calculated. Finally, the images before and after editing are fused, and the fused image and the difference are input into an image inpainting model to output the face image with the target attributes edited.

[0003] When applying 2D face images to 3D face mapping scenes, the related technologies require regenerating the 3D face map from the edited image, which is very time-consuming and may change the mesh of the 3D face. Summary of the Invention

[0004] This disclosure provides a technical solution for editing textures on 3D models.

[0005] According to one aspect of this disclosure, a method for editing textures of a 3D model is provided, comprising:

[0006] Obtain the texture of the target 3D model to be edited;

[0007] Obtain the target text description information used to edit the texture of the target 3D model;

[0008] Based on the target text description information, the target 3D model texture is edited to obtain the edited 3D model texture.

[0009] In one possible implementation, editing the target 3D model texture based on the target text description information to obtain the edited 3D model texture includes:

[0010] Generate editable potential code corresponding to the texture of the target 3D model;

[0011] Based on the target text description information and the editable potential code, generate the edited potential code;

[0012] Based on the edited potential code, an edited 3D model texture is generated.

[0013] In one possible implementation, the editable underlying code for generating the texture of the target 3D model includes:

[0014] In response to the target text description information used to perform overall editing of the target 3D model texture, editable potential code corresponding to the target 3D model texture is generated.

[0015] In one possible implementation, generating the edited potential code based on the target text description information and the editable potential code includes:

[0016] Train a mapping model based on the target text description information and the editable potential code;

[0017] In response to the completion of the mapping model training, the target text description information and the editable potential code are input into the trained mapping model, and the edited potential code is generated through the trained mapping model.

[0018] In one possible implementation, training the mapping model based on the target text description information and the editable potential code includes:

[0019] The target text description information and the editable latent code are input into the mapping model, and the predicted edited latent code is generated through the mapping model;

[0020] Obtain the predicted rendered image corresponding to the edited potential code of the prediction;

[0021] The mapping model is trained based on the predicted rendered image and the target text description information.

[0022] In one possible implementation, obtaining the predicted rendered image corresponding to the predicted edited potential code includes:

[0023] Generate the edited 3D model texture corresponding to the edited potential code of the prediction;

[0024] The predicted edited 3D model texture is applied to the mesh of the target 3D model corresponding to the target 3D model texture to obtain the predicted rendered image corresponding to the predicted edited latent code.

[0025] In one possible implementation, training the mapping model based on the predicted rendered image and the target text description information includes:

[0026] The value of the loss function corresponding to the mapping model is determined based on the similarity between the predicted rendered image and the target text description information;

[0027] The mapping model is trained based on the value of the loss function.

[0028] In one possible implementation, generating the edited potential code based on the target text description information and the editable potential code includes:

[0029] Determine the target mapping model;

[0030] The target text description information and the editable potential code are input into the target mapping model, and the edited potential code is generated through the target mapping model.

[0031] In one possible implementation, determining the target mapping model includes:

[0032] Determine the keywords of the target text description information;

[0033] Based on the keywords, determine the target mapping model.

[0034] In one possible implementation, the editable underlying code for generating the texture of the target 3D model includes:

[0035] Obtain the potential code to be optimized corresponding to the texture of the target 3D model;

[0036] Generate the 3D model texture map corresponding to the potential code to be optimized;

[0037] Based on the 3D model texture corresponding to the potential code to be optimized and the target 3D model texture, the potential code to be optimized is optimized until a preset stop optimization condition is met, thereby obtaining the editable potential code corresponding to the target 3D model texture.

[0038] In one possible implementation, the editable underlying code for generating the texture of the target 3D model includes:

[0039] The target 3D model texture is input into a pre-trained inverse model, and the inverse model generates editable potential code corresponding to the target 3D model texture.

[0040] In one possible implementation, generating the edited 3D model texture based on the edited underlying code includes:

[0041] The edited potential code is input into the first preset generation model, and the edited 3D model texture is generated through the first preset generation model.

[0042] In one possible implementation, the first preset generative model is a generative adversarial model.

[0043] In one possible implementation, editing the target 3D model texture based on the target text description information to obtain the edited 3D model texture includes:

[0044] Obtain the location information of the target editing area of ​​the target 3D model texture;

[0045] The target 3D model texture, the location information of the target editing area, and the target text description information are input into the second preset generation model, and the edited 3D model texture is generated through the second preset generation model.

[0046] In one possible implementation, obtaining the location information of the target editing area of ​​the target 3D model texture includes:

[0047] Based on the area that the user has applied to the texture of the target 3D model, the location information of the target editing area of ​​the texture of the target 3D model is determined.

[0048] In one possible implementation, the second preset generation model is an image generation model based on the diffusion process.

[0049] In one possible implementation, the target 3D model texture is a 3D face texture.

[0050] According to one aspect of this disclosure, an editing apparatus for a three-dimensional model texture is provided, comprising:

[0051] The first acquisition module is used to acquire the texture of the target 3D model to be edited;

[0052] The second acquisition module is used to acquire target text description information for editing the texture of the target 3D model;

[0053] The editing module is used to edit the target 3D model texture according to the target text description information to obtain the edited 3D model texture.

[0054] In one possible implementation, the editing module is used for:

[0055] Generate editable potential code corresponding to the texture of the target 3D model;

[0056] Based on the target text description information and the editable potential code, generate the edited potential code;

[0057] Based on the edited potential code, an edited 3D model texture is generated.

[0058] In one possible implementation, the editing module is used for:

[0059] In response to the target text description information used to perform overall editing of the target 3D model texture, editable potential code corresponding to the target 3D model texture is generated.

[0060] In one possible implementation, the editing module is used for:

[0061] Train a mapping model based on the target text description information and the editable potential code;

[0062] In response to the completion of the mapping model training, the target text description information and the editable potential code are input into the trained mapping model, and the edited potential code is generated through the trained mapping model.

[0063] In one possible implementation, the editing module is used for:

[0064] The target text description information and the editable latent code are input into the mapping model, and the predicted edited latent code is generated through the mapping model;

[0065] Obtain the predicted rendered image corresponding to the edited potential code of the prediction;

[0066] The mapping model is trained based on the predicted rendered image and the target text description information.

[0067] In one possible implementation, the editing module is used for:

[0068] Generate the edited 3D model texture corresponding to the edited potential code of the prediction;

[0069] The predicted edited 3D model texture is applied to the mesh of the target 3D model corresponding to the target 3D model texture to obtain the predicted rendered image corresponding to the predicted edited latent code.

[0070] In one possible implementation, the editing module is used for:

[0071] The value of the loss function corresponding to the mapping model is determined based on the similarity between the predicted rendered image and the target text description information;

[0072] The mapping model is trained based on the value of the loss function.

[0073] In one possible implementation, the editing module is used for:

[0074] Determine the target mapping model;

[0075] The target text description information and the editable potential code are input into the target mapping model, and the edited potential code is generated through the target mapping model.

[0076] In one possible implementation, the editing module is used for:

[0077] Determine the keywords of the target text description information;

[0078] Based on the keywords, determine the target mapping model.

[0079] In one possible implementation, the editing module is used for:

[0080] Obtain the potential code to be optimized corresponding to the texture of the target 3D model;

[0081] Generate the 3D model texture map corresponding to the potential code to be optimized;

[0082] Based on the 3D model texture corresponding to the potential code to be optimized and the target 3D model texture, the potential code to be optimized is optimized until a preset stop optimization condition is met, thereby obtaining the editable potential code corresponding to the target 3D model texture.

[0083] In one possible implementation, the editing module is used for:

[0084] The target 3D model texture is input into a pre-trained inverse model, and the inverse model generates editable potential code corresponding to the target 3D model texture.

[0085] In one possible implementation, the editing module is used for:

[0086] The edited potential code is input into the first preset generation model, and the edited 3D model texture is generated through the first preset generation model.

[0087] In one possible implementation, the first preset generative model is a generative adversarial model.

[0088] In one possible implementation, the editing module is used for:

[0089] Obtain the location information of the target editing area of ​​the target 3D model texture;

[0090] The target 3D model texture, the location information of the target editing area, and the target text description information are input into the second preset generation model, and the edited 3D model texture is generated through the second preset generation model.

[0091] In one possible implementation, the editing module is used for:

[0092] Based on the area that the user has applied to the texture of the target 3D model, the location information of the target editing area of ​​the texture of the target 3D model is determined.

[0093] In one possible implementation, the second preset generation model is an image generation model based on the diffusion process.

[0094] In one possible implementation, the target 3D model texture is a 3D face texture.

[0095] According to one aspect of this disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the method described above.

[0096] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0097] According to one aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device performs the above-described method.

[0098] In this embodiment, by acquiring the target 3D model texture to be edited, acquiring target text description information for editing the target 3D model texture, and editing the target 3D model texture according to the target text description information, an edited 3D model texture is obtained. This allows for direct editing of the 3D model texture based on text description information without altering the 3D model's mesh, thereby improving the flexibility of 3D model texture editing. In related technologies, editing models using pre-trained attributes is not flexible enough, and the editing functions are very limited. However, in this embodiment, by automatically editing the 3D model texture based on input text description information, editing becomes more flexible, and the editing effects become more diverse. By adopting this embodiment, users can be guided to edit 3D model textures through natural language descriptions. This text-based editing method is more in line with human intuitive expression habits, lowers the technical threshold, and allows more users to easily personalize 3D model textures.

[0099] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0100] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0101] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0102] Figure 1 A flowchart illustrating the method for editing 3D model textures provided in embodiments of this disclosure is shown.

[0103] Figure 2 This diagram illustrates the editing process of a 3D model texture provided in an embodiment of the present disclosure.

[0104] Figure 3 This diagram illustrates the overall editing process in the three-dimensional model texture editing method provided in this embodiment of the disclosure.

[0105] Figure 4 This diagram illustrates a partial editing process in the three-dimensional model texture editing method provided in this embodiment of the present disclosure.

[0106] Figure 5 A block diagram of an editing apparatus for three-dimensional model textures provided in an embodiment of the present disclosure is shown.

[0107] Figure 6 A block diagram of an electronic device 1900 provided in an embodiment of this disclosure is shown. Detailed Implementation

[0108] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0109] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0110] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0111] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0112] This disclosure provides a method for editing 3D model textures. It involves acquiring a target 3D model texture to be edited, obtaining target text description information for editing the target 3D model texture, and editing the target 3D model texture according to the target text description information to obtain an edited 3D model texture. This allows for direct editing of the 3D model texture based on text description information without altering the 3D model's mesh, thereby improving the flexibility of 3D model texture editing. In related technologies, editing models using pre-trained attributes is not flexible enough, and the editing functions are very limited. However, in this disclosure, by automatically editing the 3D model texture based on input text description information, editing becomes more flexible and the editing effects become more diverse. By adopting this disclosure, users can be guided to edit 3D model textures through natural language descriptions. This text-based editing method is more in line with human intuitive expression habits, lowers the technical threshold, and allows more users to easily personalize 3D model textures.

[0113] In this context, a mesh can represent the structure of a 3D model's surface. A 3D model is composed of polygons, which are actually composed of multiple triangles. Therefore, the surface of a 3D model is actually composed of multiple interconnected triangular faces. In 3D space, the set of points and edges that make up these triangles can be called a mesh.

[0114] The following description, in conjunction with the accompanying drawings, details the method for editing 3D model textures provided in the embodiments of this disclosure.

[0115] Figure 1A flowchart illustrating a method for editing a 3D model texture according to an embodiment of this disclosure is provided. In one possible implementation, the execution entity of the 3D model texture editing method can be a 3D model texture editing device. For example, the 3D model texture editing method can be executed by a terminal device, a server, or other electronic devices. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, or wearable device, etc. In some possible implementations, the 3D model texture editing method can be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the method for editing the 3D model texture includes steps S11 to S13.

[0116] In step S11, the texture of the target 3D model to be edited is obtained.

[0117] In step S12, target text description information for editing the target 3D model texture is obtained.

[0118] In step S13, the target 3D model texture is edited according to the target text description information to obtain the edited 3D model texture.

[0119] In this embodiment of the disclosure, the target 3D model texture can represent any 3D model texture requested by the user for editing.

[0120] In one possible implementation, the target 3D model texture is a 3D face texture. In this implementation, the 3D model can be a 3D face, and the 3D model texture can be a 3D face texture.

[0121] In this implementation, a target 3D face texture to be edited is obtained, along with target text description information for editing the target 3D face texture. The target 3D face texture is then edited based on the target text description information to obtain the edited 3D face texture. This allows for direct editing of the 3D face texture based on text description information without altering the 3D face mesh, thus improving the flexibility of 3D face texture editing. In related technologies, using pre-trained face attribute editing models is not flexible enough, and the editing functions are very limited. However, in this implementation, by automatically editing the 3D face texture based on the input text description information, editing becomes more flexible and the editing effects become more diverse.

[0122] Of course, the embodiments disclosed herein can also be applied to editing textures for other types of 3D models. For example, the 3D model can also be a 3D car, a 3D home appliance, a 3D toy, a 3D building, etc. Accordingly, the 3D model texture can be a 3D car texture, a 3D home appliance texture, a 3D toy texture, a 3D building texture, etc.

[0123] In this embodiment of the disclosure, the target text description information can represent text description information used for editing the texture of the target 3D model. The target text description information can be any information input by the user.

[0124] In this embodiment of the disclosure, after obtaining the target 3D model texture to be edited and the target text description information for editing the target 3D model texture, the target 3D model texture can be edited according to the target text description information to obtain the edited 3D model texture.

[0125] For example, if the target 3D model texture is a 3D face texture, and the user-inputted text description is "acartoon face", the target 3D model texture can be converted to a cartoon style based on this text description, resulting in an edited 3D model texture.

[0126] For example, if the target 3D model texture is a 3D face texture, and the user-inputted target text description is "aavatar face", then based on this target text description, the target 3D model texture can be converted into the style of the user's avatar, thus obtaining the edited 3D model texture.

[0127] For example, if the target 3D model texture is a 3D face texture, and the user-inputted target text description is "atattoo of flower", then based on this target text description, a flower tattoo can be added to the target 3D model texture, resulting in an edited 3D model texture.

[0128] For example, if the target 3D model texture is a 3D face texture, and the user-inputted target text description is "atattoo of love," then a heart-shaped tattoo can be added to the target 3D model texture based on this text description, resulting in an edited 3D model texture.

[0129] In one possible implementation, the editing type corresponding to the target text description information can be determined. The editing type can be either overall editing or partial editing.

[0130] As an example of this implementation, if the user selects a target editing area, the editing type corresponding to the target text description information can be determined to be partial editing; if the user does not select a target editing area, the editing type corresponding to the target text description information can be determined to be overall editing.

[0131] As another example of this implementation, the editing type corresponding to the target text description information can be determined based on the editing type selected by the user. In this example, a global editing control and a partial editing control can be displayed in the interactive interface; the editing type corresponding to the target text description information can be determined to be global editing in response to the triggering of the global editing control; and the editing type corresponding to the target text description information can be determined to be partial editing in response to the triggering of the partial editing control.

[0132] As another example of this implementation, edit type analysis can be performed on the target text description information to determine the corresponding edit type. For example, an edit type prediction model can be pre-trained. The target text description information can be input into the edit type prediction model, and the model can output the corresponding edit type.

[0133] In one possible implementation, the step of editing the target 3D model texture according to the target text description information to obtain the edited 3D model texture includes: generating editable potential code corresponding to the target 3D model texture; generating edited potential code according to the target text description information and the editable potential code; and generating the edited 3D model texture according to the edited potential code.

[0134] The latent code can be a set of numerical values ​​used to represent image features. In this implementation, after obtaining the texture of the target 3D model to be edited, an editable latent code corresponding to the target 3D model texture can be generated. By editing the editable latent code, the target 3D model texture can be edited.

[0135] In this implementation, editable latent code corresponding to the target 3D model texture is generated. Based on the target text description information and the editable latent code, edited latent code is generated. Then, based on the edited latent code, an edited 3D model texture is generated. This allows for overall editing of the 3D model texture, i.e., editing the entire 3D model texture. Furthermore, this implementation provides a flexible and efficient way to edit 3D model textures. Latent code, as a form of internal representation of the 3D model, can capture the inherent rules and characteristics of the 3D model texture. By editing the latent code, fine adjustments to the appearance of the 3D model can be achieved. This editing method simplifies the operation process and improves the accuracy and efficiency of editing.

[0136] In one possible implementation, the latent code can be latent code in the W+ space. Of course, in other possible implementations, the latent code can also be latent code in the W space, latent code in the Z space, etc., and this is not limited here.

[0137] In one possible implementation, generating the editable potential code corresponding to the target 3D model texture includes: generating the editable potential code corresponding to the target 3D model texture in response to the target text description information being used for overall editing of the target 3D model texture. In this implementation, when the target text description information is used for overall editing of the target 3D model texture, editable potential code corresponding to the target 3D model texture can be generated. Based on the target text description information and the editable potential code, edited potential code is generated, and an edited 3D model texture can be generated based on the edited potential code. Thus, overall editing of the 3D model texture can be achieved.

[0138] In one possible implementation, generating the editable potential code corresponding to the target 3D model texture includes: obtaining the potential code to be optimized corresponding to the target 3D model texture; generating a 3D model texture corresponding to the potential code to be optimized; optimizing the potential code to be optimized based on the 3D model texture corresponding to the potential code to be optimized and the target 3D model texture, until a preset stop optimization condition is met, thereby obtaining the editable potential code corresponding to the target 3D model texture.

[0139] In this implementation, the potential code to be optimized can first be generated through random initialization.

[0140] In this implementation, a 3D model texture corresponding to the potential code to be optimized can be generated by generating a model. Based on the differences between the 3D model texture corresponding to the potential code and the target 3D model texture, the potential code to be optimized is then optimized to obtain new potential code to be optimized. This step can be repeated until a preset stopping optimization condition is met. The preset stopping optimization condition could be that the similarity between the latest potential code's 3D model texture and the target 3D model texture is greater than or equal to a preset similarity, or that the potential code to be optimized has been optimized a preset number of times, etc.

[0141] In this implementation, in response to the satisfaction of a preset stop optimization condition, the latest potential code to be optimized can be used as the editable potential code corresponding to the texture of the target 3D model.

[0142] In this implementation, random initialization generates potential code to be optimized, providing a starting point for finding editable potential code corresponding to the target 3D model texture. Through subsequent optimization processes, the optimized potential code gradually approximates the features of the target 3D model texture. By continuously comparing the differences between the generated 3D model texture and the target 3D model texture, the potential code can be precisely adjusted to optimize the generation result. By optimizing the potential code to be optimized based on the difference information, the potential code gradually approaches the intrinsic representation of the target 3D model texture. Therefore, the editable potential code generated by this implementation can more accurately represent the target 3D model texture.

[0143] In another possible implementation, generating the editable potential code corresponding to the target 3D model texture includes: inputting the target 3D model texture into a pre-trained inverse model, and generating the editable potential code corresponding to the target 3D model texture through the inverse model.

[0144] The inverse model can be used to map input images into editable latent code. Furthermore, the inverse model can be pre-trained using a large amount of data.

[0145] In this implementation, the target 3D model texture can be input into the inverse model, and the inverse model can generate the editable potential code corresponding to the target 3D model texture.

[0146] In this implementation, editable latent code corresponding to the target 3D model texture is directly generated through a pre-trained inverse model, improving the efficiency and accuracy of generating editable latent code. Compared to traditional optimization processes, this method can directly obtain latent code highly consistent with the target 3D model texture without multiple iterations and adjustments, significantly saving computational resources and time costs. Furthermore, the inverse model has learned a large number of features and inherent patterns of the model texture during training, thus accurately capturing the unique attributes of the target 3D model texture. This ensures that the generated editable latent code not only accurately reflects the appearance of the target 3D model texture but also maintains its inherent structure and texture information, guaranteeing the quality and consistency of the generated results.

[0147] In this embodiment of the disclosure, when the target text description information is used to edit the texture of the target 3D model as a whole, the target text description information and editable latent code can be input into the mapping model, and the edited latent code can be output through the mapping model. The edited latent code has the relevant attributes described by the target text description information.

[0148] In one possible implementation, generating the edited potential code based on the target text description information and the editable potential code includes: training a mapping model based on the target text description information and the editable potential code; and, in response to the completion of the mapping model training, inputting the target text description information and the editable potential code into the trained mapping model to generate the edited potential code through the trained mapping model.

[0149] In this implementation, the mapping model can be trained using the editable latent code corresponding to the target text description information and the target 3D model texture. That is, in this implementation, different mapping models can be optimized for different target text description information and target 3D model textures.

[0150] In this implementation, a mapping model is trained by combining target text descriptions and editable latent code, thereby achieving a deep fusion of natural language descriptions and the intrinsic features of the 3D model. This allows the mapping model to learn how to accurately reflect the intent and requirements in the text descriptions onto the latent code, enabling precise editing of the 3D model textures. Furthermore, different mapping models are optimized for different target text descriptions and target 3D model textures. This personalized approach ensures that each mapping model can be precisely adjusted to specific editing needs. This significantly improves the flexibility and accuracy of editing, allowing users to perform customized editing operations according to their own requirements.

[0151] In one possible implementation, training the mapping model based on the target text description information and the editable latent code includes: inputting the target text description information and the editable latent code into the mapping model, generating a predicted edited latent code through the mapping model; obtaining a predicted rendered image corresponding to the predicted edited latent code; and training the mapping model based on the predicted rendered image and the target text description information.

[0152] In this implementation, the predicted editable latent code can represent the output of the mapping model during training, given the target text description information and the editable latent code. The predicted rendered image can represent the rendered image corresponding to the predicted editable latent code.

[0153] In this implementation, the predicted rendered image corresponding to the predicted edited potential code is obtained and compared with the target text description information, thereby evaluating the prediction accuracy of the mapping model. This evaluation mechanism helps to identify potential problems in the mapping model during the conversion process, thus guiding the subsequent training and optimization of the mapping model. By continuously adjusting the parameters of the mapping model, the prediction accuracy can be gradually improved, making it better meet the user's editing needs.

[0154] In one possible implementation, obtaining the predicted rendered image corresponding to the predicted edited latent code includes: generating a predicted edited 3D model texture map corresponding to the predicted edited latent code; and applying the predicted edited 3D model texture map onto the mesh of the target 3D model corresponding to the target 3D model texture map to obtain the predicted rendered image corresponding to the predicted edited latent code.

[0155] In this implementation, the predicted edited 3D model texture can represent the 3D model texture corresponding to the predicted edited potential code.

[0156] In this implementation, the predicted edited latent code generated by the mapping model can be converted into a 3D model texture, i.e., the predicted edited 3D model texture, through a decoding or reconstruction process. Applying this predicted edited 3D model texture to the mesh structure of the target 3D model and then rendering the entire 3D model yields the predicted rendered image corresponding to the predicted edited latent code.

[0157] In this implementation, a predicted edited 3D model texture corresponding to the predicted edited potential code is generated, and the predicted edited 3D model texture is applied to the mesh of the target 3D model corresponding to the target 3D model texture to obtain the predicted rendered image corresponding to the predicted edited potential code. This can verify the prediction accuracy of the mapping model, thereby helping to improve the reliability and stability of the editing process.

[0158] In one possible implementation, training the mapping model based on the predicted rendered image and the target text description information includes: determining the value of a loss function corresponding to the mapping model based on the similarity between the predicted rendered image and the target text description information; and training the mapping model based on the value of the loss function. In this implementation, the mapping model can be trained using backpropagation.

[0159] In another possible implementation, generating the edited potential code based on the target text description information and the editable potential code includes: determining a target mapping model; inputting the target text description information and the editable potential code into the target mapping model; and generating the edited potential code through the target mapping model.

[0160] In this implementation, a large amount of training data and textual description information can be used to train multiple mapping models.

[0161] In this implementation, by determining a target mapping model and inputting the target text description information and the editable potential code into the target mapping model, the edited potential code is generated through the target mapping model, thereby improving the speed of obtaining the edited potential code.

[0162] In one possible implementation, determining the target mapping model includes: determining keywords of the target text description information; and determining the target mapping model based on the keywords.

[0163] In this implementation, when a mapping model is needed, keywords for the target text description information are first determined, and then the target mapping model corresponding to the keywords is retrieved. If the target mapping model corresponding to the keywords is found, the target text description information and the editable potential code can be processed using the target mapping model to obtain the edited potential code. If the target mapping model corresponding to the keywords is not found, a new mapping model can be trained based on the target text description information and the editable potential code corresponding to the target 3D model texture.

[0164] In one possible implementation, generating the edited 3D model texture based on the edited potential code includes: inputting the edited potential code into a first preset generation model, and generating the edited 3D model texture through the first preset generation model.

[0165] In this implementation, the edited potential code can be processed using a first preset generation model to obtain the edited 3D model texture.

[0166] As an example of this implementation, the first preset generative model is a generative adversarial model. For example, the first preset generative model could be a styleGAN-based generative model, etc., without limitation.

[0167] In this example, by using a generative adversarial model as the first preset generative model, the edited potential code is input into the first preset generative model, and the edited 3D model texture is generated through the first preset generative model. This allows for precise control of the 3D model's properties through the first preset generative model, thereby helping to generate high-quality, realistic 3D model textures.

[0168] In one possible implementation, the step of editing the target 3D model texture according to the target text description information to obtain the edited 3D model texture includes: obtaining the location information of the target editing area of ​​the target 3D model texture; inputting the target 3D model texture, the location information of the target editing area, and the target text description information into a second preset generation model, and generating the edited 3D model texture through the second preset generation model.

[0169] In this implementation, the target editing region can represent a local area in the texture of the target 3D model that needs to be edited. The target editing region can be determined through user interaction, automatic detection, or preset parameters. The location information of the target editing region can include its coordinates, dimensions, etc.

[0170] The second preset generation model can be a pre-trained model that can locally edit the texture of a target 3D model based on the input text description and the location information of the target editing area. Through the calculation and processing of the second preset generation model, the edited 3D model texture can be obtained. The second preset generation model can adjust the texture, color, pattern, and other attributes of the target editing area according to the input text description to meet the user's editing needs. Simultaneously, because the location information of the target editing area is provided to the second preset generation model, it can accurately locate the area to be edited, achieving the effect of local editing.

[0171] As an example of this implementation, obtaining the location information of the target editing area of ​​the target 3D model texture includes: determining the location information of the target editing area of ​​the target 3D model texture based on the area painted by the user on the target 3D model texture.

[0172] In this example, the user can manually paint over the editing area. After the user finishes painting, the painted area can be designated as the target editing area.

[0173] In one example, the target 3D model texture after the target editing area is painted over, along with the target text description information, can be input into a second preset generation model. The second preset generation model then generates an edited 3D model texture, which edits the target editing area.

[0174] In this example, allowing users to specify the editing area by directly painting over the target 3D model texture provides an intuitive and easy-to-understand user interaction method. Users don't need complex parameter settings or interface operations; they can quickly locate and determine the position of the editing area simply by painting over it. Furthermore, the user-painted method makes the definition of the editing area highly flexible. Users can precisely paint editing areas of any shape and size as needed, thus meeting different editing requirements. This flexibility greatly improves the accuracy and personalization of the editing. Compared to other editing methods that require precise input of position information, the user-painted method reduces the operational difficulty. Users do not need professional graphic editing skills or precise hand-eye coordination; they can simply paint according to their intentions. This simplified operation method allows more users to easily perform local editing of 3D model textures. Because users can directly paint the editing area on the model texture, the selection and positioning of the editing area can be completed quickly. This greatly shortens the time required for the editing process and improves editing efficiency. Users can complete complex editing tasks in a shorter time, thereby accelerating project progress or improving work efficiency.

[0175] As an example of this implementation, the second preset generation model is an image generation model based on the diffusion process. For example, the second preset generation model can use a generation model based on stable diffusion, etc., and is not limited here.

[0176] In this example, by using an image generation model based on a diffusion process as the second preset generation model, the target 3D model texture, the location information of the target editing area, and the target text description information are input into the second preset generation model, and the edited 3D model texture is generated through the second preset generation model, thereby helping to generate high-quality and diverse 3D model textures.

[0177] The 3D model texture editing method provided in this disclosure can be applied to technical fields such as digital human creation, and is not limited thereto. Using the 3D model texture editing method provided in this disclosure, a text-described, description-driven 3D face texture editing system, etc., can be implemented.

[0178] The following describes the method for editing 3D model textures provided in this embodiment through a specific application scenario. In this application scenario, the 3D model texture can be a 3D face texture. Figure 2 This diagram illustrates the editing process of a 3D model texture provided in an embodiment of this disclosure. Examples of the overall and partial editing processes for 3D face textures are given below.

[0179] I. Overall Editing (corresponding to) Figure 2 Editing method 1 in the middle):

[0180] Figure 3 This diagram illustrates the overall editing process in the three-dimensional model texture editing method provided in this embodiment of the disclosure.

[0181] Combination Figure 2 and Figure 3 First, it is possible to obtain 3D face textures and target text description information (i.e., Figure 2 and Figure 3 (The "text description" in the text). Figure 3 Two examples of overall editing of 3D face textures are shown, with target text descriptions of "a cartoon face" and "a avatar face," respectively.

[0182] 3D face textures can be input into a pre-trained inverse model, which can then generate editable latent code corresponding to the 3D face textures.

[0183] The target text description information and editable potential code can be input into the mapping model, and the mapping model outputs the edited potential code.

[0184] The edited latent code can be input into a generative model (such as a styleGAN-based generative model) to obtain the edited 3D face texture. Figure 3The image shows the edited 3D face texture corresponding to the target text description information "a cartoon face" and the edited 3D face texture corresponding to the target text description information "a avatar face".

[0185] II. Local Editing (corresponding to...) Figure 2 Editing Method Two in China:

[0186] Figure 4 This diagram illustrates a partial editing process in the three-dimensional model texture editing method provided in this embodiment of the present disclosure.

[0187] Combination Figure 2 and Figure 4 First, it is possible to obtain 3D face textures and target text description information (i.e., Figure 2 and Figure 4 (The "text description" in the text). Figure 4 Two examples of local editing of 3D face textures are shown, with target text descriptions of "a tattoo of flower" and "a tattoo of love," respectively.

[0188] Users can manually paint over the editing area. After the user finishes painting, the painted area can be designated as the target editing area.

[0189] The smeared 3D face texture and target text description information can be input into the diffusion model, and the edited 3D face texture can be output through the diffusion model. Figure 4 The image shows the edited 3D face texture corresponding to the target text information "a tattoo of flower" and the edited 3D face texture corresponding to the target text description information "a tattoo of love".

[0190] By adopting this application scenario, 3D face textures can be edited based on user-inputted text descriptions, even with a fixed grid, thus enabling the editing of 3D avatars. Furthermore, this application scenario allows for both overall and partial editing.

[0191] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0192] In addition, this disclosure also provides a three-dimensional model texture editing device, electronic device, computer-readable storage medium, and computer program product, all of which can be used to implement any of the three-dimensional model texture editing methods provided in this disclosure. The corresponding technical solutions and effects can be found in the relevant descriptions in the method section, and will not be repeated here.

[0193] Figure 5 This diagram illustrates a block diagram of an editing apparatus for three-dimensional model textures provided in an embodiment of this disclosure. Figure 5 As shown, the 3D model texture editing device includes:

[0194] The first acquisition module 51 is used to acquire the texture of the target 3D model to be edited;

[0195] The second acquisition module 52 is used to acquire target text description information for editing the target 3D model texture;

[0196] The editing module 53 is used to edit the target 3D model texture according to the target text description information to obtain the edited 3D model texture.

[0197] In one possible implementation, the editing module 53 is used for:

[0198] Generate editable potential code corresponding to the texture of the target 3D model;

[0199] Based on the target text description information and the editable potential code, generate the edited potential code;

[0200] Based on the edited potential code, an edited 3D model texture is generated.

[0201] In one possible implementation, the editing module 53 is used for:

[0202] In response to the target text description information used to perform overall editing of the target 3D model texture, editable potential code corresponding to the target 3D model texture is generated.

[0203] In one possible implementation, the editing module 53 is used for:

[0204] Train a mapping model based on the target text description information and the editable potential code;

[0205] In response to the completion of the mapping model training, the target text description information and the editable potential code are input into the trained mapping model, and the edited potential code is generated through the trained mapping model.

[0206] In one possible implementation, the editing module 53 is used for:

[0207] The target text description information and the editable latent code are input into the mapping model, and the predicted edited latent code is generated through the mapping model;

[0208] Obtain the predicted rendered image corresponding to the edited potential code of the prediction;

[0209] The mapping model is trained based on the predicted rendered image and the target text description information.

[0210] In one possible implementation, the editing module 53 is used for:

[0211] Generate the edited 3D model texture corresponding to the edited potential code of the prediction;

[0212] The predicted edited 3D model texture is applied to the mesh of the target 3D model corresponding to the target 3D model texture to obtain the predicted rendered image corresponding to the predicted edited latent code.

[0213] In one possible implementation, the editing module 53 is used for:

[0214] The value of the loss function corresponding to the mapping model is determined based on the similarity between the predicted rendered image and the target text description information;

[0215] The mapping model is trained based on the value of the loss function.

[0216] In one possible implementation, the editing module 53 is used for:

[0217] Determine the target mapping model;

[0218] The target text description information and the editable potential code are input into the target mapping model, and the edited potential code is generated through the target mapping model.

[0219] In one possible implementation, the editing module 53 is used for:

[0220] Determine the keywords of the target text description information;

[0221] Based on the keywords, determine the target mapping model.

[0222] In one possible implementation, the editing module 53 is used for:

[0223] Obtain the potential code to be optimized corresponding to the texture of the target 3D model;

[0224] Generate the 3D model texture map corresponding to the potential code to be optimized;

[0225] Based on the 3D model texture corresponding to the potential code to be optimized and the target 3D model texture, the potential code to be optimized is optimized until a preset stop optimization condition is met, thereby obtaining the editable potential code corresponding to the target 3D model texture.

[0226] In one possible implementation, the editing module 53 is used for:

[0227] The target 3D model texture is input into a pre-trained inverse model, and the inverse model generates editable potential code corresponding to the target 3D model texture.

[0228] In one possible implementation, the editing module 53 is used for:

[0229] The edited potential code is input into the first preset generation model, and the edited 3D model texture is generated through the first preset generation model.

[0230] In one possible implementation, the first preset generative model is a generative adversarial model.

[0231] In one possible implementation, the editing module 53 is used for:

[0232] Obtain the location information of the target editing area of ​​the target 3D model texture;

[0233] The target 3D model texture, the location information of the target editing area, and the target text description information are input into the second preset generation model, and the edited 3D model texture is generated through the second preset generation model.

[0234] In one possible implementation, the editing module 53 is used for:

[0235] Based on the area that the user has applied to the texture of the target 3D model, the location information of the target editing area of ​​the texture of the target 3D model is determined.

[0236] In one possible implementation, the second preset generation model is an image generation model based on the diffusion process.

[0237] In one possible implementation, the target 3D model texture is a 3D face texture.

[0238] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation and technical effects can be referred to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0239] This disclosure also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.

[0240] This disclosure also proposes a computer program including computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device executes the above-described method.

[0241] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, the processor in the electronic device executes the above-described method.

[0242] This disclosure also provides an electronic device, including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the above-described method.

[0243] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0244] Figure 6 A block diagram of an electronic device 1900 provided according to an embodiment of this disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal. (Refer to...) Figure 6 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0245] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Server operating system (Windows Server). TM Apple's graphical user interface-based operating system (MacOS X) TM ), a multi-user, multi-process computer operating system (Unix) TMLinux is a free and open-source Unix-like operating system. TM ), the open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0246] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0247] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0248] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0249] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0250] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0251] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0252] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0253] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0254] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0255] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0256] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0257] If the technical solution of this disclosure involves personal information, the product applying the technical solution of this disclosure has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this disclosure involves sensitive personal information, the product applying the technical solution of this disclosure has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to indicate that the user has entered the scope of personal information collection and that personal information will be collected. If the user voluntarily enters the collection scope, it is deemed to have consented to the collection of their personal information; or on the personal information processing device, with clear signs / information informing the user of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0258] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for editing textures on a 3D model, characterized in that, include: Obtain the texture of the target 3D model to be edited; Obtain target text description information for editing the texture of the target 3D model, wherein the editing type corresponding to the target text description information includes overall editing or partial editing; Based on the target text description information, the target 3D model texture is edited to obtain an edited 3D model texture. The editing method does not change the mesh of the 3D model. Specifically, the target 3D model texture is edited based on the target text description information to obtain an edited 3D model texture, including: When the editing type corresponding to the target text description information is the overall editing, the target 3D model texture is input into a pre-trained inverse model, and the inverse model generates editable latent code corresponding to the target 3D model texture; the target text description information and the editable latent code are input into a mapping model, and the mapping model outputs the edited latent code; the edited latent code is input into a generation model to obtain the edited 3D model texture; When the editing type corresponding to the target text description information is the local editing, the target editing area of ​​the target 3D model texture is obtained; the target 3D model texture with the target editing area and the target text description information are input into the diffusion model, and the edited 3D model texture is generated through the diffusion model.

2. The method according to claim 1, characterized in that, The step of editing the target 3D model texture based on the target text description information to obtain the edited 3D model texture includes: Generate editable potential code corresponding to the texture of the target 3D model; Based on the target text description information and the editable potential code, generate the edited potential code; Based on the edited potential code, an edited 3D model texture is generated.

3. The method according to claim 2, characterized in that, The editable potential code that generates the texture corresponding to the target 3D model includes: In response to the target text description information used to perform overall editing of the target 3D model texture, editable potential code corresponding to the target 3D model texture is generated.

4. The method according to claim 2 or 3, characterized in that, The step of generating edited potential code based on the target text description information and the editable potential code includes: Train a mapping model based on the target text description information and the editable potential code; In response to the completion of the mapping model training, the target text description information and the editable potential code are input into the trained mapping model, and the edited potential code is generated through the trained mapping model.

5. The method according to claim 4, characterized in that, The step of training the mapping model based on the target text description information and the editable potential code includes: The target text description information and the editable latent code are input into the mapping model, and the predicted edited latent code is generated through the mapping model; Obtain the predicted rendered image corresponding to the edited potential code of the prediction; The mapping model is trained based on the predicted rendered image and the target text description information.

6. The method according to claim 5, characterized in that, Obtaining the predicted rendered image corresponding to the predicted edited potential code includes: Generate the edited 3D model texture corresponding to the edited potential code of the prediction; The predicted edited 3D model texture is applied to the mesh of the target 3D model corresponding to the target 3D model texture to obtain the predicted rendered image corresponding to the predicted edited latent code.

7. The method according to claim 5, characterized in that, The step of training the mapping model based on the predicted rendered image and the target text description information includes: The value of the loss function corresponding to the mapping model is determined based on the similarity between the predicted rendered image and the target text description information; The mapping model is trained based on the value of the loss function.

8. The method according to claim 2, characterized in that, The step of generating edited potential code based on the target text description information and the editable potential code includes: Determine the target mapping model; The target text description information and the editable potential code are input into the target mapping model, and the edited potential code is generated through the target mapping model.

9. The method according to claim 8, characterized in that, The determination of the target mapping model includes: Determine the keywords of the target text description information; Based on the keywords, determine the target mapping model.

10. The method according to claim 2 or 3, characterized in that, The editable potential code that generates the texture corresponding to the target 3D model includes: Obtain the potential code to be optimized corresponding to the texture of the target 3D model; Generate the 3D model texture map corresponding to the potential code to be optimized; Based on the 3D model texture corresponding to the potential code to be optimized and the target 3D model texture, the potential code to be optimized is optimized until a preset stop optimization condition is met, thereby obtaining the editable potential code corresponding to the target 3D model texture.

11. The method according to claim 2 or 3, characterized in that, The editable potential code that generates the texture corresponding to the target 3D model includes: The target 3D model texture is input into a pre-trained inverse model, and the inverse model generates editable potential code corresponding to the target 3D model texture.

12. The method according to claim 2 or 3, characterized in that, The step of generating an edited 3D model texture based on the edited potential code includes: The edited potential code is input into the first preset generation model, and the edited 3D model texture is generated through the first preset generation model.

13. The method according to claim 12, characterized in that, The first preset generative model is a generative adversarial model.

14. The method according to claim 1, characterized in that, The step of editing the target 3D model texture based on the target text description information to obtain the edited 3D model texture includes: Obtain the location information of the target editing area of ​​the target 3D model texture; The target 3D model texture, the location information of the target editing area, and the target text description information are input into the second preset generation model, and the edited 3D model texture is generated through the second preset generation model.

15. The method according to claim 14, characterized in that, The step of obtaining the location information of the target editing area of ​​the target 3D model texture includes: Based on the area that the user has applied to the texture of the target 3D model, the location information of the target editing area of ​​the texture of the target 3D model is determined.

16. The method according to claim 14 or 15, characterized in that, The second preset generation model is an image generation model based on the diffusion process.

17. The method according to claim 1, characterized in that, The target 3D model texture is a 3D human face texture.

18. A device for editing textures of a three-dimensional model, characterized in that, include: The first acquisition module is used to acquire the texture of the target 3D model to be edited; The second acquisition module is used to acquire target text description information for editing the target 3D model texture, wherein the editing type corresponding to the target text description information includes overall editing or partial editing. The editing module is used to edit the target 3D model texture according to the target text description information to obtain the edited 3D model texture. The editing method does not change the mesh of the 3D model. The editing module is used for: When the editing type corresponding to the target text description information is the overall editing, the target 3D model texture is input into a pre-trained inverse model, and the inverse model generates editable latent code corresponding to the target 3D model texture; the target text description information and the editable latent code are input into a mapping model, and the mapping model outputs the edited latent code; the edited latent code is input into a generation model to obtain the edited 3D model texture; When the editing type corresponding to the target text description information is the local editing, the target editing area of ​​the target 3D model texture is obtained; the target 3D model texture with the target editing area and the target text description information are input into the diffusion model, and the edited 3D model texture is generated through the diffusion model.

19. An electronic device, characterized in that, include: One or more processors; Memory used to store executable instructions; The one or more processors are configured to invoke executable instructions stored in the memory to perform the method according to any one of claims 1 to 17.

20. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 17.

21. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, characterized in that, When the computer-readable code is run in an electronic device, the processor in the electronic device performs the method according to any one of claims 1 to 17.

Images