Method and system for generating three-dimensional target texture on the basis of text, and storage medium

Abstract

A method and system for generating a three-dimensional target texture on the basis of text, and a storage medium. The method comprises: obtaining three-dimensional model data and corresponding descriptive text data; generating a depth map and a current normal line map at a corresponding angle of view, and inputting the descriptive text data and the depth map to a depth map diffusion model, to obtain a set of synthetic images; and performing multi-stage generative training, performing multi-stage iterative updating, and on the basis of a restored texture map and a texture map obtained after completing the multi-stage iterative updating, generating a three-dimensional target texture map. According to the present invention, a stage-by-stage advancing method is used, a region having a relatively large discontinuity in generated texture is updated by means of calculation using a normal line map, the generated texture map has a higher definition and is more delicate, and a texture map of a current stage is outputted on the basis of an updated weight score. Therefore, the consistency of the texture map of the current stage and a synthetic map can be enhanced, which is also equivalent to matching semantics of input text (i.e., the descriptive text data), so that the generated three-dimensional target texture map is more accurate.

Description

Method, system and storage medium for generating three-dimensional target texture based on text Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, system and storage medium for generating three-dimensional target texture based on text. Background Art

[0002] Artificial intelligence technology is deeply integrated into various industries, injecting innovation into traditional sectors and achieving breakthroughs. It is also driving its own development to higher levels. AI-powered content generation technology has become a key driver of this transformation into the AI ​​era. With continued in-depth research and exploration, AI-powered content generation technology has achieved significant results in multiple fields. This AI technology can creatively generate text, images, and audio content, with the generation of image content guided by text becoming a popular research and application area.

[0003] Text-guided image generation involves inputting a personalized descriptive sentence and using artificial intelligence to generate an image that matches the input description. Existing technologies mostly rely on large pre-trained models or generative adversarial networks to achieve this. With the development of text-guided image generation technology, many AI-based solutions for 3D object texture generation have been proposed. Among these, text-guided image generation methods stand out, capable of generating more creative 3D object textures.

[0004] However, existing methods for generating three-dimensional target textures based on text have large semantic differences between the generated content and the input text and are not very accurate.

[0005] Summary of the Invention

[0006] The main purpose of the present invention is to provide a method, system and storage medium for generating three-dimensional target texture based on text, aiming to solve the technical problems of existing methods for generating three-dimensional target texture based on text, such as large semantic differences between generated content and input text and low accuracy.

[0007] To achieve the above object, the present invention provides a method for generating a three-dimensional target texture based on text, which comprises the following steps: obtaining three-dimensional model data and corresponding description text data view , the description text data text view Including perspective information; based on 3D model data and descriptive text data view The viewing angle information generates the depth map depth under the corresponding viewing angle view and the current normal map normal viewi , and describe the text data textview and depth map depth view Input into the depth map diffusion model to obtain a set of synthetic image sets; perform multi-stage generative training, each stage includes at least the following steps: select a synthetic image image from the synthetic image set generate ; Get the normal update mask normal Update mask with texture texture And merge and superimpose the two to obtain the texture repair mask inpaint ; The composite image image generate Repairing the mask with texture inpaint Input to the image restoration diffusion model SD inpaint , get the repair texture inpaint ; Based on the repair texture inpaint Render and project the 3D model data to obtain a 2D image; calculate the 2D image and the composite image generate Similarity and update weight score, and output the current stage texture map based on the updated weight score; perform multi-stage iterative update, based on the repair texture map texture inpaint The texture map obtained after completing the multi-stage iterative update is used to generate a three-dimensional target texture map.

[0008] Optional, 3D model data and description text data view It is obtained after preprocessing, which includes at least the following steps: performing spatial position processing on the initial 3D model data to obtain its grid rendering image on the 2D texture coordinate system; combining the initial 3D model data with the initial description text data according to multiple preset texture synthesis perspectives to obtain 3D model data, and outputting multiple description text data including perspective information view ; The number of stages of multi-stage generative training is the same as the number of preset texture synthesis perspectives.

[0009] Optionally, select a composite image from the composite image set generate Specifically, in the first stage of multi-stage generative training, a synthetic image is randomly selected from the synthetic image set. The subsequent stage is based on the composite image selected in the previous stage, and a composite image image is selected from the composite image set. generate .

[0010] Optional, normal update mask normal Obtained by the following steps: Get the current normal map Compared with the normal map in the previous stage Based on the current normal map Compared with the normal map in the previous stage Calculate the normal update mask normal ; Texture update mask texture Obtained by the following steps: Get the initial texture map texture0 and the i-th iteration stage texture map texture i ; Based on the initial texture map texture0 and the i-th iteration stage texture map texture i , calculate the texture update mask mask texture .

[0011] Optional, mask inpaint =mask normal ∪mask texture ,in, Indicates taking the current normal map The z value of the normal vector, To obtain the z value of the normal vector of the normal map in the previous stage, when the difference between the z values ​​corresponding to the two stages is greater than 0.3, the calculated pixel point is the area that needs to be updated; mask texture ={xyz(texture i -texture0)>0.7}, xyz(texture i -texture0) represents the initial texture map texture0 and the texture map texture of the i-th iteration stage i When the pixel distance is greater than 0.7, the corresponding pixel area needs to be updated.

[0012] Optionally, in the first stage of multi-stage generative training, mask normal Blank, mask texture for the entire texture area.

[0013] Optionally, the update weight score is calculated according to the following formula: Among them, sorce represents the update weight score, whose value range is [0, 1], M represents the target plane area, j represents a point in the target plane area, noraml represents regularization, render(texture inpaint ) represents the rendering of 3D model data, pro(render(texture inpaint )) represents the projection transformation of three-dimensional model data.

[0014] Optionally, the texture map of the current stage is obtained according to the following formula:

[0015] texture=(1-N)*texture+N*((texture inpaint -texture)*sorce*r+(1-r)*texture); where texture represents the texture map at the current stage, N represents the area where the target is located in the texture map, and r is the update weight.

[0016] Corresponding to the method for generating three-dimensional target texture based on text, the present invention provides a system for generating three-dimensional target texture based on text, which includes: a data acquisition module for acquiring three-dimensional model data and corresponding description text data view , the description text data text view Including perspective information; synthetic image set generation module for 3D model data and description text data view The viewing angle information generates the depth map depth under the corresponding viewing angle view and the current normal map And describe the text data text view and depth map depth view Input into the depth map diffusion model to obtain a set of synthetic image sets; the image group difference elimination and screening module is used to select a synthetic image from the synthetic image set. generate ; Texture repair mask generation module, used to obtain normal update mask normal Update mask with texture texture And merge and superimpose the two to obtain the texture repair mask inpaint ; Repair the texture map generation module, used to synthesize the image generate Repairing the mask with texture inpaint Input to the image restoration diffusion model SD inpaint , get the repair texture inpaint ; Rendering and projection module, used to repair texture based on texture inpaint Render and project the 3D model data to obtain a 2D image; update the weight score calculation module to calculate the 2D image and the composite image generate Similarity and update weight score; Current texture map generation module, used to output the current stage texture map based on the updated weight score; 3D target texture map generation module, used to perform the restoration based on the texture map inpaint The texture map obtained after completing the multi-stage iterative update is used to generate a three-dimensional target texture map.

[0017] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which is stored a program for generating a three-dimensional target texture based on text. When the program for generating a three-dimensional target texture based on text is executed by a processor, the steps of the method for generating a three-dimensional target texture based on text as described above are implemented.

[0018] The beneficial effects of the present invention are:

[0019] (1) Compared with the prior art, the present invention can overcome the problem of large differences in sampling sources during texture generation by selecting a synthetic image, thereby ensuring the continuity of the final generated three-dimensional target texture image; and the entire texture generation process adopts a stage-by-stage approach, using the calculation of the normal map to update the generated texture in areas with large discontinuities. The texture image generated by this strategy is more high-definition and delicate, and combined with the depth map as the input of the generation model, the generated texture can better fit the target shape; at the same time, the two-dimensional image and the synthetic image are calculated. generate The similarity and updated weight score of the texture map of the current stage are output based on the updated weight score, which can enhance the consistency between the texture map of the current stage and the synthesized image. It is also equivalent to matching the semantics of the input text (i.e., describing the text data), making the generated three-dimensional target texture map more accurate.

[0020] (2) Compared with the prior art, the present invention outputs multiple descriptive text data including perspective information through preprocessing. view , the 3D model data and the corresponding description text data view As training data, a large amount of training data is not required;

[0021] (3) Compared with the prior art, the present invention can eliminate the influence of depth map differences in the depth map diffusion model by selecting a synthetic image. Since the difference in the input depth map can easily lead to inconsistent shallow features of the image output by the depth map diffusion model, including color, pattern, etc., which will have a significant impact on the continuity of the generated texture, the influence of the difference can be avoided by selecting a synthetic image.

[0022] (4) Compared with the prior art, the present invention is based on the repair texture map inpaint Render and project the 3D model data to obtain a 2D image, and then calculate the 2D image and the composite image generate The similarity and update weight score of the current stage texture map are output based on the updated weight score, which can make the generated three-dimensional target texture map consistent with the description text data view Some have higher matching accuracy;

[0023] (5) Compared with the existing technology, the present invention can greatly improve the generalization of content generation capabilities by using a pre-trained depth map diffusion model, so that the method of generating three-dimensional target texture based on text described in the present invention can be quickly put into practice. The method of generating three-dimensional target texture based on text described in the present invention not only has a significant breakthrough in effect, but also has high versatility and practical application significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] FIG1 is a simplified flow chart of a method for generating a three-dimensional target texture based on text according to an embodiment of the present invention;

[0026] FIG2 is a schematic diagram of an embodiment of a method for generating a three-dimensional target texture based on text according to the present invention;

[0027] FIG3 is a framework diagram of an embodiment of a system for generating three-dimensional target texture based on text according to the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0029] As shown in Figures 1 and 2, a method for generating a three-dimensional target texture based on text of the present invention comprises the following steps: obtaining three-dimensional model data and corresponding description text data view , the description text data text view Including perspective information; based on 3D model data and descriptive text data view The viewing angle information generates the depth map depth under the corresponding viewing angle view and the current normal map And describe the text data text view and depth map depth view Input into the depth map diffusion model to obtain a set of synthetic image sets; perform multi-stage generative training, each stage includes at least the following steps: select a synthetic image image from the synthetic image set generate; Get the normal update mask normal Update mask with texture texture And merge and superimpose the two to obtain the texture repair mask inpaint ; The composite image image generate Repairing the mask with texture inpaint Input to the image restoration diffusion model SD inpaint , get the repair texture inpaint ; Based on the repair texture inpaint Render and project the 3D model data to obtain a 2D image; calculate the 2D image and the composite image generate Similarity and update weight score, and output the current stage texture map based on the updated weight score; perform multi-stage iterative update, based on the repair texture map texture inpaint The texture map obtained after completing the multi-stage iterative update is used to generate a three-dimensional target texture map.

[0030] Preferably, the depth map diffusion model SD depth For pre-training, the stable-diffusion-2-depth model is used.

[0031] Preferably, the image restoration diffusion model SD inpaint Adopt the stable-diffusion-inpainting model.

[0032] By selecting a synthetic image, the present invention can overcome the problem of large differences in sampling sources during texture generation, ensuring the continuity of the final generated three-dimensional target texture image; and the entire texture generation process adopts a stage-by-stage approach, using the calculation of the normal map to update the generated texture in areas with large discontinuities. The texture map generated by this strategy is more high-definition and delicate, and combined with the depth map as the input of the generation model, the generated texture can better fit the target shape; at the same time, the two-dimensional image and the synthetic image are calculated. generate The similarity and updated weight score of the texture map of the current stage are output based on the updated weight score, which can enhance the consistency between the texture map of the current stage and the synthesized image. It is also equivalent to matching the semantics of the input text (i.e., describing the text data), making the generated three-dimensional target texture map more accurate.

[0033] In this embodiment, the three-dimensional model data and the description text data viewIt is obtained after preprocessing, which includes at least the following steps: performing spatial position processing on the initial 3D model data to obtain its grid rendering image on the 2D texture coordinate system, and converting the target texture in the 3D space into the texture in the 2D plane through spatial position processing; combining the initial 3D model data with the initial description text data according to multiple preset texture synthesis perspectives to obtain the 3D model data, and outputting multiple description text data including perspective information view .

[0034] Preferably, the three-dimensional model data is obj model data. The plurality of preset texture synthesis perspectives are two or more of the six perspectives of front, back, left, right, top and bottom.

[0035] The present invention outputs multiple descriptive text data including perspective information through preprocessing view , the 3D model data and the corresponding description text data view As training data, a large amount of training data is not required.

[0036] Preferably, the number of stages of the multi-stage generative training is the same as the number of preset texture synthesis perspectives. For example, if the number of preset texture synthesis perspectives is 4, the number of stages of the multi-stage generative training is also 4.

[0037] In this embodiment, a composite image image is selected from the composite image set. generate Specifically, in the first stage of multi-stage generative training, a synthetic image image is randomly selected from the synthetic image set. generate The subsequent stage is based on the composite image selected in the previous stage, and a composite image image is selected from the composite image set. generate .

[0038] Preferably, the subsequent stage is based on the composite image selected in the previous stage, and a composite image image is selected from the composite image set. generate Specifically, a composite image is selected that is complete, matches the depth map, and has the most similar image features to the composite image selected in the previous stage. generate .

[0039] In this embodiment, the selection of the composite image is performed by the image difference elimination module, which can be specifically expressed by the following formula: Among them, IRB represents the image difference elimination module, Indicates the smallest value of i in the selection condition, Represents the composite image image generate Perform feature calculations, Indicates feature calculation for a certain image in the synthetic image set.

[0040] By selecting a synthetic image, the present invention can eliminate the influence of depth map differences in the depth map diffusion model. Since the difference in the input depth map can easily lead to inconsistent shallow features of the image output by the depth map diffusion model, including color, pattern, etc., which will have a significant impact on the continuity of the generated texture, the influence caused by the difference can be avoided by selecting a synthetic image.

[0041] In this embodiment, the normal update mask mask normal Obtained by the following steps: Get the current normal map Compared with the normal map in the previous stage Based on the current normal map Compared with the normal map in the previous stage Calculate the normal update mask normal ; Texture update mask texture Obtained by the following steps: Get the initial texture map texture0 and the i-th iteration stage texture map texture i ; Based on the initial texture map texture0 and the i-th iteration stage texture map texture i , calculate the texture update mask mask texture .

[0042] Preferably, in the first stage of multi-stage generative training, the initial texture map is the default background map, and the subsequent stages are texture maps generated in the previous stage. The texture map is a synthetic image selected from a synthetic image set and expanded according to the three-dimensional model data.

[0043] In this embodiment, mask inpaint =mask normal ∪mask texture ,in, Indicates taking the current normal map The z value of the normal vector, To obtain the z value of the normal vector of the normal map in the previous stage, when the difference between the z values ​​corresponding to the two stages is greater than 0.3, the calculated pixel point is the area that needs to be updated; mask texture ={xyz(texture i -texture0)>0.7}, xyz(texture i -texture0) represents the initial texture map texture0 and the texture map texture of the i-th iteration stage i When the pixel distance is greater than 0.7, the corresponding pixel area needs to be updated.

[0044] In this embodiment, the texture repair mask determines whether it needs to be iteratively updated based on the judgment conditions. If the difference in the z values ​​corresponding to the two stages is greater than 0.3, the texture repair mask is determined to need to be iteratively updated from the normal perspective; if the pixel point distance is greater than 0.7, the corresponding pixel area (the area where the pixel point distance is greater than 0.7) is similar to the background, and the texture repair mask is determined to need to be iteratively updated from the pixel perspective.

[0045] Preferably, in the first stage of multi-stage generative training, mask normal Blank, mask texture for the entire texture area.

[0046] In this embodiment, the updated weight score is calculated according to the following formula: Among them, sorce represents the update weight score, whose value range is [0, 1], M represents the target plane area, noraml represents regularization, j represents a point in the target plane area, render(texture inpaint ) represents the rendering of 3D model data, pro(render(texture inpaint )) represents the projection transformation of three-dimensional model data.

[0047] In this embodiment, the larger the value of sorce is, the better the texture image is. inpaint The semantics of the description text is more consistent, and the effect of the final reconstructed three-dimensional texture is also guaranteed.

[0048] The present invention repairs texture by inpaint Render and project the 3D model data to obtain a 2D image, and then calculate the 2D image and the composite image generate The similarity and update weight score of the current stage texture map are output based on the updated weight score, which can make the generated three-dimensional target texture map consistent with the description text data view Some have higher matching accuracy.

[0049] In this embodiment, the texture map of the current stage is obtained according to the following formula:

[0050] texture=(1-N)*texture+N*((texture inpaint -texture)*sorce*r+(1-r)*texture); where texture represents the texture map at the current stage, N represents the area where the target (the three-dimensional target) is located in the texture map, and r is the update weight.

[0051] Preferably, r=0.6.

[0052] The present invention requires retaining the current stage texture map at each stage for subsequent iterations, ultimately outputting a complete 3D target texture map. This 3D target texture map can be rendered using rasterization based on the previous 3D model data, creating a pattern consistent with the text input description.

[0053] By using a pre-trained depth map diffusion model, the present invention can greatly improve the generalization of content generation capabilities, so that the method of generating three-dimensional target textures based on text described in the present invention can be quickly put into practice. The method of generating three-dimensional target textures based on text described in the present invention not only has a significant breakthrough in effect, but also has high versatility and practical application significance.

[0054] As shown in FIG3 , the present invention also provides a system for generating a three-dimensional target texture based on text, which includes: a data acquisition module 100 for acquiring three-dimensional model data and corresponding description text data view , the description text data text view The synthetic image set generation module 200 is used to generate the image based on the three-dimensional model data and the description text data text view The viewing angle information generates the depth map depth under the corresponding viewing angle view and the current normal map And describe the text data text view and depth map depth view Input to the depth map diffusion model to obtain a set of synthetic image sets; the image group difference elimination screening module 301 is used to select a synthetic image image from the synthetic image set generate ; Texture repair mask generation module 302, used to obtain the normal update mask mask normal Update mask with texture texture And merge and superimpose the two to obtain the texture repair mask inpaint ; Repair texture map generation module 303, used to synthesize the image generate Repairing the mask with texture inpaint Input to the image restoration diffusion model SD inpaint , get the repair texture inpaint ; Rendering and projection module 304, for repairing texture map texture inpaint Render and project the three-dimensional model data to obtain a two-dimensional image; update the weight score calculation module 305 to calculate the weight of the two-dimensional image and the composite image. Similarity and update weight score; Current texture map generation module 306, for outputting the current stage texture map based on the updated weight score; 3D target texture map generation module 400, for performing the restoration based on the texture map inpaint The texture map obtained after completing the multi-stage iterative update is used to generate a three-dimensional target texture map.

[0055] In this embodiment, the data acquisition module 100 is also used to acquire the initial texture map texture0 and the current stage texture map texture i .

[0056] An embodiment of the present invention further provides a computer-readable storage medium. This computer-readable storage medium may be included in the memory described in the above embodiment, or may be a separate computer-readable storage medium not incorporated into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the method for generating a three-dimensional target texture based on text, as shown in FIG1 . The computer-readable storage medium may be a read-only memory, a magnetic disk, an optical disk, or the like.

[0057] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between the various embodiments can be referred to in conjunction with each other. For the apparatus, device, and storage medium embodiments, since they are generally similar to the method embodiments, their descriptions are relatively simplified. For relevant parts, refer to the descriptions of the method embodiments.

[0058] Furthermore, in this document, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0059] While the foregoing description shows and describes preferred embodiments of the present invention, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments, and can be modified within the scope of the present invention by the teachings herein or by techniques or knowledge in the relevant art. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the present invention are intended to be within the scope of the appended claims.

Claims

1. A method for generating a three-dimensional target texture based on text, characterized in that: The following steps are involved: Get the 3D model data and the corresponding description text data view , the description text data text view The viewpoint information is included; Based on 3D model data and description text data view The viewing angle information is used to generate the depth map depth at the corresponding viewing angle. view and the current normal map And describe the text data text view and depth map depth view Input into the depth map diffusion model to obtain a set of synthetic images; Perform multi-stage generative training, where each stage includes at least the following steps: Select a composite image from the composite image set generate ; Get the normal update mask normal Update the mask with the texture texture , and merge and superimpose the two to obtain the texture repair mask inpaint ; The composite image generate Inpainting mask with texture inpaint Input to the image inpainting diffusion model SD inpaint , get the repair texture inpaint ; Based on the repair texture inpaint Render and project the three-dimensional model data to obtain a two-dimensional image; Calculate two-dimensional image and composite image generate similarity and update weight score, and output the current stage texture map based on the updated weight score; Perform multi-stage iterative updates based on repairing the texture map inpaint The texture map obtained after completing the multi-stage iterative update is used to generate a three-dimensional target texture map.

2. The method for generating a three-dimensional target texture based on text according to claim 1, characterized in that: 3D model data and description text data view It is obtained after preprocessing, and the preprocessing includes at least the following steps: Perform spatial position processing on the initial three-dimensional model data to obtain its mesh rendering image in the two-dimensional texture coordinate system; According to multiple preset texture synthesis perspectives, the initial 3D model data is combined with the initial description text data to obtain the 3D model data, and multiple description text data including perspective information are output. view ; The number of stages of multi-stage generative training is the same as the number of preset texture synthesis perspectives.

3. The method for generating a three-dimensional target texture based on text according to claim 1, characterized in that: Select a composite image from the composite image set generate , specifically: In the first stage of multi-stage generative training, a synthetic image is randomly selected from the synthetic image set. generate The subsequent stage is based on the synthetic image selected in the previous stage, and a synthetic image image is selected from the synthetic image set. generate .

4. The method for generating a three-dimensional target texture based on text according to claim 1, characterized in that: Normal update mask normal Obtained through the following steps: Get the current normal map Normal map from the previous stage Based on the current normal map Normal map from the previous stage Calculate the normal update mask normal ; Texture update mask texture Obtained through the following steps: Get the initial texture map texture0 and the i-th iteration stage texture map texture i ; Based on the initial texture map texture0 and the i-th iteration texture map texture i , calculate the texture update mask mask texture .

5. The method for generating a three-dimensional target texture based on text according to claim 1, characterized in that: mask inpaint =mask normal ∪mask texture ,in, Indicates taking the current normal map The z value of the normal vector, To obtain the z value of the normal vector of the normal map in the previous stage, when the difference between the z values ​​corresponding to the two stages is greater than 0.3, the calculated pixel point is the area that needs to be updated; mask texture ={xyz(texture i -texture0)>0.7}, xyz(texture i -texture0) represents the initial texture map texture0 and the texture map texture of the i-th iteration stage i When the pixel distance is greater than 0.7, the corresponding pixel area needs to be updated.

6. The method for generating a three-dimensional target texture based on text according to claim 5, characterized in that: In the first stage of multi-stage generative training, mask normal Blank, mask texture for the entire texture area.

7. The method for generating a three-dimensional target texture based on text according to claim 1, characterized in that: The update weight score is calculated according to the following formula: Among them, sorce represents the update weight score, and its value range is [0, 1], M represents the target plane area, j represents a point in the target plane area, noraml represents regularization, and render(texture inpaint ) represents the rendering of 3D model data, pro(render(texture inpaint )) represents the projection transformation of three-dimensional model data.

8. The method for generating a three-dimensional target texture based on text according to claim 7, characterized in that: The texture map of the current stage is obtained according to the following formula: texture = (1-N)*texture+N*((texture inpaint -texture)*source* r+(1-r)*texture); where texture represents the texture map at the current stage, N represents the area where the target is located in the texture map, and r is the update weight.

9. A system for generating three-dimensional target texture based on text, characterized in that: include: Data acquisition module, used to obtain 3D model data and corresponding description text data view , the description text data text view The viewpoint information is included; A synthetic image set generation module based on 3D model data and description text data view of Perspective information, generate the depth map depth under the corresponding perspective view and the current normal map And describe the text data text view and depth map depth view Input into the depth map diffusion model to obtain a set of synthetic images; The image group difference elimination and screening module is used to select a synthetic image from the synthetic image set. generate ; Texture repair mask generation module, used to obtain normal update mask normal Update the mask with the texture texture , and merge and superimpose the two to obtain the texture repair mask inpaint ; Repair texture map generation module, used to synthesize image generate Inpainting mask with texture inpaint Input to the image inpainting diffusion model SD inpaint , get the repair texture inpaint ; Rendering and projection module, used to repair texture based on texture inpaint Render and project the three-dimensional model data to obtain a two-dimensional image; Update the weight score calculation module to calculate the two-dimensional image and the composite image generate Similarity and update weight score; A current texture map generation module is used to output the current stage texture map based on the updated weight score; 3D target texture map generation module, used to perform texture restoration based on texture inpaint The texture map obtained after completing the multi-stage iterative update is used to generate a three-dimensional target texture map.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for generating a three-dimensional target texture based on text, and when the program for generating a three-dimensional target texture based on text is executed by a processor, the steps of the method for generating a three-dimensional target texture based on text as described in any one of claims 1 to 8 are implemented.

Images