Quality evaluation method and system for text-generated three-dimensional content, medium and terminal

Abstract

The invention provides a quality evaluation method and system for text-generated three-dimensional content, a medium and a terminal, and the method comprises the steps: obtaining a projection video and a text prompt word of the text-generated three-dimensional content; inputting the projection video into a pre-trained shape feature extractor, and determining shape features of the text generation three-dimensional content; inputting the projection video into a pre-trained texture feature extractor, and determining texture features of the text generation three-dimensional content; inputting the projection video and the text cue word into a pre-trained text content consistency feature extractor, and determining a text content consistency feature of the text generation three-dimensional content; and performing feature fusion processing on the shape feature, the texture feature and the text content consistency feature, and determining an evaluation score of the text generation three-dimensional content. Through the method and the device, the overall perception quality of the three-dimensional content generated by the text-guided artificial intelligence is effectively evaluated.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a quality evaluation method, system, medium and terminal for generating three-dimensional content from text. Background Art

[0002] Artificial intelligence generated content (AIGC) uses artificial intelligence technology, especially machine learning and deep learning algorithms, to automatically create content such as text, images, audio, video, and three-dimensional objects. For image and video generated content, there are currently many quality evaluation algorithms to evaluate the quality of generated images and videos; at the same time, there are also a large number of quality evaluation algorithms for traditional three-dimensional content (such as point clouds, meshes). However, there is a lack of quality evaluation algorithms for text-guided generated three-dimensional content. Therefore, how to accurately evaluate the quality of these generated three-dimensional contents has become a task that needs to be urgently solved. Developing a suitable text-guided objective quality evaluation method for generated three-dimensional content can help users screen out three-dimensional content with high consistency with the prompt text, high authenticity, and good perceived quality, saving users time and cost.

[0003] Compared with the quality evaluation task of generating image and video content, the main difference of the quality evaluation task of generating 3D content guided by text is that the content to be evaluated is 3D content, and it is necessary to effectively evaluate the consistency between the text and the generated 3D content, as well as the authenticity and perceptual quality of the generated 3D content. Compared with the quality evaluation task of traditional 3D content, the main difference of the quality evaluation task of generating 3D content guided by text is that the 3D content is generated by text prompt words, and there is no corresponding reference model, so it is necessary to evaluate the authenticity of the 3D content, the consistency between the text and the 3D content, and the perceptual quality of the 3D content without reference; at the same time, the generated 3D content is usually expressed by implicit methods such as implicit neural radiation fields, and there is no explicit expression in a fixed format, so it is necessary to find an efficient and effective quality evaluation method.

[0004] Currently, the existing quality evaluation algorithms for generated image and video content and the quality evaluation algorithms for traditional three-dimensional content cannot effectively evaluate the quality of text-guided generated three-dimensional content. Summary of the invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide a quality evaluation method, system, medium and terminal for generating three-dimensional content from text.

[0006] To achieve the above objective, according to one aspect of the present disclosure, a method for evaluating the quality of three-dimensional content generated by text is provided, comprising:

[0007] Obtaining projection video and text prompt words of text-generated three-dimensional content;

[0008] Inputting the projection video into a pre-trained shape feature extractor to determine shape features of the text to generate three-dimensional content;

[0009] Inputting the projection video into a pre-trained texture feature extractor to determine texture features of the text to generate three-dimensional content;

[0010] Inputting the projection video and the text prompt words into a pre-trained text content consistency feature extractor to determine the text content consistency features of the text generating three-dimensional content;

[0011] The shape features of the three-dimensional content generated by the text, the texture features of the three-dimensional content generated by the text, and the text content consistency features of the three-dimensional content generated by the text are subjected to feature fusion processing to determine an evaluation score of the three-dimensional content generated by the text.

[0012] Optionally, the evaluation score of the text-generated three-dimensional content includes a consistency quality score, a authenticity quality score, and a perception quality score.

[0013] Optionally, the pre-trained shape feature extractor comprises a three-dimensional encoder.

[0014] Optionally, the step of inputting the projection video into a pre-trained shape feature extractor to determine the shape features of the text to generate three-dimensional content includes:

[0015] Performing time-domain downsampling processing on the projected video to determine a downsampled video;

[0016] The downsampled video is input into the 3D encoder to determine the shape features of the text to generate 3D content.

[0017] Optionally, the pre-trained texture feature extractor includes a first two-dimensional encoder and a second two-dimensional encoder.

[0018] Optionally, inputting the projection video into a pre-trained texture feature extractor to determine texture features of the text to generate three-dimensional content includes:

[0019] Determine a front image and a back image of the projection video according to the projection video;

[0020] Inputting the front image into the first two-dimensional encoder to determine the front texture features of the text to generate three-dimensional content;

[0021] Inputting the back side image into the second two-dimensional encoder to determine the back side texture features of the text to generate three-dimensional content;

[0022] The front texture features of the three-dimensional content generated by the text and the back texture features of the three-dimensional content generated by the text are fused to determine the texture features of the three-dimensional content generated by the text.

[0023] Optionally, determining the front image and the back image of the projection video according to the projection video includes:

[0024] Using the first frame image of the projection video as the front image of the projection video;

[0025] The time domain center frame image of the projection video is used as the back image of the projection video.

[0026] Optionally, the pre-trained text content consistency feature extractor includes an image encoder and a text encoder.

[0027] Optionally, the step of inputting the projection video and the text prompt word into a pre-trained text content consistency feature extractor to determine the text content consistency feature of the text generating the three-dimensional content includes:

[0028] Using the first frame image of the projection video as the front image of the projection video;

[0029] Inputting the front image of the projection video into the image encoder to determine the image features of the text generating three-dimensional content;

[0030] Inputting the text prompt word into the text encoder to determine the text features of the text to generate three-dimensional content;

[0031] The image features of the text-generated three-dimensional content and the text features of the text-generated three-dimensional content are fused to determine the text content consistency features of the text-generated three-dimensional content.

[0032] Optionally, the acquiring of text to generate a projection video and text prompt words of three-dimensional content includes:

[0033] According to a preset projection video resolution size, a rendering camera that is arranged around the text to generate three-dimensional content is used to render the text to generate a projection video of the three-dimensional content;

[0034] Obtain preset text prompt words for generating the three-dimensional content of the text.

[0035] According to a second aspect of the present disclosure, a quality evaluation system for generating three-dimensional content from text is provided, comprising:

[0036] An acquisition module, used for acquiring projection video and text prompt words for generating three-dimensional content from text;

[0037] A shape feature extraction module, used to input the projection video into a pre-trained shape feature extractor to determine the shape features of the text to generate three-dimensional content;

[0038] A texture feature extraction module, used to input the projection video into a pre-trained texture feature extractor to determine the texture features of the text to generate three-dimensional content;

[0039] A text content consistency feature extraction module, used for inputting the projection video and the text prompt words into a pre-trained text content consistency feature extractor to determine the text content consistency feature of the text generating three-dimensional content;

[0040] The quality evaluation module is used to perform feature fusion processing on the shape features of the three-dimensional content generated by the text, the texture features of the three-dimensional content generated by the text, and the text content consistency features of the three-dimensional content generated by the text, so as to determine the evaluation score of the three-dimensional content generated by the text.

[0041] According to a third aspect of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method provided in the first aspect of the present disclosure are implemented.

[0042] According to a fourth aspect of the present disclosure, a terminal is provided, including:

[0043] a memory having a computer program stored thereon;

[0044] A processor is used to execute the computer program in the memory to implement the steps of the method provided in the first aspect of the present disclosure.

[0045] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:

[0046] Through the above technical scheme, the projection video is obtained by rendering the three-dimensional content generated by the text, and the text prompt words corresponding to the generated three-dimensional content are obtained. The shape features of the three-dimensional content generated by the text are obtained by a pre-trained shape feature extractor, the texture features of the three-dimensional content generated by the text are obtained by a pre-trained texture feature extractor, and the text content consistency feature extractor is used to obtain the text content consistency of the three-dimensional content generated by the text. Finally, the shape features, texture features and text content consistency features of the three-dimensional content generated by the text are subjected to feature fusion processing to obtain the final evaluation score of the three-dimensional content generated by the text, so as to achieve effective evaluation of the consistency, authenticity and perceived quality of the three-dimensional content generated by the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0048] Figure 1 The present invention is a flowchart of a method for evaluating the quality of three-dimensional content generated by text according to an exemplary embodiment.

[0049] Figure 2 The present invention is a process schematic diagram of a method for evaluating the quality of three-dimensional content generated by text according to an exemplary embodiment.

[0050] Figure 3 The figure is a schematic diagram of a process of extracting shape features by a pre-trained shape feature extractor according to an exemplary embodiment.

[0051] Figure 4 The present invention is a schematic diagram of a process of extracting texture features by a pre-trained texture feature extractor according to an exemplary embodiment.

[0052] Figure 5 It is a flowchart of extracting text content consistency features by a pre-trained text content consistency feature extractor according to an exemplary embodiment.

[0053] Figure 6 The invention is a block diagram of a quality evaluation system for generating three-dimensional content from text according to an exemplary embodiment. DETAILED DESCRIPTION

[0054] The present disclosure is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present disclosure, but are not intended to limit the present disclosure in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present disclosure. These all fall within the scope of protection of the present disclosure.

[0055] Figure 1 The present invention is a flowchart of a method for evaluating the quality of three-dimensional content generated by text according to an exemplary embodiment. Figure 2 The present invention is a process schematic diagram of a method for evaluating the quality of three-dimensional content generated by text according to an exemplary embodiment.

[0056] like Figure 1 , Figure 2 As shown, the present disclosure provides a method for evaluating the quality of three-dimensional content generated by text, including S11 to S15.

[0057] S11, obtaining a projection video and text prompt words for generating three-dimensional content from text.

[0058] The projected video refers to a video rendered by using text to generate three-dimensional content, and the text prompt words refer to preset text content used to generate the three-dimensional content, that is, preset text prompt words.

[0059] S12, inputting the projection video into a pre-trained shape feature extractor to determine the shape features of the text to generate three-dimensional content.

[0060] Among them, the pre-trained shape feature extractor includes a three-dimensional encoder.

[0061] S13, inputting the projection video into a pre-trained texture feature extractor to determine texture features of the text to generate three-dimensional content.

[0062] Among them, the pre-trained texture feature extractor includes a first two-dimensional encoder, a second two-dimensional encoder, a fully connected layer 1, a fully connected layer 2, and an activation function RELU1.

[0063] S14, inputting the projection video and the text prompt words into a pre-trained text content consistency feature extractor to determine the text content consistency features of the text to generate the three-dimensional content.

[0064] Among them, the pre-trained text content consistency feature extractor is a multimodal pre-trained neural network, which includes an image encoder, a text encoder, a fully connected layer 1, a fully connected layer 2, and an activation function RELU1.

[0065] S15, performing feature fusion processing on the shape features of the text-generated three-dimensional content, the texture features of the text-generated three-dimensional content, and the text content consistency features of the text-generated three-dimensional content to determine an evaluation score of the text-generated three-dimensional content.

[0066] Among them, the evaluation scores of text-generated three-dimensional content include consistency quality score, authenticity quality score and perception quality score.

[0067] The shape features, texture features and text content consistency features of the three-dimensional content generated by the text of the present invention are fused through fully connected layer 1, activation function RELU1, fully connected layer 2, activation function RELU2 and fully connected layer 3 to generate the consistency quality score, authenticity quality score and perception quality score of the three-dimensional content of the text.

[0068] Through the above technical scheme, the projection video is obtained by rendering the three-dimensional content generated by the text, and the text prompt words corresponding to the generated three-dimensional content are obtained. The shape features of the three-dimensional content generated by the text are obtained by a pre-trained shape feature extractor, the texture features of the three-dimensional content generated by the text are obtained by a pre-trained texture feature extractor, and the text content consistency feature extractor is used to obtain the text content consistency of the three-dimensional content generated by the text. Finally, the shape features, texture features and text content consistency features of the three-dimensional content generated by the text are subjected to feature fusion processing to obtain the final evaluation score of the three-dimensional content generated by the text, so as to achieve effective evaluation of the consistency, authenticity and perceived quality of the three-dimensional content generated by the text.

[0069] In a possible embodiment, S11 may include S21 to S22.

[0070] S21, according to a preset projection video resolution size, using a rendering camera that is surrounded by a text to generate a three-dimensional content, rendering the text to generate a projection video of the three-dimensional content.

[0071] The trajectory of the rendering camera is set around the text to generate three-dimensional content, and the rendering camera is used to render the text to generate a projection video corresponding to the three-dimensional content.

[0072] The preset projection video resolution size can be set according to the input requirements of the subsequent shape feature extractor, texture feature extractor, and text content consistency feature extractor.

[0073] In the present disclosure, the total number of frames of the rendered projection video is 120 frames to reduce the amount of calculation.

[0074] S22, obtaining preset text prompt words for generating the three-dimensional content from the text.

[0075] Among them, in the present disclosure, the preset text prompt words can be directly obtained.

[0076] Figure 3 The figure is a schematic diagram of a process of extracting shape features by a pre-trained shape feature extractor according to an exemplary embodiment.

[0077] like Figure 3 As shown, in a possible embodiment, S12 may include S31 to S32.

[0078] S31, performing time domain downsampling processing on the projected video to determine a downsampled video.

[0079] The time domain down-sampling process may adopt a uniform sampling method, and image frames may be selected at intervals in the projected video according to a preset time sequence to form a down-sampled video.

[0080] In the present disclosure, the total number of frames of the downsampled video is 12 frames to reduce the amount of calculation.

[0081] S32, inputting the downsampled video into a 3D encoder to determine shape features of the text to generate 3D content.

[0082] In the present disclosure, the 3D encoder may use the Swin3D-S network pre-trained on the KINETICS400 dataset, and use bilinear interpolation to downsample the input downsampled video to a resolution of 224×244.

[0083] Thus, the pre-trained shape feature extractor outputs the shape features of the text-generated three-dimensional content.

[0084] Figure 4 The present invention is a schematic diagram of a process of extracting texture features by a pre-trained texture feature extractor according to an exemplary embodiment.

[0085] like Figure 4 As shown, in a possible embodiment, S13 may include S41 to S44.

[0086] S41, determining a front image and a back image of the projection video according to the projection video.

[0087] As an example, the first frame image of the projection example is used as the front image of the projection video.

[0088] As another example, the time-domain center frame image of the projection video is used as the back image of the projection video.

[0089] S42, inputting the front image into a first two-dimensional encoder to determine the front texture features of the text to generate three-dimensional content.

[0090] The first two-dimensional encoder is a pre-trained image neural network, and the first two-dimensional encoder is used to extract front texture features from the front image.

[0091] S43, inputting the back side image into a second two-dimensional encoder to determine the back side texture features of the text to generate three-dimensional content.

[0092] The second two-dimensional encoder is a pre-trained image neural network, and the second two-dimensional encoder is used to extract back texture features from the back image.

[0093] In the present disclosure, both the first two-dimensional encoder and the second two-dimensional encoder use the Swin-S network pre-trained on the IMAGENET1K dataset, and use bilinear interpolation to downsample the input front image and back image to a resolution of 224×244.

[0094] S44, fusing the front texture features of the three-dimensional content generated by the text and the back texture features of the three-dimensional content generated by the text to determine the texture features of the three-dimensional content generated by the text.

[0095] The fully connected layer 1, activation function RELU1 and fully connected layer 2 are used to fuse the front texture features and the back texture features, and the pre-trained texture feature extractor outputs the texture features of the three-dimensional content of the text.

[0096] Figure 5 It is a flowchart of extracting text content consistency features by a pre-trained text content consistency feature extractor according to an exemplary embodiment.

[0097] like Figure 5As shown, in a possible embodiment, S14 may include S51 to S55.

[0098] S51: Use the first frame image of the projection video as the front image of the projection video.

[0099] S52, inputting the front image of the projection video into an image encoder to determine image features of the text to generate three-dimensional content.

[0100] Among them, an image encoder is used to extract the image features of the projected video to extract text from the front image to generate three-dimensional content.

[0101] S53, inputting the text prompt words into a text encoder to determine text features for generating three-dimensional content from the text.

[0102] Among them, a text encoder is used to extract text features of three-dimensional content from text prompt words.

[0103] In the present disclosure, a text content consistency feature extractor adopts a CLIP multimodal neural network of weakly supervised image-text pair data, wherein the image encoder adopts a VisionTransformer architecture, the text encoder adopts a Transformer architecture, and a bilinear interpolation method is used to downsample the image input to the image encoder to a resolution of 224×244.

[0104] S54, fusing the image features of the text-generated three-dimensional content and the text features of the text-generated three-dimensional content to determine the text content consistency features of the text-generated three-dimensional content.

[0105] The fully connected layer 1, the activation function RELU1 and the fully connected layer 2 are used to fuse the image features and text features of the text-generated three-dimensional content, and the pre-trained text content consistency feature extractor outputs the text content consistency features of the text-generated three-dimensional content.

[0106] In a possible embodiment, S15, the shape features of the three-dimensional content generated by the text, the texture features of the three-dimensional content generated by the text, and the text content consistency features of the three-dimensional content generated by the text are subjected to feature fusion processing to determine the evaluation score of the three-dimensional content generated by the text, which may include: splicing the shape features, texture features and text content consistency features of the three-dimensional content generated by the text, and inputting the spliced ​​features into a neural network composed of a fully connected layer 1, an activation function RELU1, a fully connected layer 2, an activation function RELU2 and a fully connected layer 3, and outputting the consistency quality score, authenticity quality score and perception quality score of the three-dimensional content generated by the text.

[0107] In a possible embodiment, the AIGC-T23DCQA Database is used to verify the effectiveness of a quality evaluation method for generating three-dimensional content from text provided in the present disclosure.

[0108] Among them, the AIGC-T23DCQADatabase includes 969 verified 3D contents, which are generated from 170 text prompt words through 6 popular text-to-3D content generation models. These 3D contents are given corresponding subjective quality ratings from the perspectives of perceived quality, authenticity and text content consistency.

[0109] The verification test used evaluation criteria including Spearman's Rank-Order Correlation Coefficient (SRCC), Kendall's Rank-Order Correlation Coefficient (KRCC) and Pearson linear correlation coefficient (PLCC) for authenticity, text content consistency and perceived quality.

[0110] The AIGC-T23DCQA database is used to verify traditional image quality evaluation algorithms NIQE, ILNIQE, BRISQUE, Resnet-18, Resnet-34, Swin-T, CNNIQA, Swin3D-S, CLIPScore and the quality evaluation method for generating three-dimensional content from text provided in the present invention, namely ProposedMethod.

[0111] For the traditional image quality assessment algorithm, the average score of all frames of the projected video is calculated as the final result. For the remaining quality assessment algorithms that need to be trained, the AIGC-T23DCQA database is divided into training and test sets in a ratio of 4:1. In addition, the dataset is randomly split 10 times and the results are averaged to ensure unbiased performance comparison.

[0112] The performance test results are shown in the following table:

[0113]

[0114] Table 1

[0115] It can be seen from Table 1 that the quality evaluation method for generating three-dimensional content from text proposed in the present disclosure can effectively evaluate the text content consistency, authenticity and perceived quality of the three-dimensional content generated from text.

[0116] Figure 6The invention is a block diagram of a quality evaluation system for generating three-dimensional content from text according to an exemplary embodiment.

[0117] Based on the same concept, the present disclosure also provides a quality assessment system 100 for generating three-dimensional content from text, including: an acquisition module 110 , a shape feature extraction module 120 , a texture feature extraction module 130 , a text content consistency feature extraction module 140 , and a quality assessment module 150 .

[0118] An acquisition module 110 is used to acquire projection video and text prompt words for generating three-dimensional content from text;

[0119] A shape feature extraction module 120, configured to input the projection video into a pre-trained shape feature extractor to determine shape features of the text to generate three-dimensional content;

[0120] The texture feature extraction module 130 is used to input the projection video into a pre-trained texture feature extractor to determine the texture features of the text to generate three-dimensional content;

[0121] A text content consistency feature extraction module 140 is used to input the projection video and the text prompt words into a pre-trained text content consistency feature extractor to determine the text content consistency feature of the text generating the three-dimensional content;

[0122] The quality evaluation module 150 is used to perform feature fusion processing on the shape features of the text-generated three-dimensional content, the texture features of the text-generated three-dimensional content, and the text content consistency features of the text-generated three-dimensional content to determine the evaluation score of the text-generated three-dimensional content.

[0123] Through the above technical scheme, the projection video is obtained by rendering the three-dimensional content generated by the text, and the text prompt words corresponding to the generated three-dimensional content are obtained. The shape features of the three-dimensional content generated by the text are obtained by a pre-trained shape feature extractor, the texture features of the three-dimensional content generated by the text are obtained by a pre-trained texture feature extractor, and the text content consistency feature extractor is used to obtain the text content consistency of the three-dimensional content generated by the text. Finally, the shape features, texture features and text content consistency features of the three-dimensional content generated by the text are subjected to feature fusion processing to obtain the final evaluation score of the three-dimensional content generated by the text, so as to achieve effective evaluation of the consistency, authenticity and perceived quality of the three-dimensional content generated by the text.

[0124] Regarding the embodiment of the above system, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0125] Based on the same concept as above, in another embodiment of the present disclosure, a terminal is also provided, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor executes the program to execute a quality evaluation method for generating three-dimensional content from text.

[0126] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0127] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0128] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0129] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0130] In an embodiment of the present disclosure, a non-temporary computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a quality evaluation method for generating three-dimensional content from text in any of the above embodiments are implemented.

[0131] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present disclosure.

[0136] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A quality evaluation method for three-dimensional content generated by text, characterized in that: include: Obtaining projection video and text prompt words of text-generated three-dimensional content; Inputting the projection video into a pre-trained shape feature extractor to determine shape features of the text to generate three-dimensional content; Inputting the projection video into a pre-trained texture feature extractor to determine texture features of the text to generate three-dimensional content; Inputting the projection video and the text prompt words into a pre-trained text content consistency feature extractor to determine the text content consistency features of the text generating three-dimensional content; The shape features of the three-dimensional content generated by the text, the texture features of the three-dimensional content generated by the text, and the text content consistency features of the three-dimensional content generated by the text are subjected to feature fusion processing to determine an evaluation score of the three-dimensional content generated by the text.

2. The method according to claim 1, characterized in that: The evaluation score of the text-generated three-dimensional content includes a consistency quality score, a authenticity quality score, and a perception quality score.

3. The method according to claim 1, characterized in that The pre-trained shape feature extractor includes a three-dimensional encoder; The step of inputting the projection video into a pre-trained shape feature extractor to determine the shape features of the text to generate three-dimensional content includes: Performing time-domain downsampling processing on the projected video to determine a downsampled video; The downsampled video is input into the 3D encoder to determine the shape features of the text to generate 3D content.

4. The method according to claim 1, characterized in that: The pre-trained texture feature extractor includes a first two-dimensional encoder and a second two-dimensional encoder; The step of inputting the projection video into a pre-trained texture feature extractor to determine the texture features of the text generating three-dimensional content includes: Determine a front image and a back image of the projection video according to the projection video; Inputting the front image into the first two-dimensional encoder to determine the front texture features of the text to generate three-dimensional content; Inputting the back side image into the second two-dimensional encoder to determine the back side texture features of the text to generate three-dimensional content; The front texture features of the three-dimensional content generated by the text and the back texture features of the three-dimensional content generated by the text are fused to determine the texture features of the three-dimensional content generated by the text.

5. The method according to claim 4, characterized in that The determining, according to the projection video, a front image and a back image of the projection video comprises: Using the first frame image of the projection video as the front image of the projection video; The time domain center frame image of the projection video is used as the back image of the projection video.

6. The method according to claim 1, characterized in that The pre-trained text content consistency feature extractor includes an image encoder and a text encoder; The step of inputting the projection video and the text prompt word into a pre-trained text content consistency feature extractor to determine the text content consistency feature of the text generating the three-dimensional content includes: Using the first frame image of the projection video as the front image of the projection video; Inputting the front image of the projection video into the image encoder to determine the image features of the text generating three-dimensional content; Inputting the text prompt word into the text encoder to determine the text features of the text to generate three-dimensional content; The image features of the text-generated three-dimensional content and the text features of the text-generated three-dimensional content are fused to determine the text content consistency features of the text-generated three-dimensional content.

7. The method according to claim 1, characterized in that The method of acquiring text to generate projection video and text prompt words of three-dimensional content includes: According to a preset projection video resolution size, a rendering camera that is arranged around the text to generate three-dimensional content is used to render the text to generate a projection video of the three-dimensional content; Obtain preset text prompt words for generating the three-dimensional content of the text.

8. A quality evaluation system for generating three-dimensional content from text, characterized in that: include: An acquisition module, used for acquiring projection video and text prompt words for generating three-dimensional content from text; A shape feature extraction module, used to input the projection video into a pre-trained shape feature extractor to determine the shape features of the text to generate three-dimensional content; A texture feature extraction module, used to input the projection video into a pre-trained texture feature extractor to determine the texture features of the text to generate three-dimensional content; A text content consistency feature extraction module, used for inputting the projection video and the text prompt words into a pre-trained text content consistency feature extractor to determine the text content consistency feature of the text generating the three-dimensional content; The quality evaluation module is used to perform feature fusion processing on the shape features of the three-dimensional content generated by the text, the texture features of the three-dimensional content generated by the text, and the text content consistency features of the three-dimensional content generated by the text, so as to determine the evaluation score of the three-dimensional content generated by the text.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A terminal, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

Images