Virtual character efficient generation system based on multi-modal large model

By combining the multimodal large model system with GPT-4 and Stable Diffusion models, the virtual role generation is automated, and the problems of inefficiency, insufficient personalization and consistency in traditional methods are solved, and high-quality and personalized character images are generated. , reducing labor dependence and cost.

CN118987612AInactive Publication Date: 2024-11-22TUOYUAN (GUANGZHOU) SMART TECH CO LTD
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411068050.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional virtual character generation methods are inefficient, personalized and creative, and the quality and consistency of generated images are difficult to guarantee, and rely on manual intervention to meet large-scale and personalized needs.

Method used

Using a system based on multimodal large model, combined with GPT-4 and Stable Diffusion models, high-quality images are generated from role descriptions through an automated processing module, providing user feedback optimization process, and realizing a fully automated process from user input to image generation.

Benefits of technology

The generation efficiency is significantly improved, the generated character image is highly personalized and creative, and the image quality and consistency are improved, manual intervention is reduced, generation costs are reduced, and large-scale and personalized needs are met.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118987612A_ABST
    Figure CN118987612A_ABST
Patent Text Reader

Abstract

The invention discloses an efficient virtual role generation system based on a multi-modal large model. The system comprises the following steps that S1, a user inputs preliminary description of a role through an interactive interface; s2, the preliminary description input by the user is analyzed through GTP-4, detailed role description is generated, and the role description comprises appearance features, costume styles, postures, expressions and background scenes of roles; s3, analyzing, optimizing and combining the role description generated by the GPT-4 through an automatic processing module, and converting the role description into an input format of a Stable Diffusion model; s4, generating a role image on the basis of a Stable Diffusion model; s5, providing a user evaluation feedback interface, and commenting and feeding back the generated role image by the user through the evaluation feedback interface; and S6, through a feedback optimization system, analyzing comments and feedback of the user, and adjusting and improving a generation process. According to the method, the GPT-4 model and the Stable Diffusion model are combined, and the automatic processing module is designed, so that the high-quality personalized role image is generated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of large models and automation technology, and in particular to a virtual character efficient generation system based on a multi-modal large model. Background Art

[0002] Virtual character generation technology has been widely used in many fields such as games, film and television, advertising, and social media. Traditional virtual character generation methods mainly rely on manual production by designers, which is not only time-consuming and labor-intensive, but also difficult to guarantee the quality and consistency of the generated character images. In addition, with the increasing demand for personalized and diversified virtual characters from users, traditional template-based generation methods have been unable to meet market demand and have many defects and deficiencies:

[0003] 1. Low generation efficiency: Handmade and templated generation methods require a lot of manual intervention and operation, with a long generation cycle and low efficiency, and it is difficult to meet the needs of large-scale and rapid generation. Even for machine learning-based generation methods, the generation efficiency is still low due to the complex and time-consuming training process.

[0004] 2. Limited personalization and creativity: Traditional template-based generation methods and rule-based generation methods lack personalization and creativity. The generated character images are often stereotyped and difficult to meet the diverse and personalized needs of users. Even for machine learning-based methods, the generated character images still need to be improved in terms of personalization and creativity due to the limitations of the diversity and quality of training data.

[0005] 3. Inconsistent quality of generated images: The quality and consistency of character images generated by existing methods are difficult to guarantee, especially when dealing with complex character descriptions and diverse user needs. The generated character images often have problems such as unstable quality and insufficient detail processing. This affects user experience and satisfaction to a certain extent.

[0006] 4. High dependence on manual labor: Manual production and template-based generation methods rely heavily on the skill level of designers and require a lot of manual intervention. This not only increases the generation cost, but also limits the improvement of generation efficiency.

[0007] In recent years, with the development of deep learning technology, models such as generative adversarial networks (GANs) and variational autoencoders (VAEs) have made significant progress in image generation. These models are able to learn from a large amount of data and generate high-quality images. However, these methods usually require a large amount of labeled data and computing resources, and may still have limitations when generating images of a specific type or style.

[0008] Recently, solutions that combine NLP and image generation techniques have begun to emerge. For example, some studies have shown the possibility of using text descriptions to guide image generation. These systems usually use language models to parse text descriptions and convert them into instructions or features that can be understood by image generation models. Then, image generation models (such as GANs or specific deep learning models) generate corresponding images based on these instructions.

[0009] Although these schemes have achieved the function of generating images using natural language descriptions to some extent, they are usually limited in understanding complex descriptions (such as detailed character backgrounds, complex emotional expressions, and cultural elements) and generating high-quality, highly customized images. In addition, the output of these systems usually requires further human intervention and editing to meet specific quality standards or creative purposes.

[0010] Therefore, how to provide an efficient virtual character generation system based on a multimodal large model is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0011] One purpose of the present invention is to propose an efficient virtual character generation system based on a multimodal large model. The present invention combines GPT-4 and Stable Diffusion models and designs a unique automatic processing module to achieve full process automation from character description to high-quality image generation, thereby meeting the high requirements for personalization, creativity, generation efficiency and consistency in the virtual character generation process.

[0012] According to an embodiment of the present invention, a system for efficiently generating a virtual character based on a multimodal large model comprises the following steps:

[0013] S1, the user inputs a preliminary description of the role through the interactive interface;

[0014] S2, using GTP-4 to analyze the preliminary description input by the user and generate a detailed role description, wherein the role description includes the role's appearance, clothing style, posture, expression and background scene;

[0015] S3. Analyze, optimize and combine the role description generated by GPT-4 through an automated processing module and convert it into an input format of a Stable Diffusion model;

[0016] S4, Generate character images based on the Stable Diffusion model to ensure the high quality and consistency of the generated images;

[0017] S5, providing a user evaluation and feedback interface, through which the user comments and provides feedback on the generated character image, so as to further optimize the generation process;

[0018] S6. Through the feedback optimization system, the user's comments and feedback are analyzed, and the generation process is adjusted and improved to improve the personalization and quality of the character image.

[0019] Optionally, the S2 specifically includes:

[0020] S21, receiving a preliminary description of the role input by the user through an interactive interface to form a preliminary data set D0;

[0021] S22. Use GPT-4 to perform preliminary text processing on the preliminary description input by the user, extract key elements and feature information in the description, analyze it through natural language processing technology, and generate preliminary structured data D1;

[0022] S23, based on the preliminary structured data D1, a detailed description of the appearance characteristics of the character is performed to generate detailed appearance characteristic data D2;

[0023] S24, describing the character's clothing style in detail, and generating detailed clothing style data D3;

[0024] S25, describing the character's posture in detail to generate detailed posture data D4;

[0025] S26, describing the character's expression in detail, generating detailed expression data D5;

[0026] S27, describing the background scene of the character in detail, and generating detailed background scene data D6;

[0027] S28. Integrate and optimize the generated role descriptions, ensure the coherence and consistency of the descriptions through semantic analysis and logic verification, eliminate potential conflicts and unreasonable aspects, and generate a complete role description document D7:

[0028] D7=m integration (D2,D3,D4,D5,D6);

[0029] S29, converting the complete role description document D7 into a structured data structure D8, so as to facilitate further processing and analysis by the automated processing module;

[0030] S210, passing the generated detailed role description document D8 to the automated processing module, preparing for subsequent analysis, optimization and combination steps, ensuring the accuracy and completeness of the role description, and providing high-quality input data for the image generation process.

[0031] Optionally, the automated processing module of S3 specifically includes:

[0032] S31. Simplify the grammar of the role description generated by GTP-4, and convert complex sentences and long paragraphs into more concise and clear descriptions;

[0033] S32, extracting key information from the character description, including appearance features, clothing style, posture, expression and background scene;

[0034] S33, through format optimization, convert the extracted key information into the input format accepted by the Stable Diffusion model;

[0035] S34, optimizing and combining the converted descriptions to generate multiple groups of description combination prompts, ensuring that the relationships between the various description elements are coordinated and consistent to avoid information conflicts or contradictions;

[0036] S35, sorting the plurality of description combination prompts, and preferentially selecting the combination that best matches the user's preliminary description, so as to improve the accuracy of the generated image and the user's satisfaction;

[0037] S36. Input the finalized description combination prompts into the Stable Diffusion model.

[0038] Optionally, the format optimization specifically includes: converting the description of the appearance features into specific visual elements, standardizing the description of the clothing style, converting the description of the posture into specific action instructions, converting the description of the expression into an emotion label, and converting the description of the background scene into environment setting parameters.

[0039] Optionally, the S4 specifically includes:

[0040] S41. In the Stable Diffusion model, initialize the random noise vector z, z~N(0,I), where N(0,I) represents a multivariate normal distribution with a mean of 0 and a covariance of the identity matrix I;

[0041] S42, using a diffusion process to iterate the random noise vector z for multiple times, and in each iteration, gradually removing the noise through a reverse diffusion process, so that the noise vector gradually approaches the high-quality image corresponding to the character description;

[0042] S43, in each iteration, conditionally constraining the noise vector using a conditional generative network to ensure that the generated image conforms to the appearance features, clothing style, posture, expression, and background scene in the character description;

[0043] S44, calculating the matching degree F between the generated image and the character description, wherein the matching degree F is obtained by calculating the feature vector similarity between the generated image and the character description:

[0044] F = cos(θ);

[0045] Among them, θ represents the angle between the generated image and the character description feature vector;

[0046] S45, optimizing and adjusting the generated image according to the matching degree F, and continuously adjusting the noise vector z through a gradient descent algorithm so that the similarity between the feature vector of the generated image and the feature vector of the character description gradually increases, and finally generating a high-quality character image;

[0047] S46. During the generation process, the attention mechanism is used to emphasize the key details in the character description;

[0048] S47. Output a character image with high resolution and rich detail information.

[0049] The beneficial effects of the present invention are:

[0050] (1) Significantly improved generation efficiency: By fully automating the character description generation and image generation process, this system realizes a fully automated process from user input to final image generation. Compared with traditional manual production and template generation methods, this system significantly reduces manual intervention, improves generation efficiency, and can quickly respond to large-scale, personalized character generation needs.

[0051] (2) High degree of personalization and creativity: Using GPT-4's powerful text understanding and generation capabilities, the system can generate detailed and creative character settings from the initial description input by the user, including the character's appearance, clothing style, posture, expression, and background scene. This capability has significant advantages in processing complex emotional tendencies, cultural backgrounds, and scene descriptions, making the generated character image highly personalized and unique, far exceeding the traditional template method.

[0052] (3) High-quality and consistent image generation: Through the Stable Diffusion model, the system is able to generate high-resolution, detailed, and visually consistent character images. The optimization algorithm and balance adjustment steps in the generation process ensure that the various elements of the image are coordinated and consistent, and the generated image is highly consistent with the requirements of the character description visually. In addition, the use of the attention mechanism to emphasize key details makes the character images reach high-quality standards in terms of facial expressions, clothing textures, and posture details.

[0053] (4) Continuous optimization driven by user feedback: The system provides a user evaluation feedback interface through which users can comment and provide feedback on the generated character images. Through the feedback optimization system, the system can analyze user feedback information and adjust and improve the generation process accordingly. This continuous optimization mechanism not only improves the efficiency and effectiveness of the generation process, but also further enhances the personalization of the character images and user satisfaction.

[0054] (5) Reduce manual dependence and reduce generation costs: Traditional virtual character generation methods require the participation of a large number of designers, which is costly and inefficient. This system greatly reduces the need for manual intervention and generation costs through comprehensive automation, while improving generation efficiency, making the virtual character generation process more efficient and economical. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0056] Figure 1 This is a schematic diagram of the structure of a virtual character efficient generation system based on a multi-modal large model proposed by the present invention;

[0057] Figure 2 A flowchart of a virtual character efficient generation system based on a multi-modal large model proposed by the present invention;

[0058] Figure 3 This is a detailed workflow diagram of the automated processing module of an efficient virtual character generation system based on a multimodal large model proposed by the present invention. DETAILED DESCRIPTION

[0059] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0060] refer to Figure 1-3 , an efficient virtual character generation system based on a multimodal large model, comprising the following steps:

[0061] S1, the user inputs a preliminary description of the role through the interactive interface;

[0062] S2, using GTP-4 to analyze the preliminary description input by the user and generate a detailed role description, wherein the role description includes the role's appearance, clothing style, posture, expression and background scene;

[0063] S3. Analyze, optimize and combine the role description generated by GPT-4 through an automated processing module and convert it into an input format of a Stable Diffusion model;

[0064] S4, Generate character images based on the Stable Diffusion model to ensure the high quality and consistency of the generated images;

[0065] S5, providing a user evaluation and feedback interface, through which the user comments and provides feedback on the generated character image, so as to further optimize the generation process;

[0066] S6. Through the feedback optimization system, the user's comments and feedback are analyzed, and the generation process is adjusted and improved to improve the personalization and quality of the character image.

[0067] In this implementation, S2 specifically includes:

[0068] S21, receiving a preliminary description of the role input by the user through an interactive interface to form a preliminary data set D0;

[0069] S22. Use GPT-4 to perform preliminary text processing on the preliminary description input by the user, extract key elements and feature information in the description, analyze it through natural language processing technology, and generate preliminary structured data D1;

[0070] S23, based on the preliminary structured data D1, a detailed description of the appearance characteristics of the character is performed to generate detailed appearance characteristic data D2;

[0071] S24, describing the character's clothing style in detail, and generating detailed clothing style data D3;

[0072] S25, describing the character's posture in detail to generate detailed posture data D4;

[0073] S26, describing the character's expression in detail, generating detailed expression data D5;

[0074] S27, describing the background scene of the character in detail, and generating detailed background scene data D6;

[0075] S28. Integrate and optimize the generated role descriptions, ensure the coherence and consistency of the descriptions through semantic analysis and logic verification, eliminate potential conflicts and unreasonable aspects, and generate a complete role description document D7:

[0076] D7=m integration (D2,D3,D4,D5,D6);

[0077] S29, converting the complete role description document D7 into a structured data structure D8, so as to facilitate further processing and analysis by the automated processing module;

[0078] S210, passing the generated detailed role description document D8 to the automated processing module, preparing for subsequent analysis, optimization and combination steps, ensuring the accuracy and completeness of the role description, and providing high-quality input data for the image generation process.

[0079] In this implementation, the automated processing module of S3 specifically includes:

[0080] S31. Simplify the grammar of the role description generated by GTP-4, and convert complex sentences and long paragraphs into more concise and clear descriptions;

[0081] S32, extracting key information from the character description, including appearance features, clothing style, posture, expression and background scene;

[0082] S33, through format optimization, convert the extracted key information into the input format accepted by the Stable Diffusion model;

[0083] S34, optimizing and combining the converted descriptions to generate multiple groups of description combination prompts, ensuring that the relationships between the various description elements are coordinated and consistent to avoid information conflicts or contradictions;

[0084] S35, sorting the plurality of description combination prompts, and preferentially selecting the combination that best matches the user's preliminary description, so as to improve the accuracy of the generated image and the user's satisfaction;

[0085] S36. Input the finalized description combination prompts into the Stable Diffusion model.

[0086] In this embodiment, the format optimization specifically includes: converting the description of the appearance features into specific visual elements, standardizing the description of the clothing style, converting the description of the posture into specific action instructions, converting the description of the expression into an emotional label, and converting the description of the background scene into environmental setting parameters.

[0087] In this implementation manner, the S4 specifically includes:

[0088] S41. In the Stable Diffusion model, initialize the random noise vector z, z~N(0,I), where N(0,I) represents a multivariate normal distribution with a mean of 0 and a covariance of the identity matrix I;

[0089] S42, using a diffusion process to iterate the random noise vector z for multiple times, and in each iteration, gradually removing the noise through a reverse diffusion process, so that the noise vector gradually approaches the high-quality image corresponding to the character description;

[0090] S43, in each iteration, conditionally constraining the noise vector using a conditional generative network to ensure that the generated image conforms to the appearance features, clothing style, posture, expression, and background scene in the character description;

[0091] S44, calculating the matching degree F between the generated image and the character description, wherein the matching degree F is obtained by calculating the feature vector similarity between the generated image and the character description:

[0092] F = cos(θ);

[0093] Among them, θ represents the angle between the generated image and the character description feature vector;

[0094] S45, optimizing and adjusting the generated image according to the matching degree F, and continuously adjusting the noise vector z through a gradient descent algorithm so that the similarity between the feature vector of the generated image and the feature vector of the character description gradually increases, and finally generating a high-quality character image;

[0095] S46. During the generation process, the attention mechanism is used to emphasize the key details in the character description;

[0096] S47. Output a character image with high resolution and rich detail information.

[0097] Embodiment 1:

[0098] In order to verify the feasibility of the present invention in implementation, the present invention is applied to the design team of a large game development company. The design team is faced with the challenge of efficiently and accurately generating a large number of virtual characters with different characteristics and backgrounds. The traditional method requires designers to manually draw the appearance, clothing, posture and background of each character, which is not only time-consuming and labor-intensive, but also difficult to guarantee the quality and consistency of the generated image. In addition, designers also need to communicate with game planners repeatedly to ensure that the character design conforms to the game plot and background settings, further increasing the complexity and time cost of character generation. In order to improve work efficiency, reduce labor costs, and ensure the high quality and consistency of character images, the company decided to apply the virtual character efficient generation system based on multimodal large models of the present invention.

[0099] First, the design team input a brief character description through the user interface. For example, the input description is "a warrior in red armor, holding a big sword, smiling, standing on a sunny mountain top." The system uses GPT-4 to analyze the input description and generate a detailed character description, including the character's appearance, clothing style, posture, expression and background scene.

[0100] Next, the system analyzes, optimizes, and combines the detailed descriptions generated by GPT-4 through an automated processing module and converts them into the input format of the Stable Diffusion model. The automated processing module includes language simplification, key information extraction, and format optimization steps to ensure the accuracy and consistency of the descriptions.

[0101] In the image generation module, the system generates character images based on the Stable Diffusion model. First, the random noise vector z is initialized, and the image is gradually generated through multiple iterations of the back-diffusion process. At the same time, the system applies conditional generation network constraints and attention mechanisms to ensure that the generated image has a high degree of match with the description and the accuracy of details.

[0102] The generated character images are then displayed on the user evaluation feedback interface, where the design team can comment and provide feedback on the images. The system analyzes user comments and feedback through the feedback optimization system, adjusts and improves the generation process. The high-quality character images generated are successfully used in game development.

[0103] In order to verify the effectiveness of the present invention, we conducted multiple tests and data analysis. The following is a specific data table:

[0104] Table 1 Data comparison between virtual character efficient generation system and traditional method

[0105] index Traditional methods System of the present invention Average generation time (hours) 8 0.5 Image quality rating (average score) 7.5 9.2 Feedback processing time (hours) 168 24 Number of designers 5 2 Number of planning staff 2 1 Design cycle (month) 3 1 Reduced labor costs - 70% Creativity and details rating (out of 10) 8.0 9.5

[0106] According to the data in the above table, under the traditional method, the average time for designers to design each character is about 8 hours. After applying the system of the present invention, the time to generate a character image is shortened to 30 minutes on average, and the efficiency is improved by 16 times. The design team evaluated the quality of the 100 character images generated, and the scoring criteria were appearance consistency, detail accuracy and overall visual effect. The average score of the traditional method was 7.5 points (out of 10 points), and the average score after applying the system of the present invention was increased to 9.2 points. During the design process, designers and planners provided a total of 200 feedback opinions. The system processed all feedback within 24 hours and optimized the image accordingly, while the traditional method required designers and planners to communicate and modify repeatedly, which took an average of one week. Under the traditional method, a project needs to be equipped with 5 designers and 2 planners, and the entire design cycle is 3 months. After applying the system of the present invention, only 2 designers and 1 planner are required, the design cycle is shortened to 1 month, and the labor cost is reduced by 70%. When generating character images, the system can flexibly handle different character descriptions and background settings to generate diverse image results. The design team conducted a creative evaluation of the character images generated by the system and believed that the images generated by the system were superior to traditional methods in terms of creativity and detail, with a score of 9.5 (out of 10).

[0107] In summary, the multimodal large model-based efficient virtual character generation system of the present invention realizes the automation of the whole process from character description to high-quality image generation by combining advanced natural language processing technology and image generation technology, significantly improves generation efficiency, reduces labor costs, ensures image quality and consistency, and meets the high requirements for personalization and creativity in the virtual character generation process. The system has demonstrated excellent performance and significant beneficial effects in practical applications.

[0108] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An efficient virtual character generation system based on a multimodal large model, characterized in that: The steps include: S1, the user inputs a preliminary description of the role through the interactive interface; S2, using GTP-4 to analyze the preliminary description input by the user and generate a detailed role description, wherein the role description includes the role's appearance, clothing style, posture, expression and background scene; S3. Analyze, optimize and combine the role description generated by GPT-4 through an automated processing module and convert it into an input format of a Stable Diffusion model; S4, generating character images based on the Stable Diffusion model; S5, providing a user evaluation and feedback interface, through which the user comments and provides feedback on the generated character image; S6. Through the feedback optimization system, analyze user comments and feedback, adjust and improve the generation process.

2. The highly efficient virtual character generation system based on a multimodal large model according to claim 1 is characterized in that: The S2 specifically includes: S21, receiving a preliminary description of the role input by the user through an interactive interface to form a preliminary data set D0; S22. Use GPT-4 to perform preliminary text processing on the preliminary description input by the user, extract key elements and feature information in the description, analyze it through natural language processing technology, and generate preliminary structured data D1; S23, based on the preliminary structured data D1, a detailed description of the appearance characteristics of the character is performed to generate detailed appearance characteristic data D2; S24, describing the character's clothing style in detail, and generating detailed clothing style data D3; S25, describing the character's posture in detail to generate detailed posture data D4; S26, describing the character's expression in detail, generating detailed expression data D5; S27, describing the background scene of the character in detail, and generating detailed background scene data D6; S28. Integrate and optimize the generated role descriptions, and generate a complete role description document D7 through semantic analysis and logic verification: D7=m integration (D2,D3,D4,D5,D6): S29, converting the complete role description document D7 into a structured data structure D8; S210, passing the generated detailed role description document D8 to the automatic processing module.

3. The highly efficient virtual character generation system based on a multimodal large model according to claim 1 is characterized in that: The automated processing module of S3 specifically includes: S31. Simplify the grammar of the role description generated by GTP-4, and convert complex sentences and long paragraphs into more concise and clear descriptions; S32, extracting key information from the character description, including appearance features, clothing style, posture, expression and background scene; S33, through format optimization, convert the extracted key information into the input format accepted by the Stable Diffusion model; S34, optimizing and combining the converted descriptions to generate multiple groups of description combination prompts; S35, sorting the multiple groups of description combination prompts, and preferentially selecting the combination that best matches the user's preliminary description; S36. Input the finalized description combination prompts into the Stable Diffusion model.

4. The highly efficient virtual character generation system based on multimodal large model according to claim 3 is characterized in that: The format optimization specifically includes: converting the description of the appearance features into specific visual elements, standardizing the description of the clothing style, converting the description of the posture into specific action instructions, converting the description of the expression into an emotion label, and converting the description of the background scene into environment setting parameters.

5. The highly efficient virtual character generation system based on multimodal large model according to claim 1 is characterized in that: The S4 specifically includes: S41. In the Stable Diffusion model, initialize the random noise vector z, z~N(0,I), where N(0,I) represents a multivariate normal distribution with a mean of 0 and a covariance of the identity matrix I; S42, using a diffusion process to iterate the random noise vector z multiple times, and in each iteration, gradually removing the noise through a reverse diffusion process; S43, in each iteration, conditionally constraining the noise vector using a conditional generation network; S44, calculating the matching degree F between the generated image and the character description, wherein the matching degree F is obtained by calculating the feature vector similarity between the generated image and the character description: F = cos(θ); Among them, θ represents the angle between the generated image and the character description feature vector; S45, optimizing and adjusting the generated image according to the matching degree F, and continuously adjusting the noise vector z through a gradient descent algorithm; S46. During the generation process, the attention mechanism is used to emphasize the key details in the character description; S47. Output a character image with high resolution and rich detail information.

Citation Information

Cited By

  • AI model automatic generation method

    CN119850774A

  • Virtual character construction system and method based on multi-modal information fusion

    CN119850801A

  • A virtual character construction system and method based on multimodal information fusion

    CN119850801B

  • Movie and television role and scene graph generation method and system based on AIGC

    CN119963679A

  • Data processing method and device, electronic equipment and computer readable storage medium

    CN119971503A