Prompt-Decomposed Video Generation for Controllable Actions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence-based video generation technologies face challenges in controllability, editability, personalization, and stability, particularly in complex scenarios, leading to inaccurate interactions, high resource overhead, and difficulty in generating long-form videos with user-specific characteristics.
Innovation Solution
A method involving a conversational large model to determine initial prompt texts, generate video content and object action driving data, and adjust material attributes to produce high-quality videos efficiently, reducing labor and time costs, and enhancing video coherence and personalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If existing AI-based video generation technologies are used, then video generation can be automated, but the controllability and editability are insufficient
Solution Approach 1:
The patent segments the video generation process into multiple independent modules: text prompt processing, character generation, scene generation, and video synthesis. Each module can be independently controlled and adjusted, enabling fine-grained controllability while maintaining automation. The segmented architecture allows users to modify specific components (e.g., character appearance, scene layout) without affecting the entire generation pipeline.
Solution Approach 2:
The system implements dynamic adjustment capabilities where generated video parameters can be modified in real-time based on user feedback. The architecture allows dynamic re-generation of specific frames or segments while maintaining consistency with the overall video narrative, providing editability without compromising the automated generation process.
2Extent of automation
If existing AI-based video generation technologies are used, then video generation can be automated, but the stability is insufficient leading to inaccurate interactions
Solution Approach 1:
The patent performs preliminary actions by generating detailed character descriptions, scene layouts, and interaction scripts before actual video synthesis. The system pre-calculates object relationships, camera angles, and temporal sequences to ensure consistency throughout the video. This preliminary planning phase prevents inaccurate interactions during generation by establishing clear guidelines for object behavior and spatial relationships.
Solution Approach 2:
The system incorporates feedback mechanisms where generated video segments are evaluated for consistency and accuracy. User feedback or automated quality assessment can trigger re-generation of specific segments to correct inaccuracies. This feedback loop ensures stable and reliable video output by continuously refining the generation process based on observed deviations from intended behavior.
3Extent of automation
If existing AI-based video generation technologies are used, then video generation can be automated, but the resource overhead is high
Solution Approach 1:
The patent applies partial action by generating only the necessary video segments and character animations required for the specific prompt, rather than rendering complete scenes from scratch. The system generates intermediate representations (such as keyframe images or skeletal animations) that can be efficiently synthesized into final video, reducing computational resource overhead while maintaining automation.
Solution Approach 2:
The system performs preliminary computation to create reusable character models, scene templates, and animation libraries that can be efficiently combined to generate multiple videos. By pre-computing common elements (e.g., character poses, environmental backgrounds), the system reduces redundant computational overhead during actual video generation, making the automated process more resource-efficient.
4Extent of automation
If existing AI-based video generation technologies are used, then video generation can be automated, but the personalization is insufficient
Solution Approach 1:
The patent implements local quality by allowing users to specify detailed characteristics for specific elements (characters, objects, scenes) while maintaining automated generation for the overall video structure. Users can customize character appearance, personality traits, and interaction styles for particular segments without affecting the automated narrative flow. This enables high personalization in key areas while preserving the benefits of automated generation.
Solution Approach 2:
The system provides dynamic personalization where generated content can be adjusted based on user preferences, target audience, or specific application requirements. The architecture allows real-time modification of character attributes, scene styles, and dialogue content to match user-specific needs while maintaining the automated generation pipeline. This dynamic adaptability enables the same automated system to serve diverse personalization requirements.
Data Source
AI summary
A method of generating a video, a method of processing a video, an electronic device and a storage medium, which relate to a field of artificial intelligence technology, and in particular to fields of large model technology, video processing technology, virtual digital character technology, etc. The method of generating a video includes: determining a plurality of initial prompt texts according to an initial text input by a user, where the plurality of initial prompt texts include an initial content prompt text and an initial material prompt text; determining a video content text and at least one initial object action driving data corresponding to the video content text according to the initial content prompt text; and generating an initial video according to the at least one initial object action driving data and at least one initial material corresponding to at least one initial material prompt text.