Spatiotemporal Motion AI for Text-to-Video Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to convert text data into video data representing movement information, limiting the expression of motion information beyond textual descriptions.
Innovation Solution
A device and method utilizing a motion generation artificial intelligence algorithm based on spatiotemporal features, including modules for sentence separation, text feature extraction, motion feature search, and motion reconstruction to generate motion information from text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If text data is converted to video data representing motion information, then motion information expression capability is improved, but device complexity increases due to multiple processing modules
Solution Approach 1:
The system segments the text-to-motion conversion process into five distinct modules: sentence separation module divides input text into individual sentences; text feature extraction module extracts semantic features from each sentence; motion feature search module queries a database for corresponding motion features; motion information integration module integrates temporal and spatial information; and motion reconstruction module generates final video data. This segmentation allows each module to specialize in a specific task, improving overall motion information expression while managing complexity through modular design.
Solution Approach 2:
The patent introduces a motion feature database as an intermediary component that stores pre-extracted motion features from various motion items. This database acts as a mediator between the text processing modules and the video generation module, enabling efficient retrieval and integration of motion information without requiring complex real-time processing, thus improving motion information expression capability while maintaining manageable device complexity.
2Manufacturing precision
If multiple processing modules are used to convert text to motion information, then motion reconstruction accuracy is improved, but processing time increases
Solution Approach 1:
The system performs preliminary actions by pre-extracting and storing motion features from various motion items in a motion feature database before actual text-to-motion conversion is needed. The database contains pre-processed motion information including temporal and spatial features. When text input is received, the system only needs to query and integrate these pre-extracted features, significantly reducing processing time while maintaining high motion reconstruction accuracy through the use of pre-validated motion data.
3Reliability
If temporal and spatial features are integrated in motion generation, then motion information quality is improved, but computational resources required increase
Solution Approach 1:
The patent segments temporal feature extraction and spatial feature extraction into separate processing streams within the text feature extraction module. Temporal features (such as motion duration, speed changes) are extracted and integrated separately from spatial features (such as position, orientation). This segmentation allows the system to process and integrate only the necessary features for each motion dimension, improving motion information quality through comprehensive temporal-spatial integration while optimizing computational resource usage by avoiding redundant processing.
Data Source
AI summary
Disclosed is a device including a motion generation artificial intelligence algorithm based on spatiotemporal feature of motion and an operating method thereof, and the device may include a memory configured to store at least one process for executing the motion generation artificial intelligence algorithm based on spatiotemporal features of motion; and a processor configured to execute the motion generation artificial intelligence algorithm based on spatiotemporal feature of motion according to the process.