3D Posture Data Completion Using Text-to-Image Joint Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Constructing a hand action library for a three-dimensional object using an action capture glove is resource-intensive and results in poor matching between captured hand actions and limb actions, leading to inefficient and inaccurate posture data completion.
Innovation Solution
Generate a two-dimensional posture image using a text-to-image model with a posture description text and incomplete three-dimensional joint point data, perform joint point recognition to obtain missing joint point data, and combine with existing data to complete the three-dimensional posture data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If an action capture glove is used to construct a hand action library, then hand action data can be obtained, but manpower and material resource consumption increases significantly
Solution Approach 1:
The patent uses a text-to-image model to generate a two-dimensional posture image that copies the complete posture information from the three-dimensional incomplete posture data. This generated image serves as a virtual copy that contains the missing hand action data, eliminating the need for physical action capture glove recording while preserving all necessary posture information.
Solution Approach 2:
The patent replaces the mechanical action capture glove system with an artificial intelligence text-to-image model. Instead of using physical sensors and mechanical recording devices, the system uses AI algorithms to generate the missing posture data, substituting a computational approach for a mechanical measurement approach.
2Quantity of substance
If an action capture glove is used to capture hand actions, then hand action data can be obtained, but the matching degree between captured hand actions and limb actions deteriorates
Solution Approach 1:
The patent merges the hand action data generation process with the limb action data into a unified AI-based posture completion system. By using the text-to-image model to generate the two-dimensional posture image from the same three-dimensional incomplete posture data that contains limb actions, the system ensures consistent matching between hand actions and limb actions, as both are derived from the same computational process rather than separate physical capture sessions.
3Ease of operation
If a hand action library is constructed through direct capture and matching, then hand actions can be driven, but the process consumes large amounts of manpower and material resources
Solution Approach 1:
The patent enables the three-dimensional object to self-complete its posture data by using the text-to-image model to generate the missing hand action information from its own incomplete posture data. The system serves itself by internally generating the required hand action data without requiring external capture equipment or manual library construction, thereby eliminating resource consumption while maintaining ease of operation.
Data Source
AI summary
This application discloses a posture data completion method for a three-dimensional object performed by a computer device. The method includes: obtaining three-dimensional incomplete posture data of a three-dimensional object in a preset posture, wherein the three-dimensional incomplete posture data includes first three-dimensional joint point data of the three-dimensional object; generating a two-dimensional posture image of the three-dimensional object in the preset posture by applying the three-dimensional incomplete posture data and a posture description text to a text-to-image model of the preset posture; performing joint point recognition on the two-dimensional posture image to obtain second three-dimensional joint point data of the three-dimensional object; and combining the second three-dimensional joint point data and the three-dimensional incomplete posture data to obtain three-dimensional complete posture data of the three-dimensional object.