Inpainting Filter Texture Sampling for Virtual Character Videos
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques face challenges in smoothly removing human body parts from videos in real-time to incorporate virtual characters, especially when the virtual character is smaller or does not completely occlude the human body parts, leading to difficulties in video creation using virtual characters.
Innovation Solution
The implementation of an inpainting filter that accelerates texture sampling using a pyramid ping-pong operation, allowing for real-time removal of human body parts and blending the output with camera input backgrounds to create seamless videos featuring virtual characters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional video processing methods are used to remove human body parts, then the removal process is computationally intensive and time-consuming, but real-time processing capability is lost
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing texture samples at multiple scales (pyramid structure) before they are needed for processing. This allows the inpainting filter to quickly access pre-prepared texture data at appropriate scales during real-time processing, eliminating the need for computationally intensive on-the-fly calculations while maintaining high-quality removal results
2Manufacturing precision
If detailed texture sampling is performed to ensure high-quality removal, then processing accuracy improves, but computational complexity and processing time increase
Solution Approach 1:
The patent segments the texture sampling process into a pyramid structure with multiple levels of detail. Instead of processing the entire high-resolution texture at once, the system divides it into hierarchical layers, allowing selective processing of only the necessary portions at appropriate detail levels. This reduces computational complexity while maintaining the precision needed for high-quality removal results
Solution Approach 2:
The patent introduces a new dimension by creating a multi-scale pyramid structure rather than working with a single-resolution texture. This dimensional transformation allows the system to efficiently navigate through different levels of detail, selecting the appropriate scale for each processing step, thereby reducing overall computational complexity while preserving removal quality
3Adaptability or versatility
If the virtual character is smaller or does not completely occlude human body parts, then creative flexibility increases, but the difficulty of smooth removal and blending increases
Solution Approach 1:
The patent applies local quality by processing different regions of the video frame with appropriate levels of detail from the texture pyramid. Areas requiring precise blending (where the virtual character partially occludes body parts) receive higher-resolution texture samples, while other areas use lower-resolution samples. This localized approach to quality management enables smooth blending in critical regions without increasing overall computational complexity
Data Source
AI summary
The present disclosure describes techniques for creating videos using virtual characters. Creation of a video may be initiated by a user. Camera input comprising a human body of the user may be received. The camera input may be split into a first stream for removing the human body and a second stream for animating a virtual character in the video. An inpainting filter may be applied to remove the human body in real time from the camera input. The inpainting filter may be configured to accelerate texture sampling. Output of the inpainting filter may be blended with images comprised in the camera input to generate camera input backgrounds.