Virtual Asset Insertion Using Visual Patterns for Occlusion Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video streaming technologies struggle to seamlessly integrate virtual assets, such as advertisements, into live video streams without causing unnatural occlusions or distortions, particularly when objects in the scene temporarily block the view of the virtual assets.
Innovation Solution
A method involving the use of a visual pattern, like a checkerboard, printed on a physical object within the scene, which is detected and used to insert virtual assets accurately, adjusting for occlusions and matching the scene's color and blurriness to create a natural appearance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual assets are inserted into live video streams using existing technologies, then advertising content can be provided in association with primary video content, but the virtual assets may cause unnatural occlusions or distortions when objects in the scene temporarily block the view
Solution Approach 1:
The system performs preliminary actions by detecting occlusion objects and determining occluded portions of visual patterns before completing the virtual asset insertion. This allows the system to pre-calculate which portions of virtual assets should be excluded or adjusted, preventing unnatural occlusions and distortions from occurring in the final rendered video stream.
2Loss of time
If real-time virtual asset insertion is implemented, then latency is reduced and viewer engagement is maintained, but the complexity of detecting and adjusting for occlusions increases
Solution Approach 1:
The system uses visual patterns as intermediaries to simplify occlusion detection. Instead of directly analyzing complex scene geometry and object boundaries, the system detects predefined visual patterns that are easier to identify and track. This intermediary approach reduces computational complexity while maintaining real-time performance capability.
3Measurement precision
If visual patterns are used to mark insertion locations, then virtual assets can be inserted accurately, but the scene requires additional physical objects with visual patterns
Solution Approach 1:
The system creates a virtual copy of the visual pattern at the detected location, rendering the virtual asset to match the pattern's position, orientation, and scale. This copying approach allows accurate virtual asset placement based on the detected visual pattern without requiring permanent physical modifications to the scene, as the pattern can be temporarily placed and then digitally replicated.
Data Source
AI summary
At least part of a visual pattern may be detected, in an image of a video, at a location within the image. An occlusion determination may be performed corresponding to the at least part of the visual pattern. At least part of a virtual asset may be inserted, based at least in part on the occlusion determination, into the image at the location. An appearance of the at least part of the virtual asset within the image may be adjusted based at least in part on an appearance of the at least part of the visual pattern within the image.