Camera Pose Estimation Using Reference Patterns for Virtual Scene Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual film production methods often result in rendering errors due to synchronization issues between pose information obtained by a locator and the actual picture, leading to poor presentation effects.
Innovation Solution
An image processing method that determines camera pose information based on position reference patterns in different coordinate systems, synchronizing the photographing and pose determination, and fuses the actual object image with a virtual background using intrinsic information and target pose information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a locator is installed on a photographing device to obtain pose information, then the pose information can be obtained for controlling rendering of a virtual scene, but the virtual scene often does not match with the real subject, causing rendering errors
Solution Approach 1:
A position reference pattern is introduced as an intermediary element that appears in both the real scene (photographed by the camera) and the virtual scene (rendered background). This pattern serves as a common reference that enables accurate alignment and synchronization between the real and virtual components, resolving the mismatch problem without requiring complex hardware locators.
Solution Approach 2:
The position reference pattern is placed in the real scene and its corresponding image is captured by the camera. The same pattern is then reproduced in the virtual scene at the calculated position. This copying approach ensures that the real subject and virtual background are spatially aligned, eliminating rendering errors while maintaining synchronization.
2Reliability
If hardware locators and professional camera equipment are used to ensure synchronization, then accurate pose information can be obtained, but the cost of virtual film production increases
Solution Approach 1:
The position reference pattern is a simple, inexpensive graphical element that can be easily created and displayed. It replaces expensive hardware locators and professional camera equipment, achieving the same synchronization function at minimal cost. The pattern is temporary and does not require permanent installation or complex hardware infrastructure.
Solution Approach 2:
The mechanical system of hardware locators attached to cameras is replaced with a computational approach using position reference patterns and image processing. The pose information is derived from analyzing the position reference pattern in the captured image rather than from mechanical sensors, eliminating the need for complex hardware while maintaining accuracy.
3Productivity
If a locator is used to obtain pose information, then rendering can be controlled, but the presentation effect of the video is poor due to rendering errors
Solution Approach 1:
The position reference pattern provides a feedback mechanism where the camera captures the pattern in the real scene, the system calculates the camera pose based on the pattern's position, and this pose information is used to correctly position the virtual background. This closed-loop feedback ensures that the rendering is accurately aligned with the real subject, improving picture rendering precision.
Data Source
AI summary
Embodiments of this application provide an image processing method performed by a computer device. The method includes: obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern; determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively; determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; and fusing a target object image and a virtual background image in the actual image based on intrinsic information and the pose information to obtain a target image.