Synthesizing Depth Images from Multiple Cameras Using Virtual Camera Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth cameras have a limited field of view, making it difficult to meet the requirements of practical applications such as video monitoring and motion sensing games, as they cannot capture a large area effectively.
Innovation Solution
A method and device for synthesizing depth images from multiple depth cameras into a single depth image with a larger field of view by establishing a virtual depth camera with calculated camera parameters, including lens optical center positions and focal distances, and using geometric relationships to map pixel points from real depth cameras to the virtual depth camera.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of moving object
If depth images from multiple depth cameras are synthesized, then the field of view is improved, but the device complexity increases
Solution Approach 1:
The patent creates a virtual depth camera that copies the functionality of multiple real depth cameras. By establishing a virtual camera model with synthesized parameters (optical center, focal length, projection matrix) based on the real cameras' parameters and relative positions, the system achieves a large field of view without physically deploying multiple cameras. This virtual copy approach resolves the contradiction by achieving the desired field of view while avoiding the complexity of managing multiple physical devices.
Solution Approach 2:
The patent introduces a virtual depth camera as an intermediary between multiple real depth cameras and the final synthesized depth image. This virtual camera serves as a mediator that integrates information from multiple real cameras through coordinate transformations and parameter synthesis. The intermediary approach simplifies the system by providing a unified virtual perspective rather than directly combining multiple complex camera systems.
2Area of stationary object
If multiple depth cameras are used to increase field of view, then the coverage area is improved, but the system complexity increases
Solution Approach 1:
The patent merges multiple depth cameras into a single virtual depth camera system. By combining the intrinsic parameters (focal length, optical center) and extrinsic parameters (position, orientation) of multiple real cameras, the system creates a unified virtual camera that provides a comprehensive coverage area. This merging approach achieves large coverage while reducing system complexity by treating multiple cameras as one integrated virtual entity rather than separate devices.
Solution Approach 2:
The virtual depth camera serves multiple functions simultaneously: it provides the coverage area of multiple real cameras, simplifies the system model to a single camera entity, and enables unified image synthesis and processing. This multi-functionality resolves the contradiction by making the virtual camera a universal solution that handles both the expanded coverage requirement and the complexity reduction goal.
Data Source
Figure 1~2a
Figure 2b~3b
Figure 4~5a
AI summary
Embodiments of the present application disclose a method and a device for synthesizing depth images, which relate to the technical field of image processing. The method includes: obtaining depth images to be synthesized, which are shot by real depth cameras respectively, wherein y-axes of respective camera coordinate systems of the real depth cameras are parallel to each other; calculating a mapping pixel point in a camera coordinate system corresponding to a virtual depth camera for each of pixel points in the depth images to be synthesized, according to pre-obtained pixel point mapping relationship, wherein camera parameters of the virtual depth camera are determined according to camera parameters of the real depth cameras, a lens optical center of the virtual depth camera is located in a connecting line of lens optical centers of the real depth cameras, an x-axis of the camera coordinate system corresponding to the virtual depth camera is parallel to the connecting line of lens optical centers of the real depth cameras; and generating a target synthesized depth image corresponding to the depth images to be synthesized, according to calculated mapping pixel points. By applying the solutions provided by the embodiments of the present application, a depth image with a large field of view can be obtained by image synthesis.