Selective Extended Depth-of-Field Correction for 3D Telepresence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image generation techniques for 3D telepresence are inadequate in correcting images for out-of-focus blurring, leading to low-quality, unrealistic images with latency, due to limitations in machine learning algorithms and neural network models, which result in a poor immersive experience.
Innovation Solution
A system and method that selectively applies extended depth-of-field correction to images and three-dimensional models based on camera pose and optical focus, using a server and data repository to process images, generate 3D models, and determine the need for EDOF correction, applying it to specific portions of images or models to enhance realism and fidelity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning-based algorithms are used to correct images by solving inverse convolution function, then image correction capability is provided, but electronic noise, temperature drift, and camera variations cause poor correction quality
Solution Approach 1:
The patent introduces an optical model of the camera as an intermediary between the raw image and the correction process. By modeling the camera's optical characteristics (lens parameters, aperture, focal length) and using this model to guide the correction algorithm, the system achieves more reliable and precise correction results that are robust to noise, temperature drift, and camera variations.
2Reliability
If neural network models are used to remove visual artifacts, then artifact removal capability is achieved, but computational intensity and time consumption increase significantly
Solution Approach 1:
The patent segments the image correction process into distinct stages: optical parameter estimation, depth map generation, and selective correction application. By dividing the task and applying correction only to specific regions (out-of-focus areas) rather than the entire image, the computational load is significantly reduced while maintaining high correction quality.
Solution Approach 2:
Instead of applying full neural network-based correction to the entire image, the patent applies correction selectively only to portions of the image that require it (out-of-focus regions). This partial action approach reduces computational intensity and processing time while still achieving the desired artifact removal capability.
3Measurement precision
If full image correction is applied to enhance visual fidelity, then image quality improves, but processing time and latency increase
Solution Approach 1:
The patent applies different processing quality to different regions of the image. Out-of-focus regions receive full correction processing to enhance visual fidelity, while in-focus regions are left unchanged. This local quality approach ensures high fidelity where needed while minimizing processing time overall.
Data Source
AI summary
A system including server(s) and data repository, wherein server(s) is/are configured to receive images of real-world environment captured using camera(s), corresponding depth maps, and at least one of: pose information, relative pose information; generate three-dimensional model (3D) of real-world environment; store 3D model; utilise 3D model to generate output image from perspective of new pose; determine whether extended depth-of-field (EDOF) correction is required to be applied to any one of: at least one of images captured by camera(s) representing given object(s), 3D model, output image, based on whether optical focus of camera(s) was adjusted according to optical depth of given object from given pose of camera; and when it is determined that EDOF correction is required to be applied, apply EDOF correction to at least portion of any one of: at least one of images captured by camera(s), 3D model, output image.