Virtual Viewpoint Image Distortion Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating virtual viewpoint images from multiple cameras suffer from unnatural object deformation due to distortion correction and require accurate object distance information, leading to instability in video generation.
Innovation Solution
An image processing apparatus that corrects distortion and ambient light decrease characteristics in virtual viewpoint images using specific calculation units and correction processes, ensuring pixel value accuracy and natural image representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If distortion correction is applied to create a virtual viewpoint image, then the image can be generated from multiple cameras, but object deformation in peripheral parts becomes larger and the image appears unnatural
Solution Approach 1:
The patent applies different distortion correction methods to different regions of the image. For the virtual viewpoint image, it uses distortion characteristics of a virtual camera to correct coordinates, while for the captured image from the imaging unit, it uses the imaging unit's distortion characteristics. This local differentiation ensures that each region receives appropriate correction, preventing excessive object deformation in peripheral parts while maintaining natural appearance.
Solution Approach 2:
The patent introduces an intermediary correction process that maps coordinates between the virtual viewpoint image and the captured image. By calculating coordinates in the virtual viewpoint image after distortion correction, then mapping to ideal coordinates and finally to real coordinates in the captured image, it creates a smooth transition that prevents abrupt deformation and maintains natural object appearance.
2Manufacturing precision
If geometrical projective transformation is used to create a video without distortion, then distortion is corrected, but object distance information is required which is difficult to acquire accurately
Solution Approach 1:
The patent extracts and utilizes only the necessary distortion characteristics parameters from the imaging unit and virtual camera, rather than requiring complete object distance information. By focusing on distortion characteristics (k1, k2, k3 values) and using them to correct coordinates directly, it avoids the need for complex object distance estimation while still achieving effective distortion correction.
Solution Approach 2:
The patent changes the approach from using object distance information to using distortion characteristics parameters. Instead of requiring accurate depth data for projective transformation, it uses distortion parameters (k1, k2, k3) to directly correct coordinates, simplifying the information requirements while maintaining correction accuracy.
3Stability of the object's composition
If multiple cameras are used to capture identical scenes, then continuity can be improved, but brightness and color adjustment between cameras becomes complex
Solution Approach 1:
The patent applies preliminary correction of distortion and ambient light amount decrease characteristics to each captured image before combining them into the virtual viewpoint video. By pre-correcting the imaging unit's distortion and vignetting effects, it simplifies the subsequent brightness and color adjustment process, as the images are already normalized to similar characteristics.
Solution Approach 2:
The patent equalizes the characteristics of multiple captured images by applying the same correction process (distortion correction and ambient light amount decrease correction) to all images from different imaging units. This creates equipotential conditions where all images have similar distortion and lighting characteristics, simplifying the blending process and reducing complexity in brightness and color adjustment.
Data Source
AI summary
An image processing apparatus executes a distortion correction on coordinates of a target pixel in a virtual viewpoint image based on distortion characteristics of a virtual camera and calculates coordinates in the virtual viewpoint image after the distortion correction. The image process apparatus calculates ideal coordinates in a captured image from the coordinates in the virtual viewpoint image after the distortion correction and calculates real coordinates in the captured image from the ideal coordinates in the captured image based on distortion characteristics of an imaging unit. The image process apparatus calculates a pixel value corresponding to the real coordinates from image data of the virtual viewpoint image and corrects the pixel value corresponding to the real coordinates based on ambient light amount decrease characteristics of the imaging unit and ambient light amount decrease characteristics of the virtual camera.