3D Point Cloud Image Restoration via Pseudo-Projection Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques fail to effectively restore an image captured by a camera from a projection diagram where a three-dimensional shape is projected onto a two-dimensional plane along a sensor's line-of-sight direction.
Innovation Solution
An information processing apparatus and method that generates a pseudo-projection image using three-dimensional point cloud data and a captured image, employing supervised learning with the pseudo-projection image as an explanatory variable and the captured image as ground truth data to infer the original image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a projection diagram is generated by projecting three-dimensional data onto a two-dimensional plane, then the three-dimensional coordinates can be accurately estimated, but the technique cannot restore the original captured image from the projection diagram
Solution Approach 1:
The patent creates a pseudo-projection image that copies the characteristics of both the projection diagram and the captured image. By generating this intermediate representation that mimics the appearance and structure of the original captured image while being derived from the projection diagram, the system enables restoration of the captured image content without requiring the original image data
Solution Approach 2:
The patent introduces a pseudo-projection image as an intermediary between the projection diagram and the captured image. This intermediary representation serves as a bridge that allows information to be transferred and transformed from the three-dimensional projected data back into a two-dimensional captured image format, enabling the restoration process
2Measurement precision
If supervised learning is performed using captured images and projection diagrams, then image restoration accuracy improves, but a large amount of paired training data is required which is difficult to obtain
Solution Approach 1:
The system generates its own training data by automatically creating pseudo-projection images from available projection diagrams and captured images. This self-service approach eliminates the need for manual pairing of training data, as the system can generate unlimited training samples from existing three-dimensional data and captured images, thereby solving the data scarcity problem
Solution Approach 2:
The patent performs preliminary generation of pseudo-projection images to create training data sets before the actual image restoration task. By pre-generating these training samples with known ground truth (the original captured images), the system prepares充足的 training data in advance, enabling effective supervised learning without requiring external data collection efforts
Data Source
AI summary
Provided is an information processing apparatus including: an acquisition unit configured to acquire three-dimensional point cloud data and a captured image captured by an imaging apparatus under a first imaging condition; a generation unit configured to generate a projection image under a second imaging condition, based on the three-dimensional point cloud data, and generate a pseudo-projection image, based on the projection image and the captured image; and a setting unit configured to set learning data for performing supervised learning by using the pseudo-projection image as an explanatory variable and using the captured image as ground truth data.