View-Synthesis Depth Estimation With GAN-Based Distortion Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing self-supervised depth estimation methods for autonomous driving face challenges in generating high-quality synthetic images due to limitations in learning and increased inference time and memory usage, with distortions in the synthetic images often occurring.
Innovation Solution
A method involving a synthetic image model trained with a generative adversarial network (GAN) to correct distortions in synthetic images by generating virtual images that approximate the original image, using a first and second virtual image loss function to enhance the authenticity and consistency of the synthetic image generation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If self-supervised depth estimation methods are used to reduce network learning cost, then training cost is reduced, but synthetic images are generated with distorted shapes
Solution Approach 1:
The patent introduces a pose refinement network as an intermediary component between the depth model and view synthesis process. This refinement network corrects the pose estimates before they are used to generate synthetic images, thereby eliminating the distortion problem while maintaining the self-supervised training framework and avoiding additional training costs.
2Measurement precision
If CRF or RNN-based methods are added to improve depth estimation, then depth accuracy is improved, but inference time and memory usage increase drastically
Solution Approach 1:
The patent employs a self-refinement mechanism where the pose refinement network uses the output from the depth model itself to improve its own predictions. The network refines pose estimates iteratively using the synthesized images and depth information it generates, eliminating the need for external CRF or RNN modules while maintaining depth estimation accuracy.
3Manufacturing precision
If pose refinement network is trained to correct pose estimates, then synthetic image distortion is reduced, but model complexity increases
Solution Approach 1:
The patent merges the pose refinement network with the existing depth estimation and view synthesis components into a unified training framework. All components are trained jointly using a single self-supervised loss function based on image reconstruction, which simplifies the overall system architecture and training process while effectively reducing synthetic image distortion.
Data Source
AI summary
A method for controlling autonomous driving of a vehicle is introduced. The method may comprise, training, based on an inference depth and an inference pose, a synthetic image model for generating a synthetic image, generating, based on the synthetic image, a first virtual image to be associated with the original image, generating, based on the original image, a second virtual image, training a generative adversarial network (GAN) for determining, based on the original image, authenticity of the first virtual image and the second virtual image, training, based on the trained GAN, a depth network, wherein the trained GAN outputs a determination of the authenticity of the first virtual image, outputting, based on the trained depth network, signal, and controlling, based on the signal, autonomous driving of the vehicle.