GAN-Based Image Stabilization for Autonomous Vehicle Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicle CNNs face challenges in tracking objects due to camera shake-induced jittering, particularly in off-road conditions, with existing methods either increasing camera weight and cost or causing image distortion when correcting for jittering.
Innovation Solution
A learning method using a Generative Adversarial Network (GAN) with a generating network and a discriminating network to adjust initial images, calculate naturality, maintenance, and similarity scores, and learn parameters to reduce distortion in warped images, thereby stabilizing jittered images effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If physical correction of jittered images by measuring camera movement is used, then jittering is reduced, but camera weight increases and cost becomes expensive
Solution Approach 1:
The patent replaces the mechanical stabilization system (physical camera movement correction) with a software-based image processing system. The generating network and discriminating network work together to correct jittered images through computational methods, eliminating the need for additional mechanical components and reducing camera weight while maintaining image stability.
Solution Approach 2:
The patent introduces a software intermediary (the image processing system with generating and discriminating networks) that mediates between the jittered input image and the stabilized output image. This intermediary processes the image data to reduce jittering effects without requiring physical modification of the camera system.
2Stability of the object's composition
If software technique with optical flow is used to correct jittered images, then jittering is reduced, but image distortion increases and objects become undetectable
Solution Approach 1:
The patent employs feedback mechanisms where the discriminating network evaluates the generated corrected images and provides feedback to the generating network. This feedback loop continuously refines the image correction process, adjusting the generated images to reduce distortion while maintaining stability, thereby preventing objects from becoming undetectable.
Solution Approach 2:
The patent changes the parameters of the image processing by using a generative adversarial network approach instead of traditional optical flow methods. The generating network learns optimal transformation parameters to correct jittering, while the discriminating network ensures these parameters do not introduce excessive distortion, thus maintaining object detectability.
3Manufacturing precision
If GAN is used to reduce distortion in warped images, then image naturalness is improved, but system complexity increases
Solution Approach 1:
The patent segments the complex image processing task into two specialized networks: a generating network that creates corrected images and a discriminating network that evaluates them. This segmentation allows each network to focus on a specific function, improving image naturalness through specialized processing while managing system complexity through modular design.
Solution Approach 2:
The patent creates a multi-functional system where the generating network performs both image correction and the discriminating network performs both evaluation and feedback provision. This universal design allows the GAN system to handle multiple aspects of image stabilization and distortion reduction within a unified framework, managing complexity through integrated functionality.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for learning reduction of distortion occurred in a warped image by using a GAN is provided for enhancing fault tolerance and fluctuation robustness in extreme situations. And the method includes steps of: (a) if an initial image is acquired, instructing an adjusting layer included in the generating network to adjust at least part of initial feature values, to thereby transform the initial image into an adjusted image; and (b) if at least part of (i) a naturality score, (ii) a maintenance score, and (iii) a similarity score are acquired, instructing a loss layer included in the generating network to generate a generating network loss by referring to said at least part of the naturality score, the maintenance score and the similarity score, and learn parameters of the generating network. Further, the method can be used for estimating behaviors, and detecting or tracking objects with high precision, etc.