Frame Correction ML Model for Degraded Image Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous driving systems face challenges in processing image frames due to degradation caused by faulty cameras or external factors like bad lighting, haze, or fog, which renders images unreliable for tasks such as bird's-eye-view generation, object detection, and path planning.
Innovation Solution
A frame correction machine learning (ML) model is employed to receive and correct image frames by applying weights or biases to samples from the same camera or overlapping frames from other cameras, along with contextual information like depth data, to generate a corrected image frame, accompanied by a confidence value indicating the accuracy of the correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image frames are captured under degraded conditions (faulty cameras, bad lighting, haze, fog), then the system can continue operating, but the reliability of the image frames for downstream tasks deteriorates
Solution Approach 1:
A frame correction ML model is introduced as an intermediary component between the degraded image capture and downstream processing tasks. This model receives degraded image frames as input and outputs corrected image frames, acting as a mediator that transforms unreliable data into reliable data for subsequent processing stages such as BEV generation, object detection, and path planning
Solution Approach 2:
The patent replaces traditional mechanical/optical image correction methods with a machine learning-based computational approach. Instead of using physical filters, lenses, or optical systems to correct image degradation, the system uses an ML model that processes image data computationally to generate corrected frames, substituting physical correction mechanisms with intelligent algorithmic correction
2Reliability
If a frame correction ML model is applied to correct degraded image frames, then the reliability of image frames improves, but the device complexity increases
Solution Approach 1:
The frame correction ML model is designed to handle multiple types of image degradation (faulty cameras, bad lighting, haze, fog) and various downstream tasks (BEV generation, object detection, path planning) through a single unified system. This multi-functional approach consolidates what could be multiple separate correction systems into one versatile model, managing complexity through generalization rather than specialization
3Measurement precision
If multiple image frames from the same camera and other cameras are used as input to the frame correction ML model, then the accuracy of corrected image frames improves, but the processing time increases
Solution Approach 1:
The system performs preliminary actions by capturing and buffering multiple image frames from the same camera and other cameras before the correction process. These pre-captured frames are stored and made readily available as input data for the frame correction ML model, allowing the model to leverage temporal and spatial information without requiring real-time sequential capture, thus reducing processing delays
Data Source
AI summary
A method of processing image data includes receiving, with a frame correction machine-learning (ML) model executing on processing circuitry, an image frame captured from a first camera of a plurality of cameras; performing, with the frame correction ML model executing on the processing circuitry, image frame correction to generate a corrected image frame based on weights or biases of the frame correction ML model applied to two or more of: samples of the image frame, samples of previously captured image frames from the first camera, or samples from image frames from other cameras of the plurality of cameras; and performing, with the processing circuitry, post-processing based on the corrected image frame.