Image Frame Fetching via Bounding Region Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face performance bottlenecks due to the inefficiencies in fetching image data from external memory on a block-by-block basis, leading to reduced memory bandwidth utilization and increased computational complexity during image processing.
Innovation Solution
The method involves selecting a base image frame and an alternative image frame, determining alignment information, and fetching image data from a bounding region that encompasses matching patches within the alternative frame, rather than separate patches, to combine and enhance image data for improved signal-to-noise ratio and dynamic range.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If image data is fetched on a block-by-block basis from external memory, then the image processing can be performed with fine-grained control, but the memory bandwidth utilization is reduced and the number of read operations increases
Solution Approach 1:
The patent merges multiple block-level fetch operations into a single frame-level fetch operation. By fetching the entire image frame at once from external memory instead of fetching individual blocks separately, the system reduces the number of read operations and improves memory bandwidth utilization while still enabling fine-grained processing through subsequent in-memory block operations.
2Productivity
If image data is fetched on a block-by-block basis, then memory access can be optimized for processing, but the computational complexity and bus transaction latencies increase
Solution Approach 1:
The patent performs preliminary action by fetching the entire image frame into memory before processing begins. This upfront fetch operation eliminates the need for repeated memory access control logic and reduces bus transaction overhead during processing, thereby reducing computational complexity despite the initial larger memory transfer.
3Ease of operation
If separate read operations are performed for each block, then precise memory access control is achieved, but the performance of the computing system is impacted
Solution Approach 1:
The patent combines multiple separate read operations into a single consolidated read operation for the entire image frame. This merging approach maintains precise memory access control through software-managed block operations while eliminating the performance penalties associated with multiple bus transactions and read operations.
Data Source
AI summary
At least one processor may be configured to receive a first image frame of a sequence of image frames from an image capture device and select a first portion of a first image frame. The at least one processor may also be configured to obtain alignment information and determine a first portion and a second portion of a second image frame based on the alignment information. Further, the at least one processor may be configured to determine a bounding region within the second image frame and fetch image data corresponding to the bounding region of the second image frame from memory. In some examples, the first image frame may comprise a base image and the second image frame may comprise an alternative image frame. Further, the first image frame may comprise any one of the image frames of the sequence of image frames.