ROI Image Sensor Readout for High-Frame-Rate Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image sensors on autonomous vehicles generate more data than can be timely processed by the control system due to high resolution and large number of sensors, limiting data transfer bandwidth and processing power.
Innovation Solution
The image sensor is configured to operate at a higher frame rate and divide into regions of interest (ROIs), allowing selective readout and processing of ROI images containing objects of interest, reducing data generation and analysis while maintaining high resolution and frame rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the image sensor operates at high resolution and high frame rate to capture all ROI images, then the measurement precision and productivity are improved, but the data generation volume increases beyond processing capability
Solution Approach 1:
The image sensor is divided into multiple independently readable regions of interest (ROIs). Instead of reading out the entire high-resolution image at high frame rate, only selected ROIs containing objects of interest are read out at high frame rate, while other ROIs are read out at lower frame rate or not at all. This segmentation allows maintaining high productivity for critical areas while reducing overall data generation volume.
Solution Approach 2:
Different ROIs are assigned different readout frame rates based on their content importance. ROIs containing objects of interest are processed at high frame rate to maintain measurement precision and productivity for critical detection, while ROIs without objects of interest are processed at lower frame rate. This local differentiation resolves the contradiction by applying high quality only where needed.
2Measurement precision
If all ROI images are processed at high frame rate, then the object detection precision is improved, but the processing power and bandwidth requirements exceed available resources
Solution Approach 1:
The system extracts only the necessary ROI images containing objects of interest for high-frame-rate processing, while excluding ROIs without objects of interest from high-frame-rate processing. This extraction approach maintains high measurement precision for object detection in critical areas while significantly reducing the total processing power and bandwidth requirements.
Solution Approach 2:
Instead of processing all ROI images at high frame rate (excessive action), the system applies partial action by processing only the subset of ROIs that contain objects of interest at high frame rate. This partial processing approach maintains sufficient measurement precision for detection while reducing processing power requirements to match available resources.
3Measurement precision
If the full-resolution image is captured and processed, then the measurement precision is improved, but the data transfer bandwidth is exceeded
Solution Approach 1:
The full-resolution image is segmented into multiple ROIs. Instead of transferring the complete high-resolution image data, only the selected ROIs containing objects of interest are transferred at high frame rate. This segmentation maintains measurement precision for critical detection areas while significantly reducing data transfer bandwidth requirements.
Solution Approach 2:
The system extracts and transfers only the necessary ROI portions containing objects of interest rather than the entire full-resolution image. This extraction approach preserves measurement precision for critical detection while reducing data transfer bandwidth to match available capacity.
Data Source
AI summary
A system includes an image sensor having a plurality of pixels that form a plurality of regions of interest (ROIs), and configured to operate at a frame rate higher than a threshold rate. The system also includes an image processing resource. The system further includes control circuitry configured to perform operations that include obtaining, from the image sensor, a full-resolution image of an environment. The full-resolution image contains each respective ROI of the plurality of ROIs. The operations also include selecting a particular ROI based on the full-resolution image, and detecting an object of interest in the particular ROI. The operations include determining a mode of operation by which subsequent image data generated by the particular ROI is to be processed. The operations further include processing, based on the mode of operation and the frame rate, the image data comprising a plurality of ROI images of the object of interest.