Vehicle Image Frame Reconstruction With Motion-Selective Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high volume of data generated by vehicle sensors for autonomous driving systems (ADS) increases memory demands and data transfer costs, and buffering camera images at a lower frame rate risks missing critical information during resimulation.
Innovation Solution
An image frames handling system that stores object-level environmental data and image frames at predeterminable rates, extracts key frames and pixels with significant motion, and transfers these to an offboard entity for reconstruction, allowing interpolation to achieve full frame rate with reduced data volume.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If camera images are buffered at a lower frame rate to reduce data volume, then data transfer costs and memory demands are reduced, but the risk of missing critical information increases
Solution Approach 1:
The patent segments the image data into key frames and non-key frames, and further divides non-key frames into pixels with significant motion and pixels without significant motion. This segmentation allows selective transmission of only the most important data portions, reducing overall data volume while preserving critical information about motion events and scene changes.
Solution Approach 2:
The patent extracts only the essential information from the complete image data stream. Specifically, it extracts key frames that capture important scene states and extracts only pixels from non-key frames where motion exceeds a threshold. This extraction approach reduces data volume by removing redundant information while maintaining reliability through preservation of motion-critical data.
2Measurement precision
If full sensor data is transferred at full sampling rate for resimulation, then measurement precision is maintained, but memory allocation demands and data transfer costs increase significantly
Solution Approach 1:
The patent applies local quality by treating different parts of the image data differently based on their importance. Key frames are transmitted at full resolution, while non-key frames are transmitted with only motion-significant pixels. This localized approach maintains high measurement precision for critical regions and time points while significantly reducing overall data volume for less critical areas.
Solution Approach 2:
The patent changes the parameter of data transmission quality dynamically. Instead of always transmitting at full resolution, the system adjusts the transmission quality parameter based on the importance of each frame and pixel region. This allows full precision where needed (key frames, high-motion pixels) and reduced precision where acceptable (static regions, non-critical time points), optimizing the balance between precision and data volume.
Data Source
AI summary
An image frames handling system for reconstruction of image frames obtained by an image capturing device of an Automated Driving System, ADS, of a vehicle at an entity. The image system stores in a surrounding state data buffer obtained object-level environmental data indicating states of vehicle surroundings derived from perception data output from an onboard perception system; stores image frames of vehicle surroundings captured by the capturing device in an image data buffer; processes the image data buffer to determine motions of objects in the image frames and respective values of the motions; extracts key image frames from the image data buffer and pixels having motion values exceeding a predeterminable level from non-key image frames of the image data buffer; transfers the extracted key image frames and the extracted pixels; and reconstructs at the entity interpolated image frames at the predeterminable frame rate based on the selected data.