Segmented Video Data Synchronization for Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently combining and synchronizing compressed video streams with other sensor data in autonomous driving vehicles, leading to difficulties in storage and real-time use for motion planning and control.
Innovation Solution
A computer-implemented method and sensor unit that segment video data from cameras into independently decodable segments or smaller video units, synchronize them with sensor data from other sensors like LIDAR and RADAR, and combine these with corresponding time-stamped data to generate combined sensor data for efficient storage and real-time use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If video data is compressed using standard compression algorithms, then storage efficiency is improved, but the ability to combine and synchronize with other sensor data deteriorates
Solution Approach 1:
The patent segments compressed video data into individual frame units that can be independently processed and synchronized with other sensor data. Each video frame is treated as a discrete unit with timestamp metadata, enabling granular synchronization with LIDAR, radar, and other sensor readings without requiring full video stream processing.
Solution Approach 2:
The patent introduces an intermediary processing layer that receives both compressed video streams and other sensor data, performs time-synchronization based on timestamps, and combines them into a unified dataset. This intermediary mechanism bridges the gap between compressed video format and multi-sensor integration requirements.
2Ease of operation
If video data is segmented into smaller units for synchronization, then ease of combining with sensor data is improved, but processing complexity increases
Solution Approach 1:
The patent divides video data into frame-level segments with embedded timestamps, enabling independent processing of each segment. This segmentation allows parallel processing of multiple video frames simultaneously with corresponding sensor data, reducing overall processing complexity through divide-and-conquer approach.
Solution Approach 2:
The patent changes the parameter of video data representation by adding timestamp metadata to each segmented frame. This parameter change enables time-based synchronization with other sensors without requiring complex spatial or temporal transformations, simplifying the combining process.
3Speed
If real-time synchronization is implemented for motion planning, then responsiveness is improved, but computational load increases
Solution Approach 1:
The patent performs preliminary actions by pre-segmenting video data into frames with embedded timestamps during video encoding or preprocessing. This preliminary segmentation and timestamping reduces the computational load during real-time synchronization, as the data is already organized for efficient time-based matching with sensor readings.
Solution Approach 2:
The patent implements partial synchronization by focusing computational resources on synchronizing only the necessary video frames with sensor data for current motion planning decisions. Rather than continuously processing all video data at full resolution, the system selectively processes relevant segments, reducing overall computational load while maintaining real-time responsiveness.
Data Source
AI summary
An ADV includes a method to combine data from multiple sensors. The method compresses video data from a camera to generate compressed video data. The compressed video data are segmented. The method time synchronizes each segment of the compressed video data with data from other sensors. The method then combines each segment of the compressed video data with the corresponding time-synchronized sensor data for the other sensors. In one embodiment, each segment of the compressed video data is independently decodable. In another embodiment, each segment of the compressed video data includes a compressed video unit that is prepended with a buffered portion of the compressed video data that immediately precede the compressed video unit. The length of the compressed video unit is smaller than the length of the independently decodable segment to offer finer granularity in time synchronizing the compressed video data with the other sensor data with a tradeoff.