Indexed Pocket File Format for Selective ROS Bag Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing simulation systems for autonomous vehicles require the entire ROS bag file to be downloaded and opened to selectively play small portions of data, consuming significant time and bandwidth.
Innovation Solution
A new file format, referred to as a 'pocket' file, with an indexed data section that allows selective streaming of data by loading only the index, enabling the host platform to download and stream only the desired intervals of data without opening the entire file.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the entire ROS bag file is downloaded and opened to selectively play small portions of data, then the desired data can be accessed, but significant time and bandwidth are consumed
Solution Approach 1:
The patent segments the continuous bag file into discrete, addressable pockets of data organized in an index structure. Each pocket represents a selectable unit of data that can be independently requested and streamed, eliminating the need to download entire files while maintaining random access capability.
Solution Approach 2:
The patent performs preliminary indexing of data pockets during file creation, organizing data into addressable segments with metadata about their location and content. This pre-organization enables direct seeking and selective streaming without requiring full file downloads, resolving the contradiction between accessibility and bandwidth consumption.
2Ease of operation
If the entire bag file is downloaded to selectively play small portions of data, then the desired intervals can be accessed, but significant time is consumed
Solution Approach 1:
By segmenting the file into indexed pockets, the system enables direct access to specific time intervals without downloading unnecessary data. The index structure allows O(1) or O(log n) lookup of pocket locations, dramatically reducing retrieval time compared to sequential file processing.
Solution Approach 2:
The patent extracts only the necessary data pockets corresponding to requested time intervals from the full bag file. This extraction approach, enabled by the index structure, allows the system to retrieve only the required portions of data, eliminating wasted time downloading and processing irrelevant content.
3Reliability
If the entire file is opened and downloaded, then all data is available, but bandwidth is significantly consumed when only small portions are needed
Solution Approach 1:
The segmentation of data into indexed pockets maintains data completeness and reliability while enabling selective transmission. The index acts as a catalog that ensures all data is available on the server side, but only the necessary segments are transmitted over the network, resolving the contradiction between data availability and bandwidth consumption.
Solution Approach 2:
The system performs partial action by downloading and streaming only the specific data pockets needed for the current playback interval, rather than the entire file. The index structure ensures that even if only a small portion is needed, the system can reliably access and stream exactly that portion without wasting bandwidth on unnecessary data transfer.
Data Source
AI summary
Provided are systems and methods for selectively streaming content using a new binary file format. In one example, a method may include storing a plurality of binary files, establishing a network communication session between the computing system and a computing terminal via a network, receiving identifiers of one or more intervals of time from the computing terminal via the network communication session, identifying a subset of data within a data section of a binary file stored in memory which is mapped to the identifiers of the one or more intervals of time based on an index within the binary file, and transmitting a stream including the identified subset of data to the computing terminal via the network communication session.