Hierarchical Data Structure for Autonomous Vehicle File Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges with large file sizes and inefficient data management in autonomous vehicles, leading to bandwidth, storage, and compute resource inefficiencies due to the need to transmit and access extensive data sets, particularly when only a small portion is needed for analysis.
Innovation Solution
A hierarchical data structure is implemented to organize data from autonomous vehicles, partitioning files by time windows and topics, allowing for efficient storage, access, and analytics, with features like customizable topic groupings and granular file sizes to facilitate faster data retrieval and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If data is logged to a single file, then data storage is simple, but file size becomes very large causing problems with transmission, access, and manipulation
Solution Approach 1:
The patent divides the single large data file into multiple smaller files organized in a hierarchical directory structure. Data is segmented by time windows (e.g., daily, hourly) and further divided into topic-specific subdirectories. This segmentation reduces individual file sizes, making them easier to transmit, access, and manipulate while maintaining organized storage.
2Loss of information
If all data is transmitted to remote locations, then complete data is available for analysis, but bandwidth and storage resources are wasted since only a small portion is needed
Solution Approach 1:
The patent extracts and transmits only the specific data subsets needed for analysis rather than transmitting all collected data. The hierarchical structure allows remote systems to request and receive only particular topics or time ranges, extracting the necessary information while leaving unnecessary data at the source, thereby reducing bandwidth consumption.
3Quantity of substance
If large data files are accessed offline, then all data is available, but local copying and opening becomes inefficient
Solution Approach 1:
The hierarchical file structure segments data into smaller, manageable files organized by time and topic. This allows offline systems to access only the specific smaller files needed rather than copying and opening a single large file, significantly improving data access efficiency while maintaining full data availability when needed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure is directed to a hierarchical data structure, and systems and techniques for using the same. In one example, first device data including a plurality of files and corresponding to message data generated by one or more components of a device during a time period, may be received. The first plurality of files may be stored on one or more data stores and at a first location according to a hierarchical data structure, the hierarchical data structure based at least in part on a message time, a data identifier, or a device identifier. Second device data, associated with the time period and including a second plurality of files, may be obtained. The second device data may then be stored at a second location on the one or more data stores according to the hierarchical data structure.