Content-Hashed Object Storage for Edge Data Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing edge computing applications, such as video surveillance systems, face challenges with data scalability, real-time response latency, and the need for power-consuming data analytics processing, particularly in conventional computing architectures that rely on file-based storage systems.
Innovation Solution
The implementation of a content-hashed object storage architecture that uses edge processing nodes to detect features from sensor signals via deep-learning networks, creating machine-learned metadata and hashing it for efficient storage and retrieval, allowing for in-store computing and flexible scalability without burdening customer systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If file-based storage systems are used in conventional computing architectures, then data can be stored and accessed, but data scalability is limited and real-time response latency increases
Solution Approach 1:
The system segments storage operations by creating separate hash tables for metadata and data objects at the edge processing node. This segmentation enables independent optimization of metadata access and data storage, improving real-time response speed by quickly locating data through hash-based metadata lookup without scanning entire file systems.
Solution Approach 2:
The patent introduces an edge processing node as an intermediary between sensors and remote data centers. This intermediary performs local feature detection, metadata generation, and data storage, enabling real-time processing at the edge while reducing the need for continuous cloud communication, thus lowering response latency.
2Productivity
If deep-learning networks are deployed for feature detection at edge processing nodes, then real-time feature detection is enabled, but power consumption increases
Solution Approach 1:
The system applies partial action by performing only essential feature detection at the edge processing node using deep-learning networks, rather than complete data analysis. Metadata is generated from detected features and stored locally, while full data archiving occurs remotely. This selective processing enables real-time response with reduced power consumption compared to complete local processing.
3Device complexity
If all sensor data is archived at remote data centers, then centralized storage is simplified, but computing and storage burdens on customer systems increase
Solution Approach 1:
The patent extracts essential metadata from sensor data at the edge processing node through feature detection and hashing. This extracted metadata is stored locally in hash tables, enabling quick data retrieval without burdening customer systems with complete data archiving. Only necessary metadata and selected data objects are maintained at the edge, reducing computing and storage burdens while keeping the system simple through centralized remote archiving.
4Productivity
If content-hashed object storage is implemented at edge processing nodes, then data retrieval efficiency is improved, but device complexity increases
Solution Approach 1:
The system changes the parameter of data organization by implementing hash-based indexing at the edge processing node. Data objects are stored with content-derived hash keys that enable direct retrieval without sequential searching. This parameter change from file-based to hash-based storage improves data retrieval efficiency while maintaining manageable device complexity through automated hashing and structured hash table management.
Data Source
AI summary
Features are detected from a sensor signal via a deep-learning network or other feature engineering methods in an edge processing node. Machine-learned metadata is created that describes the features, and a hash is created with the machine-learned metadata. The sensor signal is stored as a content object at the edge processing node, the object being keyed with the hash at, the edge processing node.