IoT Data Storage System Metadata-Driven Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems for IoT data face high costs due to the need for high-quality data provision, which is linked to latency and query flexibility, and manual assignment of data to storage devices leads to errors and inefficiencies.
Innovation Solution
A method for automatically selecting data storage based on metadata associated with each data point, considering latency, access frequency, criticality, and lifetime, to optimize storage locations and reduce unnecessary costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all IoT data is stored in a single shared data repository with high-quality provisioning, then data latency and query flexibility are improved, but operating costs increase significantly
Solution Approach 1:
The patent segments the single shared data repository into multiple specialized storage locations (time-series database, columnar database, file system, data lake) and divides data allocation based on metadata characteristics. This allows different data types to be stored in optimized locations, reducing overall storage costs while maintaining quality provisioning where needed.
Solution Approach 2:
The patent applies local quality by assigning different storage locations to different data points based on their specific metadata characteristics (latency requirements, query patterns, data type). Each data point receives storage quality appropriate to its local needs rather than uniform high-quality storage for all data, optimizing the balance between performance and cost.
2Adaptability or versatility
If manual assignment of data to storage devices is used, then flexibility in storage selection is maintained, but errors and inaccuracies increase
Solution Approach 1:
The patent implements self-service by enabling the system to automatically allocate data to storage locations based on metadata analysis. The metadata-driven automatic allocation eliminates manual assignment errors while maintaining flexibility through rule-based decision making, allowing the system to self-optimize data placement without human intervention.
3Speed
If expensive in-memory databases are used for all IoT data, then data latency is minimized, but operating costs increase
Solution Approach 1:
The patent applies local quality by allocating only critical time-series data requiring low latency to expensive in-memory databases, while placing other data types in cheaper storage solutions. This selective allocation based on data characteristics minimizes latency for essential data while controlling overall storage costs.
Solution Approach 2:
The patent uses a hierarchy of storage solutions where expensive high-performance storage (in-memory databases) is used only when necessary for short-term critical data, while cheaper storage options (file systems, data lakes) handle less time-sensitive data. This creates a cost-effective multi-tiered storage architecture.
4Loss of energy
If NoSQL databases are used for all IoT data, then operating costs are reduced, but data query flexibility and latency performance deteriorate
Solution Approach 1:
The patent segments data storage across multiple database types (time-series, columnar, file system, data lake) rather than using a single NoSQL database for all data. This segmentation allows each data type to be stored in the most appropriate system, maintaining query flexibility for complex queries while controlling costs through selective use of database features.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method for operating a data storage system (20) that comprises a plurality of data stores (22). In said method, IoT data (14-17) generated at a specific data point (10-13) are stored in at least one data store (22), and at least one data store (22) for storing the IoT data (14-17) is selected automatically on the basis of metadata (40), wherein said metadata (40) are associated with a data point (10-13). The invention also relates to a computer program (50) involving carrying out the method, and a data storage system (20) that works according to said method.