IoT Data Collection Storage Split for Fast Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In IoT systems, the increase in data volume leads to reduced processing speed and access speed on servers, affecting display speed for users.
Innovation Solution
A data collection system with a data collection apparatus that includes a receiver, data analyzer, first data storage for raw data, and a second data storage for analyzed data, which is faster but smaller, allowing for efficient data processing and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data from all devices is stored in a single large storage system, then data volume capacity increases, but processing speed and access speed decrease
Solution Approach 1:
The storage system is segmented into multiple storage units with different characteristics (high-speed storage and large-capacity storage). Data is divided and stored across these units based on access frequency and processing requirements, allowing the system to maintain high processing speed for frequently accessed data while preserving large data volume capacity in the overall system.
2Quantity of substance
If more data is stored on the server, then data collection completeness improves, but access speed and display speed decrease
Solution Approach 1:
Different storage units are assigned different data quality characteristics based on local requirements. High-speed storage units hold frequently accessed data with optimized access properties, while large-capacity storage units hold less frequently accessed data. This local optimization allows the system to maintain high access speed for critical data while collecting and storing complete data sets across the entire system.
Data Source
AI summary
A data collection system including a device having a communication function and a data collection apparatus configured to collect data transmitted from the device. The data collection apparatus includes a receiver configured to receive data transmitted from the device, a data analyzer configured to acquire analyzed data by analyzing the received data according to analysis data format different for each device, a first data storage which is a non-rewritable storage, a second data storage which is storage that can be accessed faster than the first data storage and that stores a smaller amount of data than the first data storage, a data writer configured to writes and stores the data received by the receiver to the first data storage, and writes and stores the analyzed data acquired by the data analyzer to the second data storage.