Edge Snapshot Differencing for Core Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computing systems connected over a network, transmitting all data from edge source systems to core target systems for analysis is inefficient, especially with the increasing data from IoT devices, as target systems are only interested in subsets of the collected data.
Innovation Solution
A method where a source system generates snapshots of collected data with associated time references, determining a subset by finding the most closely spaced start and end snapshots inclusive of a requested time interval, and providing the difference between these snapshots to the target system, utilizing data virtualization platforms for efficient data storage and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all data is transmitted from edge source systems to core target systems, then data completeness is improved, but network load and transmission efficiency deteriorate
Solution Approach 1:
The patent extracts only the necessary subset of data from the complete dataset at the edge system. By identifying and transmitting only the required data portions to the core system, it eliminates the waste of transmitting unnecessary data, thereby reducing network load while maintaining data completeness for analysis purposes
Solution Approach 2:
The patent segments the complete dataset into relevant and irrelevant portions. By dividing the data transmission into selective segments based on analysis requirements, it enables efficient transmission of only necessary data portions, resolving the contradiction between data completeness and network efficiency
2Loss of information
If all data is transmitted from edge source systems to core target systems, then data availability for analysis is improved, but storage requirements at target system deteriorate
Solution Approach 1:
The patent extracts only the essential data subset required for analysis from the complete dataset. By transmitting only this extracted portion to the core system, it ensures data availability for analysis while minimizing the storage burden at the target system
Solution Approach 2:
The patent segments the data transmission to include only analysis-relevant portions. This segmentation approach ensures that the core system receives sufficient data for analysis purposes without accumulating unnecessary storage overhead from redundant or irrelevant data
3Adaptability or versatility
If snapshots are generated and stored for all data, then data retrieval flexibility is improved, but storage complexity and resource consumption deteriorate
Solution Approach 1:
The patent extracts only the necessary snapshot data portions that are relevant to analysis requirements. By selecting and storing only essential snapshot information rather than complete data sets, it maintains retrieval flexibility for analysis while reducing storage complexity and resource consumption
Solution Approach 2:
The patent segments the snapshot storage into manageable, relevant portions. By organizing snapshots in a segmented manner that focuses on analysis-critical data, it enables flexible data retrieval without requiring complex storage infrastructure to handle complete dataset snapshots
Data Source
AI summary
A source system generates snapshots of collected data. The snapshots have respective associated time references. Responsive to a request from a target system for data collected over a time interval, the source system generates a subset of the data collected by determining a start snapshot and an end snapshot. The start snapshot and the end snapshot are determined as a pair of snapshots that have respective associated time references that are most closely spaced and are inclusive of the time interval. The source system determines a difference in the data included in the end snapshot and the start snapshot and provides the subset of the data as the difference in the data included in the end snapshot and the start snapshot.