Edge Cloud Data Synchronization via Classification and Queuing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud computing systems, synchronizing and reconciling large file systems between local and cloud storage nodes is inefficient, leading to prolonged downtime and outdated cloud file systems, which is problematic for immediate access and collaboration.
Innovation Solution
A processor-implemented method that classifies data from edge nodes, queues it with timestamps and metadata, determines a frequency count based on data quantity and application type, and synchronizes it with cloud nodes using a synchronization table to replicate data without duplication, ensuring real-time updates and fault tolerance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is synchronized from edge nodes to cloud nodes using traditional copying methods, then data backup is achieved, but synchronization time becomes very long (days to weeks) and file systems become unavailable during this time
Solution Approach 1:
The system performs preliminary actions by maintaining continuous real-time data streams from edge nodes to cloud nodes before any synchronization is needed. Data is pre-processed, classified, and staged in cloud databases continuously, so when synchronization is required, the data is already prepared and immediately available, eliminating the need for lengthy copy operations.
Solution Approach 2:
The patent introduces cloud databases as intermediary storage between edge nodes and the final cloud file system. Data flows from edge nodes through cloud databases (which continuously receive and buffer data streams) before being synchronized to the cloud file system. This intermediary layer decouples the continuous data generation at edge nodes from the periodic synchronization operations, enabling real-time availability without prolonged downtime.
2Measurement precision
If continuous synchronization is performed to keep cloud file systems up to date, then data accuracy is improved, but system resources are consumed and file system availability is reduced
Solution Approach 1:
The system uses periodic action by implementing scheduled synchronization operations that occur at predetermined intervals rather than continuously. Cloud databases continuously accumulate and buffer data streams from edge nodes between synchronization cycles. When synchronization is triggered, the system transfers accumulated data in batches, achieving data accuracy updates without requiring constant system availability or continuous resource consumption.
Solution Approach 2:
The patent implements dynamics by making the synchronization system adaptive and flexible. The synchronization trigger can be event-driven (based on data changes, time intervals, or manual initiation) rather than rigidly continuous. The system dynamically adjusts between maintaining continuous data streams in cloud databases and performing periodic synchronization operations, optimizing the balance between data accuracy and system availability based on actual needs.
3Productivity
If data is classified and routed to different service classes with different synchronization priorities, then high-priority data is synchronized first, but system complexity increases
Solution Approach 1:
The patent applies local quality by implementing service class differentiation within the data stream processing architecture. Data from edge nodes is classified into different service classes (e.g., real-time, near-real-time, batch) with different synchronization priorities. Each service class is routed to appropriate cloud database instances or synchronization queues, allowing high-priority data to be synchronized first while low-priority data is buffered. This localized quality differentiation enables efficient priority-based synchronization without requiring complete system redesign.
Data Source
AI summary
A processor-implemented method for synchronizing and reconciling data stream in real-time between an edge node and a cloud node is provided. The method includes (i) generating classified data by classifying data of the data stream that is obtained from a plurality of edge nodes in real-time to synchronize in a location at a plurality of cloud nodes, (ii) synchronizing the classified data in the plurality of cloud nodes by detecting a first location and a type of database in the plurality of cloud nodes and storing the classified data in the first location in the plurality of cloud, (iii) replicating the classified data in a plurality of second locations in the plurality of cloud nodes, and (iv) receiving, from the plurality of edge nodes, a request signal for synchronizing the classified data from the plurality of cloud nodes to the plurality of edge nodes when the first location in the plurality of cloud nodes fails to transfer the classified data during a transaction.