Aggregation Computing Node for Low-Latency Distributed Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed data computing systems, data access delays occur due to the need for computing nodes to exchange data before writing into a storage node, leading to inefficient data writing and increased latency.
Innovation Solution
A computing node cluster with an aggregation computing node that aggregates partial data from multiple nodes into consecutive addresses, allowing each node to return a write success and perform other tasks while data is being aggregated, decoupling IO from data aggregation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is exchanged between computing nodes to achieve consecutive addresses before writing to storage node, then data writing efficiency is improved, but data access delay increases
Solution Approach 1:
The patent segments the data writing process into two independent phases: (1) data processing and local caching phase, where computing nodes process data and store it in local caches without waiting for address consecutiveness; and (2) data aggregation phase, where the aggregation computing node collects data from all nodes and reorganizes it into consecutive addresses before writing to storage. This segmentation allows computing nodes to complete their processing tasks independently without blocking for address consecutiveness, thereby reducing data access delay while maintaining writing efficiency.
Solution Approach 2:
The patent applies preliminary action by having computing nodes perform data processing and caching before the actual data aggregation and writing operation. The local caches store data temporarily while the aggregation computing node prepares to collect and reorganize the data. This preliminary processing allows nodes to complete their work early without waiting for the entire data aggregation process, reducing the time loss associated with sequential address requirements.
2Reliability
If computing nodes wait for data aggregation completion before returning write success, then data consistency is ensured, but CPU resource utilization decreases
Solution Approach 1:
The patent segments the write operation into two independent parts: (1) local data processing and caching, which can be completed independently and return write success immediately; and (2) data aggregation and consolidation, which is performed asynchronously by the aggregation computing node. This segmentation allows computing nodes to return write success as soon as their local processing is complete, ensuring data consistency through the aggregation process while maximizing CPU resource utilization by eliminating unnecessary waiting time.
Solution Approach 2:
The aggregation computing node acts as an intermediary between the computing nodes and the storage node. It receives data from multiple computing nodes, ensures data consistency by collecting all necessary data blocks, reorganizes them into consecutive addresses, and then writes to the storage node. This intermediary role allows computing nodes to complete their processing and return success independently, while the aggregation node handles the consistency and coordination tasks separately.
3Manufacturing precision
If sequential IO and data aggregation operations are performed, then data correctness is maintained, but data processing efficiency decreases
Solution Approach 1:
The patent segments the data processing workflow into independent computing node operations and a separate aggregation operation. Computing nodes perform their processing and caching operations in parallel without waiting for aggregation, while the aggregation computing node collects and validates data from all nodes. This segmentation enables parallel processing that maintains data correctness through coordinated aggregation while significantly improving overall processing efficiency.
Solution Approach 2:
The patent ensures continuity of useful action by allowing computing nodes to continuously process and cache data without interruption, while the aggregation computing node continuously collects and aggregates data from all nodes. Both operations proceed simultaneously and independently, with the aggregation process ensuring data correctness by validating all data blocks before final writing. This continuous parallel operation maximizes processing efficiency while maintaining data integrity.
Data Source
AI summary
A computing node cluster provided in this embodiment of this application includes a plurality of computing nodes, and the plurality of computing nodes include an aggregation computing node. The plurality of computing nodes are configured to jointly perform a write operation on to-be-written data. Each of the plurality of computing nodes is configured to return a write success after writing partial data in the to-be-written data to a local cache. The aggregation computing node is configured to aggregate the partial data stored in caches of the plurality of computing nodes into aggregated data with consecutive addresses, and write the aggregated data into a storage node.