Autonomous Aggregation Identifier Calculation in Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed data processing systems face challenges in handling large data sets, leading to resource exhaustion and failures due to increased computing and memory demands as data grows.
Innovation Solution
The implementation of a networked system that utilizes container technology and allows constituents to autonomously compute their aggregation, eliminating the need for centralized data processing and reducing memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If centralized data processing is used to handle large data sets, then data processing capability is improved, but memory resources are exhausted and system failures occur
Solution Approach 1:
The patent segments the centralized data processing task into distributed microservices that autonomously compute their own aggregations. Each microservice processes a portion of the data independently, eliminating the need to load entire data sets into centralized memory. This segmentation resolves the contradiction by maintaining high data processing capability while reducing memory resource consumption through distributed computation.
Solution Approach 2:
The patent transitions from a single-dimensional centralized processing model to a multi-dimensional distributed architecture. By introducing spatial distribution across multiple microservices and temporal autonomy in computation, the system achieves scalable data processing without proportionally increasing memory resources. This dimensional transformation allows the system to handle large data sets efficiently.
2Quantity of substance
If data sets grow in size, then data processing capacity is improved, but computing resources are exhausted
Solution Approach 1:
The patent implements self-service computation where each microservice autonomously determines and executes its own aggregation logic without requiring centralized coordination. This self-service approach eliminates redundant computations and optimizes resource utilization, allowing the system to process growing data sets efficiently without proportionally increasing computing resource consumption.
3Measurement precision
If centralized aggregation computation is performed, then aggregation accuracy is improved, but system scalability is reduced
Solution Approach 1:
The patent merges the aggregation computation capability into each individual microservice while maintaining consistent aggregation logic across all services. This merging approach ensures that each service produces accurate aggregations independently, and the overall system maintains scalability by allowing services to be added or removed without affecting the aggregation accuracy of existing services.
Data Source
AI summary
The subject technology performs a transaction locally at a computing node. The subject technology determines that the transaction has been completed. The subject technology determines a set of immutable attributes from the completed transaction. The subject technology generates an aggregate identifier based on the set of immutable attributes. The subject technology publishes the generated aggregate ID. The subject technology stores the published aggregate ID to an external storage location.