Distributed Data Grid State Owner Process
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computing systems face challenges in handling high demand from millions of users accessing applications simultaneously, leading to increased transaction calls that exceed the capacity of databases and metadata stores, necessitating a scalable solution for distributed data management.
Innovation Solution
A distributed data grid system with multiple buckets, each configured to contain elements of a distributed data structure or queue, and a state owner process that manages state information, allowing for efficient operations across multiple client processes without the need for constant state refreshment and providing fault tolerance through replication and locking mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is centralized in a single database or metadata store, then data consistency is maintained, but the system cannot handle high demand from millions of simultaneous users
Solution Approach 1:
The patent divides the centralized database into multiple distributed data grids across multiple servers. Each server maintains a portion of the data structure (buckets, queues, stacks) independently, allowing parallel transaction processing while maintaining overall system consistency through coordinated state management across the distributed network.
2Productivity
If data is distributed across multiple servers, then system capacity and throughput increase, but system complexity increases
Solution Approach 1:
The patent implements a universal state owner process that handles multiple data structure operations (buckets, queues, stacks) across all servers. This single process provides consistent state management, locking, and coordination functions universally across the entire distributed system, reducing complexity by consolidating control logic in one place rather than duplicating it across all servers.
Solution Approach 2:
The state owner process acts as an intermediary between client processes and the distributed data structures. It mediates all state-related operations, providing a single point of coordination that simplifies client interactions and reduces the complexity of direct peer-to-peer coordination between multiple servers.
3Measurement precision
If state information is frequently refreshed to maintain consistency, then data accuracy is maintained, but system performance and throughput decrease
Solution Approach 1:
The patent implements preliminary locking mechanisms where the state owner process acquires locks on data structures before operations are performed. This preliminary action prevents inconsistent state access without requiring continuous state refreshing, as the locks ensure that only one process can modify the state at a time, maintaining accuracy while allowing high throughput of locked operations.
Solution Approach 2:
The state owner process maintains its own internal state information and serves it to clients without requiring frequent external validation or refreshing. The process independently manages its state consistency through its locking and coordination mechanisms, eliminating the need for continuous state refreshment while maintaining accuracy.
4Reliability
If locking mechanisms are implemented to ensure data consistency, then reliability is improved, but contention increases and throughput decreases
Solution Approach 1:
The patent segments the data structure into multiple independent buckets distributed across servers. The state owner process can acquire locks on individual buckets rather than the entire data structure, allowing parallel operations on different buckets. This segmentation reduces contention by enabling concurrent access to multiple segments simultaneously while maintaining consistency within each locked segment.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and method supports a distributed data structure in a distributed data grid. The grid includes a plurality of buckets, wherein each bucket is configured with a capacity to contain a number of elements in the distributed data structure. The distributed data grid includes a state owner process configured to hold state information for the distributed data structure and provides the state information for the distributed data structure to a client process. The distributed queue can include a queue of buckets stored on a plurality of processes, wherein each bucket is configured to contain a number of elements of the distributed queue. The distributed queue can include a named queue that holds a local version of state information for the distributed queue, wherein said local version of the state information contains a head pointer and a tail pointer to the queue of buckets in the distributed data grid.