Distributed Data Management System Using Digital Hash Map
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems in multicomputer environments face challenges in achieving both scalability and high performance concurrently, often requiring transactional scope management and distributed locking, which lead to performance bottlenecks and limitations in concurrent data access.
Innovation Solution
A distributed data management system using a digital hash map data structure that enables concurrent, non-blocking access by storing equivalent representations of complex data types as compressed byte arrays, reducing resource storage footprint and speeding data transfer, and employing caching and synchronization to minimize node-to-node data transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If distributed DBMS solutions use transaction management and distributed locking to guarantee data correctness, then data reliability is improved, but concurrent data access performance deteriorates due to expensive transaction overhead and locking
Solution Approach 1:
The patent extracts and removes the transaction management and distributed locking mechanisms from the system. By taking out these heavyweight concurrency control components, the system achieves high concurrent data access performance while maintaining reliability through alternative means (digital hash map with non-blocking access patterns).
Solution Approach 2:
The patent replaces the mechanical locking and transaction management system with a digital hash map data structure that enables concurrent non-blocking access. This substitution eliminates the need for traditional concurrency control mechanisms while maintaining data correctness through the properties of the hash map implementation.
2Adaptability or versatility
If prior SDDS solutions load balance data uniformly across multiple computer nodes, then scalability is improved, but access performance deteriorates because concurrent requests block access to data on particular nodes
Solution Approach 1:
The patent segments data access by ensuring that each node manages only a subset of data (through uniform load balancing), and by making each segment independently accessible. This segmentation allows parallel access to different data segments without blocking, achieving both scalability and high access performance.
Solution Approach 2:
The patent implements dynamic scalability where nodes can join and leave the system ad-hoc without service interruption. The digital hash map structure dynamically adapts to changing node configurations while maintaining uniform data distribution and non-blocking access patterns, resolving the contradiction between scalability and performance.
3Ease of operation
If SDDS solutions virtualize data access by resolving client requests from server nodes, then data accessibility is improved, but network bandwidth consumption increases due to repeated large object transfers
Solution Approach 1:
The patent uses compressed byte array representations as efficient copies of complex data types. These compressed representations are stored in the digital hash map and transferred only when needed, reducing network bandwidth consumption compared to transferring full large objects. The compressed format maintains data accessibility while minimizing network transfer costs.
Solution Approach 2:
The patent changes the representation parameter of data from full object format to compressed byte array format. This parameter change reduces the size of data transferred over the network while maintaining accessibility. The compression transformation occurs during storage and retrieval operations, significantly reducing network bandwidth consumption.
Data Source
AI summary
A distributed data management system has multiple virtual machine nodes operating on multiple computers that are in communication with each other over a computer network. Each virtual machine node includes at least one data store or “bucket” for receiving data. A digital hash map data structure is stored in a computer readable medium of at least one of the multiple computers to configure the multiple virtual machine nodes and buckets to provide concurrent, non-blocking access to data in the buckets, the digital hash map data structure including a mapping between the virtual machine nodes and the buckets. The distributed data management system employing dynamic scalability in which one or more buckets from a virtual machine node reaching a memory capacity threshold are transferred to another virtual machine node that is below its memory capacity threshold.