Data Access Gateway Node Selection via Bidding Counters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data cluster systems face inefficiencies in accessing data due to the random selection of data nodes by hosts, which leads to suboptimal use of computational resources and increased latency in servicing requests.
Innovation Solution
A method and system utilizing a data access gateway (DAG) that identifies and selects data nodes with the highest bidding counters and appropriate metadata mappings to service requests, optimizing resource utilization and improving computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hosts randomly select data nodes to access data, then the system operation is simple, but the computational resource utilization is suboptimal and latency increases
Solution Approach 1:
A data access gateway is introduced as an intermediary component between hosts and data nodes. The gateway receives data access requests from hosts, obtains bidding counters from multiple data nodes, and selects the optimal data node based on the highest bidding counter. This mediator architecture enables intelligent resource allocation without requiring hosts to implement complex selection logic, thus improving computational efficiency while maintaining operational simplicity.
Solution Approach 2:
The system implements a feedback mechanism where data nodes send bidding counters to the data access gateway, which uses this information to make informed selection decisions. The bidding counter reflects the current resource availability state of each data node, creating a feedback loop that enables dynamic and adaptive data node selection based on real-time system conditions, thereby optimizing resource utilization.
2Productivity
If data nodes are selected without considering resource availability, then the access process is fast, but resource utilization is inefficient
Solution Approach 1:
The system performs preliminary actions by having data nodes send their bidding counters to the data access gateway before actual data access occurs. This advance information gathering allows the gateway to pre-calculate optimal data node selections and make informed routing decisions, ensuring that resources are allocated efficiently while minimizing access latency through proactive rather than reactive selection.
Solution Approach 2:
The system changes the selection parameter from random or fixed data node choice to dynamic selection based on bidding counter values. By using the bidding counter as the key selection parameter, the system can adaptively choose data nodes with the most available resources, simultaneously improving resource utilization efficiency and reducing access latency through parameter-driven optimization.
Data Source
AI summary
A method that is performed to access data nodes of a data cluster. The method includes obtaining, by a data access gateway (DAG), a request from a host; and in response to the request, obtaining bidding counters from the data nodes; obtaining metadata mappings from the data nodes; identifying, based on the bidding counters and metadata mappings, a data node of the data nodes associated with a highest bidding counter of the bidding counters and an appropriate metadata mapping of the metadata mappings; and sending the request to the data node.