Grid Data Partition Mapping for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database-intensive applications face challenges in grid computing environments due to high latency and increased execution times when accessing central databases, which diminish the benefits of distributed computing resources.
Innovation Solution
A system that includes a partition analyzer and a provisioning manager to identify data partitions and map them to tasks, determining optimal nodes for processing and data storage based on latency, allowing for the relocation of data partitions to reduce execution time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If database-intensive applications access the central database in a grid computing environment, then the distributed computing resources are utilized, but the latency and execution time increase significantly
Solution Approach 1:
The patent segments the central database into multiple data partitions distributed across different nodes in the grid network. Each partition is mapped to specific tasks, allowing data to be stored closer to where it is needed, thereby reducing latency and execution time while maintaining the benefits of distributed computing resources.
Solution Approach 2:
The patent introduces a data partition mapping mechanism that acts as an intermediary between tasks and the central database. This mapping system determines optimal data partitions for each task based on data access patterns and task requirements, enabling efficient data retrieval without direct access to the central database, thus reducing latency.
2Productivity
If data partitions are relocated to data nodes closer to processing nodes, then execution time is reduced, but system complexity increases
Solution Approach 1:
The patent performs preliminary actions by pre-mapping data partitions to tasks and pre-positioning data at appropriate nodes before task execution begins. The provisioning manager determines optimal data partitions and locations in advance, so when tasks execute, the data is already available at nearby data nodes, reducing execution time without adding runtime complexity.
Solution Approach 2:
The system employs self-service mechanisms where the partition analyzer and provisioning manager automatically determine optimal data partitions and mappings based on task requirements and data access patterns. This automated approach reduces the need for manual intervention and simplifies system management while achieving faster execution times through intelligent data placement.
Data Source
AI summary
A partition analyzer may be configured to designate a data partition within a database of a grid network, and to perform a mapping of the data partition to a task of an application, the application to be at least partially executed within the grid network. A provisioning manager may be configured to determine a task instance of the task, and to determine the data partition, based on the mapping, where the data partition may be stored at an initial node of the grid network. A processing node of the grid network having processing resources required to execute the task instance and a data node of the grid network having memory resources required to store the data partition may be determined. The task instance may be deployed to the processing node, and the data partition may be re-located from the initial node to the data node, based on the comparison.