Distributed Virtual Space Data Mapping for Scalable Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed programming environments, particularly those using distributed hash tables (DHTs), face challenges in scalability and efficiency for large-scale calculations, and lack effective data access mechanisms based on structured topologies.
Innovation Solution
A method and system that create a virtual space using resources from multiple nodes, allowing data to be mapped to specific locations based on attributes, enabling efficient data processing and access through a distributed programming environment that integrates benefits of space-based architecture and DHTs, with features like data replication and locality-based access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If DHT technology is used for large-scale calculation in distributed environment, then scalability and robustness are improved, but processing efficiency and data access speed deteriorate
Solution Approach 1:
The virtual space is divided into multiple regions, each managed by a specific node. Data is mapped to specific regions based on attributes, allowing segmented processing and access. This segmentation enables parallel processing across multiple nodes while maintaining efficient data location through region-based organization.
Solution Approach 2:
A space manager acts as an intermediary between applications and the distributed storage system. The space manager handles data mapping, region assignment, and coordination, enabling applications to access data efficiently without directly managing the complex DHT structure. This intermediary layer abstracts the complexity while maintaining scalability.
2Ease of operation
If all distributed processes perform mutual communications and resource sharing based on space, then ease of operation is improved, but network load and communication overhead increase
Solution Approach 1:
Each node is assigned specific regions of the virtual space and manages data locally within its responsibility. Applications can access data in nearby regions more efficiently, reducing the need for widespread communication across the entire network. This local quality approach maintains ease of operation through simple region-based access while reducing overall network load.
3Adaptability or versatility
If data are distributed across multiple nodes without structured topology, then scalability is improved, but data access speed and localization efficiency deteriorate
Solution Approach 1:
The patent introduces a structured topology dimension to the distributed data storage system. By organizing nodes and data regions in a structured manner (such as hierarchical or grid-based topologies), the system achieves both scalability through additional dimensions and fast data access through localized region mapping. This dimensional organization allows efficient data localization while maintaining the ability to scale.
Data Source
AI summary
Disclosed herein are a method, a system, and a computer-readable recording medium for providing distributed programming environment by using a distributed space.According to an aspect of the present invention, there is provided a method for processing data in distributed environment, the method including: generating a virtual space using resources provided by a plurality of nodes; and reading or writing data from or in the virtual space by a first application, wherein the data are mapped to a specific location region on the virtual space determined according to attributes of the data and the first application performs a reading operation or a writing operation for the data in the location region.