Cache Memory Association with Work Process for Distributed I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data-intensive distributed applications, existing technologies face challenges in efficiently processing large data sets across clusters of computers, as they often require data to be loaded and copied between nodes, leading to inefficiencies in input/output operations.
Innovation Solution
The solution involves associating cache memory with work processes, where a kernel or support process monitors file mapping attempts and allocates shared cache service memory directly to Java virtual machines or other work processes, allowing them to access data without the need for loading and copying, thereby enhancing input/output operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is loaded and copied between nodes in distributed applications, then data processing can be performed across clusters, but input/output operations become inefficient and processing speed decreases
Solution Approach 1:
The patent merges the cache memory with the virtual machine memory space by establishing a shared memory region that is directly accessible by both the cache service process and the work process. This merging eliminates the need for data copying between separate memory spaces, allowing direct access to cached data and significantly improving I/O operation speed while maintaining high data processing efficiency across the distributed cluster.
2Ease of operation
If separate memory spaces are used for cache service and work process, then memory management is simplified, but data access requires loading and copying operations
Solution Approach 1:
The patent implements a nested memory structure where the cache memory is nested within the virtual machine's address space. The cache service process and work process share a common memory region through file mapping, creating a nested hierarchy where cached data is directly accessible within the work process memory space without requiring data transfer. This eliminates data transfer time while maintaining manageable memory allocation through the support process.
3Adaptability or versatility
If data is copied between nodes, then distributed processing can be achieved, but the complexity of data transfer and synchronization increases
Solution Approach 1:
The patent extracts the data copying operation from the distributed processing workflow by implementing direct shared memory access. Instead of copying data between nodes, the cache service and work process directly access the same memory region through file mapping. This extraction eliminates the complexity of data transfer and synchronization operations while preserving the distributed processing capability across cluster nodes.
Data Source
AI summary
Systems, methods, and software described herein provide accelerated input and output of data in a work process. In one example, a method of operating a support process within a computing system for providing accelerated input and output for a work process includes monitoring for a file mapping attempt initiated by the work process. The method further includes, in response to the file mapping attempt, identifying a first region in memory already allocated to a cache service, and associating the first region in memory with the work process.