Cache Service Protocol Translation for Virtual Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Configuring host systems in a virtual data-processing environment for processing jobs is complex due to variations in storage systems' capacities, object access protocols, and access credentials, leading to potential delays or improper processing.
Innovation Solution
A cache service and configuration management system that translate requests between virtual machines and storage systems using different object access protocols, allowing virtual machines to access data using a single protocol while handling multiple protocols used by storage systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple storage systems with different object access protocols are used to handle large data sets, then data processing capability is improved, but configuration complexity increases
Solution Approach 1:
The patent introduces a gateway as an intermediary component that sits between the processing cluster and multiple storage systems. This gateway handles protocol translation and authentication, allowing the processing cluster to access diverse storage systems without direct configuration of multiple protocols. The gateway mediates all interactions, simplifying the overall system configuration while maintaining access to multiple storage systems with different protocols.
2Productivity
If a single processing job runs on multiple host systems and virtual machines, then processing throughput is improved, but configuration precision requirements increase
Solution Approach 1:
The gateway is designed as a universal component that handles multiple functions: protocol translation, authentication, authorization, and data access management. By consolidating these functions in a single gateway, the system can run processing jobs across multiple host systems and virtual machines without requiring precise configuration of each individual component. The gateway provides a unified interface that works consistently across the distributed environment.
3Adaptability or versatility
If storage systems with different capacities and protocols are integrated, then data access versatility is improved, but access time increases
Solution Approach 1:
The gateway performs preliminary actions by pre-establishing connections to multiple storage systems and pre-configuring protocol translations. Authentication and authorization are handled in advance, creating cached credentials and access policies. When processing jobs need to access data from diverse storage systems, the gateway can quickly retrieve pre-configured access information rather than establishing connections and handling authentication in real-time, thus reducing access time while maintaining versatility.
Data Source
AI summary
Described herein are systems, methods, and software for translating data requests in a data processing cluster. In one example, a method of operating a cache service to interface between a virtual machine cluster and job data associated with a job executed by the virtual machine cluster includes identifying a request initiated by the virtual machine cluster to access at least a portion of the job data in accordance with a first distributed object access protocol. The method further includes in response to the request, accessing at least the portion of the job data in accordance with a second distributed object access protocol, and presenting at least the portion of the job data to the virtual machine cluster in accordance with the first distributed object access protocol.