Hybrid Cloud Storage Block Caching and Protocol Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud storage solutions do not effectively integrate with network-attached storage (NAS) devices, leading to suboptimal data storage and retrieval performance, as they lack standard file sharing protocols and result in slower access times compared to local storage.
Innovation Solution
A hybrid cloud storage system that divides data files into blocks, assigns block codes, and uses a contract with cloud storage services to optimize data transfer by caching frequently accessed blocks and synchronizing data between local storage and cloud services, enabling efficient data retrieval and storage through a cloud connector that manages block codes and caching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in cloud storage services, then data is stored in multiple locations and protected against local disasters, but accessing files is slower than accessing local storage devices
Solution Approach 1:
The system divides files into multiple blocks and stores them across different locations including local cache and cloud storage. This segmentation allows the system to retrieve only necessary blocks rather than entire files, improving access speed while maintaining data protection through distributed storage.
Solution Approach 2:
The system pre-loads and caches frequently accessed data blocks in local storage before they are actually needed. This preliminary action reduces access latency by having data ready locally, while still maintaining the benefit of cloud-based data protection and redundancy.
2Quantity of substance
If cloud storage services are used, then mass storage is provided through web services, but they do not implement standard file sharing protocols and require special client software
Solution Approach 1:
The system implements multiple file sharing protocols including CIFS, NFS, and WebDAV within a unified architecture. This allows the cloud storage service to work with standard protocols while maintaining mass storage capabilities, eliminating the need for special client software and improving compatibility across different platforms and applications.
3Reliability
If data is synchronized remotely using hosted storage services, then data is stored in multiple locations, but installation of client software on each PC is required which increases deployment costs
Solution Approach 1:
The system introduces a cloud connector as an intermediary component that can be deployed once on a server rather than on every client PC. This connector handles the synchronization and protocol translation, reducing deployment complexity while maintaining data redundancy across multiple locations through centralized management.
Data Source
AI summary
A method and system for an optimized transfer of data files to a cloud storage service (CSS) are presented. The method comprises dividing a data file into a plurality of data blocks; assigning a block code to each of the plurality of data blocks; generating, based on a contract with the CSS, a first list of block codes from the plurality of data blocks, wherein the contract defines at least data blocks guaranteed to exist in the CSS; querying the CSS with the first list of block codes; responsive of the query, receiving a second list of block codes from the CSS, wherein the second list of block codes includes block codes of data blocks designated in the first list of block codes but missing in the CSS; and transmitting to the CSS data blocks designated by their block codes in the second list.