Shadowing Storage Gateway for Data Center Cost Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of managing and provisioning physical computing resources in large-scale data centers has increased, with conventional on-premise storage solutions facing high administrative and hardware costs, while virtualization technologies offer benefits but may not fully address the scalability and cost-effectiveness needed for large-scale storage management.
Innovation Solution
A storage gateway is implemented as a virtual or physical appliance that acts as an interface between a customer's data center and a remote storage service, providing a cost-effective, flexible, and scalable solution by exposing block, file, and object storage protocols, and managing data access and storage through a Web service interface, with a focus on security and administrative cost reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional on-premise storage solutions are used, then data storage capacity is sufficient, but administrative costs and hardware costs are high
Solution Approach 1:
A storage gateway is introduced as an intermediary component between the customer's data center and the remote storage service. The gateway handles protocol conversion, data management, and authentication, allowing the customer to maintain control over their data while leveraging remote storage infrastructure. This intermediary approach reduces administrative overhead by automating many storage management tasks while preserving data accessibility and control.
Solution Approach 2:
The storage system implements self-service capabilities through automated provisioning, monitoring, and management of storage resources. The storage service provider automatically handles capacity allocation, performance optimization, and maintenance tasks, eliminating the need for customer personnel to perform complex administrative functions. This self-service model significantly reduces administrative costs while maintaining sufficient storage capacity.
2Productivity
If virtualization technologies are deployed, then resource sharing efficiency is improved, but scalability and cost-effectiveness for large-scale storage management are not fully addressed
Solution Approach 1:
The storage gateway is designed with multi-functionality to handle various storage protocols (iSCSI, NFS, CIFS, FTP) and operate in different modes (cached gateway, shadowing gateway, pass-through mode). This universal design allows the same infrastructure to serve multiple customers with diverse needs, improving resource sharing efficiency while maintaining scalability through standardized interfaces and flexible configuration options.
Solution Approach 2:
The system transitions from traditional horizontal scaling by adding more physical storage devices to a new dimension of scalability through virtualized storage pools and software-defined storage architecture. This allows storage capacity to be dynamically allocated and scaled without proportionally increasing hardware infrastructure, achieving cost-effective scalability for large-scale storage management.
3Device complexity
If remote storage service is implemented, then cost of ownership is reduced, but data access performance and security requirements are affected
Solution Approach 1:
The storage system is segmented into two distinct components: a storage gateway located in the customer's data center and a remote storage service infrastructure. The gateway handles local data access operations, caching, and protocol conversion to maintain performance, while the remote infrastructure provides cost-effective storage capacity. This segmentation allows cost reduction through remote storage while preserving data access performance through local gateway operations.
Solution Approach 2:
The storage gateway creates and maintains local copies of frequently accessed data through caching mechanisms. This copying strategy allows the system to serve common read operations from the local gateway, maintaining fast data access performance, while less frequently accessed data resides in the cost-effective remote storage infrastructure. The copy mechanism reconciles the trade-off between remote storage costs and data access performance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, apparatus, and computer-accessible storage media for shadowing data stored on a local store to a remote store provided by a service provider. A gateway may be configured as a Methods, apparatus, and computer-accessible storage media for shadowing data stored on a local store to a remote store provided by a service provider. A gateway may be configured as a shadowing gateway on a customer network in response to receiving configuration information. The shadowing gateway may receive reads and writes to the local store. The gateway passes the requests to the local store, and also uploads write data indicated by the writes to the service provider to update a snapshot of the local store maintained by the service provider on the remote store. The write data may be buffered to a write log for uploading, and may be uploaded as blocks according to a block storage format used by the service provider. The shadowing process may be transparent to processes on the customer network. The shadowed data may be used to recover data on the local store.