Server-Side Virtual Array Storage for Flash Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems for virtualized environments are inefficient due to legacy characteristics, requiring elaborate data structures and proprietary protocols, and suffer from access latency issues, especially when using flash devices, which hinders their scalability and cost-effectiveness.
Innovation Solution
A distributed virtual array data storage system where storage nodes with inexpensive disks and processors communicate directly with servers over a network, managed by server-side software that employs a standard protocol, eliminating the need for dedicated backplanes and fibre channel connections, and optimizes flash memory usage for enhanced performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If flash devices are used for caching data to increase I/O operations per second, then productivity is improved, but access latency increases due to network and fibre channel protocols
Solution Approach 1:
The patent extracts the storage control functionality from dedicated storage controllers and relocates it to server-side virtual machine instances. This eliminates the need for fibre channel protocols and dedicated backplanes, allowing flash devices to be accessed directly over standard network protocols, thereby reducing access latency while maintaining high I/O throughput
Solution Approach 2:
The patent introduces a virtual machine instance as an intermediary between the server and flash storage devices. This virtual machine runs storage management software that handles data operations, eliminating the need for proprietary storage protocols and reducing communication overhead, thus decreasing access latency while preserving productivity
2Reliability
If dedicated storage controllers and fibre channel connections are used, then reliability is improved, but device complexity increases due to dedicated backplanes and proprietary protocols
Solution Approach 1:
The patent makes standard network infrastructure universal by enabling it to perform the functions previously reserved for dedicated fibre channel storage networks. The same network infrastructure used for general data communication is now used for storage operations, eliminating the need for separate dedicated controllers and backplanes, thus reducing complexity while maintaining reliability through proven network technology
Solution Approach 2:
The patent inverts the traditional storage architecture by moving the intelligence from the storage side (dedicated controllers) to the compute side (server virtual machines). Instead of having specialized storage controllers manage flash devices, standard server instances with standard network interfaces manage storage operations, simplifying the overall system architecture while maintaining reliability
3Adaptability or versatility
If elaborate data storage structures (LUNs, Volumes) are defined, then adaptability is improved, but device complexity and ease of operation worsen due to software layers translating protocols
Solution Approach 1:
The patent enables the virtual machine instances to self-manage storage operations without requiring complex external software layers. The virtual machines running on servers directly manage the flash storage devices using standard network protocols, eliminating the need for proprietary translation software and complex data structure definitions, thus reducing complexity while maintaining adaptability
Solution Approach 2:
The patent extracts the protocol translation functionality from separate software layers and integrates it directly into the virtual machine instances. This eliminates the need for elaborate data storage structures and proprietary protocol translations, simplifying both the data structure definitions and operational complexity while preserving system adaptability
4Device complexity
If standard network connections are used instead of fibre channel, then device complexity is reduced, but speed worsens due to additional access latency
Solution Approach 1:
The patent implements preliminary action by having virtual machine instances pre-cache frequently accessed data in memory before it is needed. This anticipatory caching reduces the frequency of slow network accesses to flash storage, compensating for the slower speed of standard network connections compared to fibre channel while maintaining reduced device complexity
Solution Approach 2:
The patent ensures continuous useful action by implementing data caching and prefetching mechanisms in the virtual machine instances. Frequently accessed data is kept in memory, and additional data is pre-loaded before it is needed, ensuring that the system maintains high effective data access speed despite using slower standard network connections, thus offsetting the speed penalty while keeping complexity low
Data Source
AI summary
Embodiments of a distributed virtual array data storage system and method are disclosed. Storage nodes made up of relatively unsophisticated disks with associated processors are scalable to store very large amounts of data. The storage nodes communicate with servers directly over a network through, for example, an Ethernet connection. Control of the storage nodes and access to the storage nodes is handled entirely on the server side of the system by distributed virtual array (DVA) software running on the server side and employing a particular protocol over the standard network connection. In an embodiment, server-side virtual machine (VM) hosts host application VMs that are associated with vDisks. The DVA software distributes data for the vDisk over the storage nodes. In the case of failure of one or more of the storage nodes, the DVA software reconstructs the data, for example by reading redundant data from surviving storage nodes.