Distributed Lookup Storage System with Local Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for high-capacity, high-performance storage solutions in cloud storage environments leads to significant energy consumption and costs due to the large amount of data being transferred between data centers and users, necessitating a reduction in data transfer volumes.
Innovation Solution
A distributed lookup system utilizing a storage chassis with a query storage device, candidate storage devices, an Ethernet switch, and a Baseboard Management Controller (BMC) that stores metadata, allowing for efficient identification and retrieval of objects conforming to queries without extensive data transmission, by using Ethernet and PCIe switches for communication and metadata management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is transferred between data centers and users to provide cloud storage services, then storage accessibility and service capability are improved, but energy consumption and costs increase
Solution Approach 1:
The patent extracts metadata from the main data storage system and places it in dedicated query storage devices within the storage chassis. This separation allows query operations to be performed locally on metadata without transferring actual data objects, thereby maintaining storage accessibility while significantly reducing energy-consuming data transfers between data centers and users.
Solution Approach 2:
The patent introduces a new architectural dimension by implementing distributed query storage devices within storage chassis that operate independently from main data storage locations. This dimensional separation enables local metadata queries to resolve many access requests without involving long-distance data transfers, thus improving accessibility while reducing energy consumption.
2Device complexity
If metadata is stored in centralized location for object storage queries, then query management is simplified, but query response time and system scalability deteriorate
Solution Approach 1:
The patent segments the centralized metadata storage into distributed query storage devices within individual storage chassis. Each query storage device maintains local metadata for objects stored in its associated candidate storage devices, enabling parallel query processing across multiple chassis simultaneously. This segmentation reduces query response time while maintaining manageable complexity through standardized interfaces.
Solution Approach 2:
The patent implements local metadata storage in query storage devices within each storage chassis, tailored to the specific candidate storage devices in that chassis. This local quality optimization allows each chassis to independently and quickly resolve queries for its local objects without waiting for centralized metadata retrieval, significantly improving query response time.
3Quantity of substance
If data is stored in distributed storage devices for high capacity, then storage scalability is improved, but data transfer volume and energy consumption increase
Solution Approach 1:
The patent introduces query storage devices as intermediaries between users and distributed candidate storage devices. These intermediaries maintain local metadata that enables direct client-to-storage-device queries without requiring data transfer between distributed storage locations, thus supporting high storage capacity while minimizing energy-consuming data transfers.
Solution Approach 2:
The patent performs preliminary action by pre-storing metadata in query storage devices within each storage chassis. This preliminary placement of metadata information enables future query operations to be resolved locally without triggering energy-intensive data transfers across the distributed storage network, thereby supporting scalability while reducing energy loss.
Data Source
AI summary
A storage system includes a plurality of storage devices in a storage chassis and configured to store objects, an Ethernet switch in the storage chassis, a Baseboard Management Controller (BMC) connected to the storage devices, and a memory connected to the BMC. The plurality of storage devices include a query storage device and a candidate storage device, and the storage devices are connected to each other via the Ethernet switch in the storage chassis. The memory or the query storage devices is configured to store metadata corresponding to the objects stored in the candidate storage device.