Composite Object Storage for Direct Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual computing systems face inefficiencies due to separate constructs for compute and object storage, leading to unnecessary network hops, latency, and complexity when processing data, particularly in hyper-converged infrastructure (HCI) solutions where data is aggregated before being processed, resulting in increased latency and resource management challenges.
Innovation Solution
The system specifies objects as executable and constructs composite objects that associate data and processing environments, allowing these composite objects to run automatically on available resources, thereby eliminating the need for separate compute and storage management and reducing network hops and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is aggregated before processing in separate compute and storage constructs, then data processing can be performed, but network hops and latency increase
Solution Approach 1:
The patent combines separate compute and storage constructs into a unified object storage system where executable code, environment specifications, and data are integrated into single composite objects. This merging eliminates the need for separate data aggregation and computation steps, allowing processing to occur directly within the storage system and significantly reducing network hops and latency.
Solution Approach 2:
The patent introduces composite objects as intermediaries that encapsulate both data and the executable logic needed to process it. These composite objects act as self-contained units that can be directly executed without requiring separate compute resources, thereby eliminating the intermediary steps of data retrieval, aggregation, and separate processing that cause latency.
2Ease of manufacture
If separate compute and object storage constructs are used, then resource management is modular, but system complexity increases
Solution Approach 1:
The patent creates a universal object storage system that can both store data and execute processing logic within the same construct. The composite objects serve multiple functions - they are simultaneously data containers, executable programs, and environment specifications. This multi-functionality reduces system complexity by eliminating the need for separate compute and storage management while maintaining the benefits of modular resource allocation.
3Productivity
If data aggregation is performed before processing, then batch processing is enabled, but resource management becomes challenging
Solution Approach 1:
The patent segments processing workloads into independent composite objects that can be executed individually or in batches. Each composite object is a self-contained unit with its own data, executable code, and environment specifications. This segmentation enables flexible resource management where individual composite objects can be scheduled and executed independently, while still allowing batch processing when multiple composite objects need to run together, thereby simplifying resource allocation and management.
Data Source
AI summary
An illustrated embodiment disclosed herein relates to a hyper converged infrastructure (HCI) having an object storage system (OSS) with an apparatus having a processor with programmed instructions that when executed cause the processor to perform operations. The operations include specifying a first object in the OSS as an executable, specifying a second object in the OSS as an environment on which to run the first object, specifying a third object in the OSS as data to be processed by the first object, and constructing and storing a composite object in the OSS that associates the first, second, and third objects. The composite object provides a technical improvement, providing more direct access to compute resources.