Object Storage I/O Path Code Execution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data storage systems face challenges in enforcing data manipulation functions directly within the storage system and require complex external services to manage data manipulations, which can be insecure and inefficient.

Innovation Solution

The integration of an on-demand code execution system with object storage services allows for the insertion of serverless tasks into the I/O path, enabling data manipulations through input and output streams without requiring network communication, thus simplifying code execution and enhancing security.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data manipulation functions are enforced within the storage system using external services, then data processing capability is improved, but system complexity and security risks increase

Engineering Contradiction:

Improvedata processing capabilityVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent merges the code execution environment directly into the storage system by integrating a runtime environment that can execute user-submitted code within the object storage service. This consolidation eliminates the need for separate external services while maintaining data manipulation capabilities, thereby improving productivity without increasing overall system complexity.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The storage system is designed to perform multiple functions: traditional object storage operations plus execution of user-submitted code for data manipulation. By making the storage system universal and capable of both data storage and data processing, the patent eliminates the need for separate specialized services, reducing system complexity while maintaining enhanced data processing capability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If external services are used for data manipulation, then processing flexibility is improved, but security risks increase

Engineering Contradiction:

Improveprocessing flexibilityVSAvoidsecurity

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent introduces a sandboxed runtime environment as an intermediary between the storage system and user-submitted code. This intermediary provides the flexibility to execute diverse user code while maintaining security by isolating the code execution from the core storage system, thus preventing unauthorized access or system compromise.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent creates a controlled, isolated execution environment (sandbox) where user-submitted code runs in an inert atmosphere that prevents it from affecting the host system. This approach maintains processing flexibility by allowing any user code to execute while ensuring security through environmental isolation, similar to how an inert atmosphere prevents unwanted chemical reactions.

Inventive Principle:

Principle #39Inert atmosphere (Inert environment)

3Adaptability or versatility

If complex external services manage data manipulations, then functionality is improved, but ease of operation deteriorates

Engineering Contradiction:

Improvedata manipulation functionalityVSAvoidoperational simplicity

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

By merging code execution capabilities directly into the storage system, the patent allows users to perform data manipulations using simple storage operations. The complex functionality of executing user code is combined with basic storage operations, making the system easier to operate while maintaining advanced data manipulation capabilities.

Inventive Principle:

Principle #5Merging (Combining)

4Productivity

If user-submitted code is executed during I/O operations, then data processing efficiency is improved, but system resource consumption increases

Engineering Contradiction:

Improvedata processing efficiencyVSAvoidsystem resource consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent enables continuous data processing by executing user-submitted code during I/O operations without requiring separate processing steps. The code execution is integrated into the data flow, allowing processing to continue seamlessly while data moves through the storage system, improving efficiency without proportionally increasing resource consumption.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS11656892B1Sequential execution of user-submitted code and native functions

Publication Date: 2023.05.23 AMAZON TECH INC

AI summary

Systems and methods are described for modifying input and output (I/O) to an object storage service by implementing any combination of and any number of owner-specified functions and native functions. A function can implement a data manipulation. The functions can be applied prior to implementing a request method (e.g., GET, PUT, LIST, etc.) specified within the I/O request, such that the data to which the method is applied may not match the object specified within the request. For example, a user may request to obtain a data set. The data set may be passed to a native function that filters sensitive data to the data set, the output of the native function may be passed to an owner-specified function that redacts data from the filtered data set, and the request method may then be applied to the output of the owner-specified function.