On-Demand Code Execution in Object Storage I/O Path
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face challenges in enforcing data manipulation functions and ensuring data privacy, as they often rely on external enforcement and complex network communication requirements, which can lead to security issues and inefficiencies.
Innovation Solution
The integration of an on-demand code execution system with object storage services allows for the insertion of data processing pipelines into the I/O path, enabling serverless task executions that manipulate data using input and output handles, thereby simplifying code execution and enhancing security by restricting network communications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data manipulation functions are enforced externally with complex network communication, then data privacy can be ensured, but security issues and system complexity increase
Solution Approach 1:
The patent extracts the data manipulation function from external network communication and embeds it directly into the object storage service's I/O path. By taking out the complex network enforcement mechanism and integrating a simplified code execution environment within the storage service, the system maintains data privacy enforcement while reducing overall system complexity and security vulnerabilities associated with external network communication.
2Productivity
If code execution is integrated into object storage I/O path, then data manipulation efficiency improves, but security risks may increase
Solution Approach 1:
The patent introduces an intermediary code execution environment that acts as a mediator between the object storage service and user-provided code. This intermediary layer allows efficient data manipulation by executing code within the I/O path, while simultaneously mitigating security risks by controlling and isolating the code execution environment, preventing direct access to sensitive storage resources.
3Adaptability or versatility
If user-specified code is executed during I/O operations, then data policy enforcement capability improves, but execution latency may increase
Solution Approach 1:
The patent applies preliminary action by preparing and validating user-specified code in advance before it is executed during I/O operations. The system pre-compiles, pre-validates, and pre-configures the code execution environment, so that when data I/O operations occur, the code can be executed immediately with minimal latency. This maintains high adaptability for data policy enforcement while reducing execution time.
Data Source
AI summary
Systems and methods are described for modifying input and output (I/O) to an object storage service by implementing one or more owner-specified functions to I/O requests. A function can implement a data manipulation, such as filtering out sensitive data before reading or writing the data. The functions can be applied prior to implementing a request method (e.g., GET or PUT) specified within the I/O request, such that the data to which the method is applied my not match the object specified within the request. For example, a user may request to obtain (e.g., GET) a data set. The data set may be passed to a function that filters sensitive data to the data set, and the GET request method may then be applied to the output of the function. In this manner, owners of objects on an object storage service are provided with greater control of objects stored or retrieved from the service.