On-demand Data Obfuscation in Object Storage I/O Path
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face challenges in enforcing data manipulations at storage systems and require complex external services to manage data manipulations, which can be insecure and inefficient.
Innovation Solution
The integration of serverless task executions with object storage services allows for direct insertion of data processing pipelines into the I/O path, using local stream handles to simplify code execution and enhance security by restricting network communications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data manipulations are enforced at storage systems using external services, then data processing capability is improved, but system complexity and security risks increase
Solution Approach 1:
The patent merges data manipulation capabilities directly into the storage system by integrating serverless function execution within the storage service infrastructure. This allows data processing pipelines to be embedded in the I/O path without requiring separate external services, thereby reducing system complexity while maintaining enhanced data processing capability.
Solution Approach 2:
The storage system is designed to perform multiple functions: traditional data storage plus on-demand code execution for data manipulation. By making the storage system universal and multi-functional, it eliminates the need for specialized external processing services, reducing overall system complexity while improving data processing versatility.
2Adaptability or versatility
If external services are used for data manipulations, then data processing flexibility is improved, but security risks increase
Solution Approach 1:
The patent introduces a controlled intermediary layer within the storage system that mediates between data I/O operations and code execution. This intermediary enforces security policies by restricting network communications and controlling what operations user-provided code can perform, thereby maintaining processing flexibility while mitigating security risks.
Solution Approach 2:
The system creates a secure, isolated execution environment for data manipulation code that restricts network access and other potentially harmful operations. This 'inert environment' allows flexible data processing while preventing security threats by limiting what the executed code can do.
3Adaptability or versatility
If complex external services manage data manipulations, then processing capability is improved, but operational efficiency decreases
Solution Approach 1:
By combining data storage and data manipulation functions into a single integrated service, the patent eliminates the overhead of inter-service communication and coordination. Data processing pipelines execute directly within the storage system's I/O path, improving operational efficiency while maintaining advanced processing capability.
4Productivity
If data processing pipelines are inserted into I/O path, then data manipulation efficiency is improved, but system complexity increases
Solution Approach 1:
The system provides self-service capabilities by automatically managing the execution of data processing pipelines within the I/O path. The storage service handles code invocation, execution, and result integration without requiring complex external orchestration, thereby improving manipulation efficiency while keeping system complexity manageable through automated self-management.
Data Source
AI summary
Systems and methods are described for modifying input and output (I/O) to an object storage service by implementing one or more owner-specified functions to I/O requests. A function can implement a data manipulation, such as filtering out sensitive data before reading or writing the data. The functions can be applied prior to implementing a request method (e.g., GET or PUT) specified within the I/O request, such that the data to which the method is applied my not match the object specified within the request. For example, a user may request to obtain (e.g., GET) a data set. The data set may be passed to a function that filters sensitive data to the data set, and the GET request method may then be applied to the output of the function. In this manner, owners of objects on an object storage service are provided with greater control of objects stored or retrieved from the service