Storage Proxy Data Auditing for Public Cloud Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The security of data stored in public object storage clouds is compromised due to the eventual consistency model, which makes it difficult to monitor and detect sensitive information, and the rate limits of public object storage, making it challenging to filter and remove such data in a timely manner.
Innovation Solution
Implementing a data auditing system that includes a service broker, a storage proxy, and a data auditor to assess and qualify data before it is stored in public object storage. The service broker determines if data auditing is necessary, creates a storage unit and proxy, and directs the data to be stored temporarily in the proxy. The data auditor then retrieves the data, determines its qualification, and notifies the storage proxy, ensuring that only compliant data is stored.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is stored directly in public object storage, then storage capacity and accessibility are improved, but security and compliance control deteriorate due to the eventual consistency model making monitoring difficult
Solution Approach 1:
The patent introduces a storage proxy as an intermediary component between the client and public object storage. The storage proxy interceptates data writes, performs security scanning and compliance checks, and only allows approved data to be stored in the public object storage. This mediator resolves the contradiction by maintaining storage accessibility while adding security control that was previously impossible due to the eventual consistency model.
Solution Approach 2:
The system performs security scanning and compliance verification in advance, before data is written to public object storage. The storage proxy scans data upfront, determines its qualification status, and blocks potentially non-compliant data from being stored. This preliminary action eliminates the need for continuous monitoring of stored data, resolving the security-monitoring dilemma.
2Reliability
If continuous monitoring is implemented to detect sensitive information, then data security is improved, but system complexity and processing overhead increase due to rate limits
Solution Approach 1:
Instead of implementing continuous monitoring of stored data, the system performs security scanning and compliance checks in advance, before data is written to storage. The storage proxy scans incoming data, determines its qualification, and blocks non-compliant data from being stored. This preliminary action eliminates the need for complex continuous monitoring systems while maintaining high data security.
Solution Approach 2:
The patent extracts the security scanning and compliance verification function from the main storage system and places it in the storage proxy layer. This separation allows the monitoring function to operate independently with dedicated resources, reducing the complexity burden on the core storage system while maintaining effective security oversight.
3Reliability
If data filtering is performed after storage, then compliance enforcement is improved, but response time and data removal speed deteriorate due to rate limits
Solution Approach 1:
The system performs compliance verification and data qualification checks before data is written to public object storage. The storage proxy scans incoming data, determines its compliance status, and blocks non-compliant data from being stored in the first place. This preliminary compliance enforcement eliminates the need for post-storage filtering and data removal operations, thereby maintaining fast response times while ensuring compliance.
4Reliability
If a storage proxy is introduced for data auditing, then data security and compliance control are improved, but system complexity increases
Solution Approach 1:
The storage proxy is designed as a multi-functional component that combines data scanning, compliance verification, qualification determination, and data routing capabilities in a single system. By consolidating multiple functions into one universal component, the patent reduces the overall system complexity that would otherwise result from having separate independent systems for each function.
Solution Approach 2:
The storage proxy serves as a standardized intermediary layer between clients and public object storage, providing a uniform interface for data writes while handling complex security and compliance logic internally. This mediator pattern simplifies the overall architecture by centralizing complexity in a single, well-defined component rather than distributing it across multiple interacting systems.
Data Source
AI summary
An example method for data auditing for object storage public clouds includes a service broker receiving a request to store data in public object storage, where the request includes user information or a container image. The service broker, based on either the user information or the container image, determines that data auditing is necessary. The service broker creates a storage unit, in public object storage, and a storage proxy. The method further includes the storage proxy storing data, and a data auditor retrieving data from the storage proxy. The data auditor determines a data qualification for the data, and notifies the storage proxy of the data qualification.