Random Access in Compressed Tar Archives
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Third-party cloud service cybersecurity solutions face performance and cost issues due to the need to copy and scan customer data, violating privacy laws and increasing scanning time and cost, especially when scanning data stored in Amazon Elastic Block Store (EBS) volumes.
Innovation Solution
Implementing serverless functions within the customer's cloud account to scan files, using AWS Lambda platform for orchestration, and employing snapshot and scanning mechanisms that maintain data privacy and reduce data transfer, allowing for efficient scanning of cloud resources without moving large data sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a cybersecurity service takes a snapshot of customer data, copies it to its account, and reformats it into a new storage volume for scanning, then the scanning can be performed externally, but this approach violates privacy laws, increases scanning time, and increases scanning cost
Solution Approach 1:
Instead of copying customer data out of the customer's account for scanning, the patent inverts the approach by bringing the scanning capability into the customer's account. The decompressor and scanner are deployed as functions within the customer's cloud account, allowing scanning to occur in-place without data movement, thereby maintaining privacy compliance while reducing scanning time
Solution Approach 2:
The patent extracts the decompression and scanning functionality from the traditional external cybersecurity service model and places it directly within the customer's account environment. This extraction eliminates the need to move data while retaining the scanning capability, resolving both the privacy compliance issue and the time consumption issue
2Adaptability or versatility
If a cybersecurity service copies customer data to its account for scanning, then external scanning is enabled, but data transfer costs increase and scanning time increases
Solution Approach 1:
The customer's account becomes self-sufficient by hosting the decompressor and scanner functions directly within it. The scanning operation serves itself by using locally deployed resources rather than external services, eliminating data transfer costs while maintaining scanning capability
3Reliability
If large amounts of customer data are moved between cloud accounts for scanning, then comprehensive security scanning is achieved, but scanning time increases and cost increases
Solution Approach 1:
The patent segments the scanning operation into discrete, independently deployable functions (decompressor function and scanner function) that can be executed within the customer's account. This segmentation allows the scanning process to be performed on data in-place without moving the entire data set, maintaining scanning completeness while dramatically improving efficiency by eliminating data transfer overhead
Data Source
AI summary
A cloud computing infrastructure hosts a web service with customer accounts. In a customer account, files of the customer account are listed in an index. Files indicated in the index are arranged in groups, with files in each group being scanned using scanning serverless functions in the customer account. The files in the customer account include a compressed tar archive of a software container. Member files of a compressed tar archive in a customer account are randomly-accessed by way of locators that indicate a tar offset, a logical offset, and a decompressor state for a corresponding member file. A member file is accessed by seeking to the tar offset in the compressed tar archive, restoring a decompressor to the decompressor state, decompressing the compressed tar archive using the decompressor, and moving to the logical offset in the decompressed data.