Index-Based Smart Folder Scan for Cloud Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud computing environments lack efficient methods for searching files across different block storage volumes, as existing file search techniques are manual, resource-intensive, and time-consuming, involving cumbersome multi-step processes and extensive scanning of snapshots.
Innovation Solution
An index-based smart scan system is implemented, which includes a processor and memory configured to access and scan snapshots of block storage volumes, generate a folder index table with file paths and modification times, and update it periodically, allowing for efficient location and identification of files across multiple block storage volumes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual multi-step scanning methods are used to search for files across volume snapshots, then file search completeness is achieved, but search time and resource consumption increase significantly
Solution Approach 1:
The system performs preliminary scanning of block storage volume snapshots to generate folder index tables in advance. These index tables contain file path and modification time information that is prepared beforehand, allowing rapid file location without performing full scans during actual search operations. This preliminary indexing action resolves the contradiction by pre-processing data to enable fast subsequent searches.
Solution Approach 2:
The folder index table serves as an intermediary data structure between the raw snapshot data and the file search query. Instead of directly scanning snapshot data during searches, the system uses this intermediate index table that contains pre-extracted file metadata, dramatically reducing search time while maintaining search completeness.
2Measurement precision
If full scan of all snapshots is performed to locate files, then accurate file identification is achieved, but resource consumption and processing time increase
Solution Approach 1:
The system extracts only the essential file metadata (file path and modification time) from the complete snapshot data during preliminary scanning. This extracted information is stored in folder index tables, allowing the system to perform file identification without processing the entire snapshot data during search operations, thus improving productivity while maintaining accuracy.
Solution Approach 2:
Instead of performing excessive full scans of all snapshot data during each file search, the system performs partial scanning only when necessary to update the index tables. The folder index tables contain sufficient information for most file location operations, eliminating the need for excessive full scans and improving processing efficiency.
3Measurement precision
If folder index tables are updated frequently to maintain accuracy, then file location precision is improved, but system resource overhead increases
Solution Approach 1:
The system updates folder index tables periodically rather than continuously or with every snapshot change. This periodic update approach maintains adequate file location precision by refreshing the index at regular intervals, while significantly reducing system resource overhead compared to continuous synchronization of the index tables with snapshot data.
Data Source
AI summary
A system and method for an index-based smart scan for a cloud-computing provider network is provided. The system includes a memory having computer-readable instructions stored therein and a snapshot repository configured to store a plurality of snapshots of a plurality of block storage volumes. Each of the plurality of block storage volumes is configured to perform volume based block storage operations for the cloud-computing provider network. The system further includes a processor communicatively coupled to the snapshot repository. The processor is configured to access contents of each of the plurality of snapshots. Each of the plurality of snapshots includes a point-in-time capture of the respective block storage volume. In addition, the processor is configured to perform a full scan of each of the plurality of snapshots to identify one or more files of the respective block storage volume. The processor is further configured to generate a folder index table for each of the files based upon the scan. The folder index table includes a listing of the file path and an associated modification time for each of the files. Furthermore, the processor is configured to update the folder index table over a pre-determined period of time and locate a file in the plurality of block storage volumes based on the folder index table.