Cloud Controller Anti-Virus Scanning for Distributed Filesystem Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based storage systems face challenges in maintaining data consistency and accessibility due to increased access latency and network failures, which can lead to data consistency issues and client timeouts, especially when multiple clients access shared data sets.
Innovation Solution
A distributed filesystem architecture where cloud controllers collectively manage and cache data, ensuring data consistency by performing anti-virus scans and using transactional filesystems to maintain metadata and file data snapshots, allowing only valid data to be accessed and minimizing latency through incremental snapshot propagation and caching techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If cloud-based storage systems are used to provide large-scale remote network storage solutions, then storage capacity and ease of access are improved, but access latency increases and data consistency problems occur
Solution Approach 1:
The system performs preliminary actions by pre-scanning files for viruses when they are written to the distributed filesystem and marking them with clean field metadata before clients access them. This proactive approach ensures that security checks are completed in advance, preventing security issues from arising during client access operations.
2Quantity of substance
If cloud-based storage systems are used to provide large-scale remote network storage solutions, then storage capacity is improved, but data consistency problems occur when multiple clients simultaneously access shared data sets
Solution Approach 1:
The system implements feedback mechanisms where cloud controllers continuously monitor and update the clean field metadata for files in the distributed filesystem. When files are scanned and found to be virus-free, the clean field is set to true and this information is propagated across the distributed system, ensuring all clients have access to the latest security status information.
3Reliability
If anti-virus scans are performed on target files in the distributed filesystem, then data security is improved, but access latency increases due to the additional scanning time
Solution Approach 1:
The system performs preliminary anti-virus scans when files are written to the distributed filesystem, completing security checks before clients attempt to access the files. This ensures that security validation occurs in advance, and clients can access already-scanned files without experiencing additional latency.
Solution Approach 2:
The system implements incremental scanning where the clean field metadata is updated progressively as files are scanned. Rather than requiring complete re-scanning of entire filesystems, the system performs partial scans on individual files or directories and updates metadata incrementally, reducing the overall impact on system performance.
4Reliability
If cloud controllers collectively manage and cache distributed filesystem data to ensure data consistency, then reliability is improved, but device complexity increases
Solution Approach 1:
The system segments the distributed filesystem management into distinct cloud controllers, each responsible for specific portions of the filesystem metadata and data caching. This segmentation allows the system to maintain data consistency through distributed coordination while avoiding the complexity of a monolithic management system.
Data Source
AI summary
The disclosed embodiments disclose techniques that facilitate the process of performing anti-virus checks for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a write request from a client system that seeks to store a target file in the distributed system. A scan is then performed for this target file. For instance, the scan may be an anti-virus scan that ensures that viruses are not spread to the distributed filesystem or the clients of the distributed filesystem.