Workload File Synchronization System for Compute Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing workload files across multiple compute systems is inefficient, leading to outdated files being used during compute sessions, resulting in wasted time and expense due to the complexity of file management across different cloud and bare metal computing systems.
Innovation Solution
A synchronization system acts as an arbiter of truth, ensuring that workload files are consistently updated by copying files from central storage to compute systems and vice versa, using hashing to verify file integrity and avoid redundant transfers, thereby maintaining the most current files across all systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If files are manually managed across multiple compute systems, then users have flexibility in selecting different compute systems for workloads, but file synchronization becomes complex and outdated files may be used
Solution Approach 1:
The patent introduces a file synchronization service as an intermediary component that manages file copies across multiple compute systems. This service automatically tracks file locations, monitors for updates, and handles replication transparently, allowing users to select different compute systems without manually managing file synchronization complexity.
Solution Approach 2:
The synchronization service implements self-service mechanisms by automatically detecting when files are updated on one compute system and autonomously replicating those updates to other systems. The system monitors file change events and triggers synchronization operations without user intervention, resolving the complexity of manual file management across distributed systems.
2Productivity
If files are copied to multiple compute systems for local operation, then compute performance improves, but storage requirements and costs increase
Solution Approach 1:
The patent implements local quality by maintaining local copies of files on compute systems that are frequently accessed or require high-performance operations, while relying on the synchronization service to manage the broader distribution. This selective local caching approach ensures compute performance for critical systems without unnecessarily duplicating storage across all systems.
3Reliability
If file synchronization is continuously monitored across all systems, then file accuracy is maintained, but system overhead and processing time increase
Solution Approach 1:
The synchronization service employs periodic action by implementing event-driven synchronization that activates only when file changes are detected. Rather than continuously monitoring all systems, the service uses change event notifications to trigger synchronization operations, maintaining file accuracy while minimizing unnecessary processing overhead and time consumption.
Data Source
AI summary
A synchronization system configured to identify a first workload file on a central storage and identify a first workload file on a compute system remote from the central storage. The synchronization system determines if the first workload file on the central storage is different than the first workload file on the compute system. If the two first workload files are different, the synchronization system automatically copies the first workload file on the central storage to the compute system so the a compute system workload will be performed using the copy of the first workload file from the central storage and the compute system workload will not be performed using the first workload file on the compute system.