Cloud Upload Module Version Conflict Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud storage systems face challenges in efficiently managing data uploads from both online dedicated data servers and offline transportable storage devices, particularly in handling multiple versions of data and ensuring the most recent version is stored correctly, while also dealing with limitations in bandwidth and latency.
Innovation Solution
The solution involves using 'data box edges' and 'data box pods' that are communicatively coupled to store and manage data locally until upload, maintaining metadata for version control, and using an object upload module to determine conflict status and upload only the most recent versions to the cloud storage system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred online through dedicated data servers, then transfer speed is improved, but bandwidth is limited
Solution Approach 1:
The system segments data transfer into two independent pathways: online transfer through dedicated data servers for speed-critical operations, and offline transfer through transportable storage devices for bulk data movement. This segmentation allows each pathway to optimize for its specific strength without being constrained by the other's limitations.
Solution Approach 2:
The invention adds a temporal dimension to data transfer by introducing offline storage devices that can be physically transported. This creates a third dimension beyond traditional online/offline binary choices, enabling data to be staged locally and transferred at convenient times when network bandwidth is available, thus bypassing both speed and bandwidth constraints.
2Adaptability or versatility
If multiple versions of data are stored, then data availability is improved, but version management complexity increases
Solution Approach 1:
The system performs preliminary version validation by comparing object identifiers and metadata before committing data uploads. This preliminary check prevents duplicate or outdated versions from being stored, maintaining data availability while avoiding the complexity of managing multiple conflicting versions.
Solution Approach 2:
The system implements feedback mechanisms where the cloud storage system responds with conflict status information when version conflicts are detected. This feedback loop enables automatic conflict resolution by providing clear guidance on whether to proceed with upload or retrieve the latest version, simplifying version management through automated decision-making.
3Quantity of substance
If offline storage devices are used for data transfer, then bandwidth limitations are overcome, but transfer latency increases
Solution Approach 1:
The system dynamically selects between online and offline transfer modes based on real-time conditions such as data size, network availability, and urgency requirements. This dynamic approach allows the system to use offline high-bandwidth transfer for non-urgent bulk data while using online fast transfer for time-critical operations, optimizing both bandwidth utilization and latency performance.
4Reliability
If conflict detection is performed before upload, then data integrity is improved, but processing time increases
Solution Approach 1:
The system performs partial conflict detection by checking only critical metadata fields such as object identifiers and version timestamps rather than进行全面 data analysis. This partial checking approach provides sufficient data integrity protection while minimizing the processing time overhead, avoiding excessive validation that would unnecessarily delay uploads.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Transportable storage devices are communicatively coupled behind one or more dedicated "edge" data servers to take advantage of the benefits of the data servers and transportable storage devices while limiting their disadvantages. Each edge data server ingests data from one or more client devices and copies the data to one or more of the transportable storage devices for eventual transport to a cloud data center for upload to a cloud storage system. Multiple versions of a segment of data (e.g., of a file or a blob) are maintained at various points in the disclosed system and the disclosed technology ensures that the most recent version is the version ultimately stored on the cloud storage system.