Aggregation Model Replication Using Differential Bucket Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for transferring aggregation model information from a primary site to a disaster recovery site require rebuilding the entire model, consuming substantial processing resources and time, especially in cloud infrastructure, leading to increased compute billing and inefficiency.
Innovation Solution
A mechanism for differential replication of aggregation models and training datasets from a primary site to a disaster recovery site, using a bucket tree structure and metadata mapping to efficiently update machine learning models without full reconstruction, reducing compute and data transmission requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire aggregation model is rebuilt at the disaster recovery site, then the model can be fully synchronized with the primary site, but substantial processing resources and time are consumed
Solution Approach 1:
The patent extracts only the changed portions (differential data) from the training data buckets at the primary site and transfers them to the disaster recovery site, rather than transferring or rebuilding the entire model. This differential update approach significantly reduces the amount of data that needs to be processed and transmitted.
Solution Approach 2:
The patent implements change detection mechanisms at the primary site that identify modifications in training data buckets before they propagate to the aggregation model. By detecting changes early and preparing differential updates in advance, the system can quickly synchronize the disaster recovery site without waiting for complete model rebuilds.
2Reliability
If the entire aggregation model is rebuilt at the disaster recovery site, then complete model information is available, but compute billing and processing resources increase substantially
Solution Approach 1:
The system extracts only the essential change information from the primary site's training data buckets and transfers this differential data to the disaster recovery site. This approach maintains model completeness at the DR site while minimizing the computational resources required for updates.
Solution Approach 2:
Instead of performing complete model rebuilding (excessive action), the system applies partial updates using only the changed portions of the training data. This partial action approach is sufficient to maintain synchronization while dramatically reducing compute resource consumption and associated billing costs.
3Reliability
If all training data is transmitted to the disaster recovery site, then complete model replication is achieved, but data transmission requirements and network bandwidth increase
Solution Approach 1:
The patent implements a change detection and extraction mechanism that identifies only the modified portions of training data in the buckets at the primary site. These differential data portions are then transmitted to the disaster recovery site, maintaining data completeness for model updates while minimizing network bandwidth consumption.
4Reliability
If the aggregation model is updated frequently at the disaster recovery site, then synchronization with the primary site is maintained, but processing overhead and system complexity increase
Solution Approach 1:
The system performs preliminary change detection at the primary site, identifying modifications in training data buckets before they affect the aggregation model. This advance detection allows the disaster recovery site to receive targeted update information without requiring complex continuous synchronization mechanisms.
Solution Approach 2:
The patent implements a feedback mechanism where the disaster recovery site can report its current state and received updates back to the primary site. This feedback loop ensures synchronization accuracy while allowing the system to optimize update frequency and reduce unnecessary processing overhead.
Data Source
AI summary
An approach is provided in which the approach establishes a connection between a first data site and a second data site. The first data site executes a first aggregation model that includes multiple first machine learning models and the second data site executes a second aggregation model that includes multiple second machine learning models. The approach detects, at the first data site, a change of data in a selected one of multiple first training data buckets corresponding to a selected one of the multiple first machine learning models. The approach sends the change of data from the first data site to the second data site, and replicates at the second data site, one of multiple second data buckets corresponding to a selected one of the multiple second machine learning models based on the change of data.