Dynamic Batch Strategy for Parallel Online Learning

The patent applies the dynamics principle by transitioning from a static, constant batch size to a dynamic batch size that automatically adjusts based on the measured divergence between model states. The system calculates the divergence metric after each training iteration and uses this measurement to determine the appropriate batch size for the next merge operation, allowing the merging frequency to adapt to the actual state of model divergence without requiring manual intervention or complex configuration.

The patent implements the feedback principle by introducing a divergence measurement mechanism that continuously monitors the state of model states after each training iteration. This feedback loop provides real-time information about model divergence to the control system, which then adjusts the batch size accordingly. When divergence is low, the system reduces merging frequency to minimize communication overhead; when divergence exceeds a threshold, it increases merging frequency to maintain accuracy, thus optimizing the trade-off between reliability and time efficiency.

The patent applies the parameter changes principle by dynamically modifying the batch size parameter based on the divergence metric. Instead of using a fixed, constant batch size, the system adjusts the batch size parameter in response to changes in model state divergence. When the divergence metric indicates that model states are highly consistent across nodes, the system reduces the batch size or skips merges entirely, thereby conserving computational resources. When divergence increases, the system increases the batch size to ensure sufficient synchronization, thus optimizing resource utilization while maintaining stability.