Score-Based Model Selection for Video Next-Frame Prediction

The patent combines multiple machine learning models (MLMs) into an ensembling system where each model processes the same input frames and produces predictions. The system then aggregates these predictions through a scoring mechanism to select the best prediction. This merging approach allows the system to leverage the strengths of different models for different video types, thereby improving overall prediction accuracy without requiring a single complex universal model.

The ensembling system creates a universal prediction framework that can handle diverse video content types (human motion, machine movement, animal movement, etc.) through a single unified architecture. Multiple specialized models are integrated into one system that automatically adapts to different video types by evaluating and selecting the most appropriate model based on the input frames, making the system universally applicable across various video domains.

The system segments the prediction task by dividing it into multiple specialized models, each trained on specific video content types. Instead of creating one large complex model, the system breaks down the prediction function into smaller, focused models (e.g., one for human motion, another for machine movement). The ensembling framework then coordinates these segmented models through a scoring and selection mechanism, managing complexity through modular organization.

The system dynamically selects which models to activate and how to weight their predictions based on the characteristics of the input video frames. The scoring mechanism dynamically adjusts the contribution of each model according to the specific video content, allowing the system to adapt its complexity level for each prediction task. This dynamic selection enables the system to maintain high adaptability while managing computational resources efficiently.