Cross-Set Knowledge Distillation for ML Model Training

The patent segments the training process into multiple stages: first training an initial model on a large data set, then using that model to generate synthetic training data, and finally training a second model on both the original and synthetic data. This segmentation allows the system to leverage large data sets for accuracy while managing model size through progressive refinement.

The patent performs preliminary action by using the initially trained ML model to infer and generate synthetic training data before final model training. This preliminary generation of augmented data allows the system to capture patterns from large data sets without requiring the final model to directly process all original data, thus managing model size while preserving accuracy benefits.

The patent implements self-service by enabling the ML model to automatically generate synthetic training data from its own inferences on the original data set. This self-generated data augmentation reduces the need for manual data collection and annotation, thereby managing training complexity while supporting increased content versatility.

The patent applies parameter changes by transforming the training data through the ML model's inference process, converting original data into synthetic training examples with modified parameters. This transformation allows the system to adapt to diverse content types without proportionally increasing training complexity, as the model learns generalized patterns rather than specific data instances.

The patent introduces an intermediary mechanism where the initially trained ML model acts as a mediator to generate synthetic training data. This intermediary process automates what would otherwise require manual annotation, producing high-quality training examples through model inference while dramatically reducing the time and cost associated with human annotation.

The patent uses copying by generating synthetic training data that replicates the patterns and structures of the original data set. The ML model creates copied and transformed versions of training examples, preserving the quality characteristics of manually annotated data while avoiding the time-consuming manual annotation process entirely.

The patent segments the modeling process into two phases: training a complex initial model on large data sets to capture accurate patterns, then using that model to generate synthetic data for training a final model that is optimized for deployment. This segmentation allows the system to achieve high accuracy through complex initial training while producing a final model that is more suitable for client device deployment.

The patent performs preliminary action by using the complex initial model to generate synthetic training data before final model deployment. This preliminary use of the complex model captures its accuracy benefits in the data generation phase, allowing the final deployed model to be simpler while still inheriting the accuracy advantages through the synthetic training data.