Adaptive large language model training

By identifying and optimizing the computational resources of large language models and utilizing multiple training types for adaptive training, the problem of low model training efficiency is solved, achieving more efficient resource utilization and better model adaptability.

CN122397015APending Publication Date: 2026-07-14INTERNATIONAL BUSINESS MACHINE CORPORATION
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
INTERNATIONAL BUSINESS MACHINE CORPORATION
Filing Date
2024-11-21
Publication Date
2026-07-14

AI Technical Summary

Technical Problem

Training large language models (LLMs) becomes extremely challenging as the model size grows, especially as the demand for and supply of computing resources are difficult to optimize, leading to low training efficiency and poor resource utilization.

Method used

By identifying computational resources in the model topology, predictive vectors for changes are generated. A series of optimization experiments are conducted using optimization algorithms and multiple training types to determine the optimal supply and demand of computational resources. The training type with the lowest accuracy penalty value is selected for training, and adaptive training is performed using encoder and decoder stacks.

Benefits of technology

It improves the efficiency and accuracy of the training process, enhances the model's adaptability to different domains, optimizes the utilization of computing resources, and improves the performance of LLM.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122397015A_ABST
    Figure CN122397015A_ABST
Patent Text Reader

Abstract

A method, system, and computer program product for adaptively training a large language model based on predicted changes in computing resources is provided. A processor can identify data elements of computing resources used to train a large language model. The identified data elements include a configured model topology. The processor can generate a forecast vector that captures predicted changes in the configured model topology over a period of time. The processor can perform a series of optimization experiments using an optimization algorithm and the predicted vector, and using each of a plurality of training types, to determine an optimal computing resource supply and demand over the period of time. The processor can determine an accuracy penalty value associated with each training type based on the series of optimization experiments. The processor can train the large language model using a first training type having a lowest accuracy penalty value.
Need to check novelty before this filing date? Find Prior Art