Method for adaptive training of multimodal large language model based on selective co-location

By adopting an adaptive training scheduling method for multimodal large language models with selective colocation, the mapping between model layers and devices is dynamically adjusted, which solves the problems of load imbalance, communication bottlenecks and memory risks in the training of multimodal large language models, and improves training efficiency and stability.

CN122173274APending Publication Date: 2026-06-09CHENGDU UNIV OF INFORMATION TECH
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
CHENGDU UNIV OF INFORMATION TECH
Filing Date
2026-02-11
Publication Date
2026-06-09

AI Technical Summary

Technical Problem

Existing multimodal large language model training frameworks suffer from problems such as load balancing failure, cross-modal communication bottlenecks, uncontrollable GPU memory risks, and heavy reliance on manual configuration when dealing with heterogeneous architectures, resulting in limited training efficiency and stability.

Method used

A selective co-location multimodal large language model adaptive training scheduling method is adopted. A closed-loop control framework is constructed through an analyzer, a planner, and an executor. The input image resolution and computational load are monitored in real time. Mixed-integer linear programming is used to optimize the model and dynamically adjust the mapping between the model layer and the device to achieve dynamic allocation and optimization of resources.

Benefits of technology

It effectively solves the problems of load imbalance, communication bottleneck and memory overflow, improves the computing utilization and training throughput of GPU clusters, reduces cross-node communication latency, and ensures the stability and security of training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122173274A_ABST
    Figure CN122173274A_ABST
Patent Text Reader

Abstract

This invention discloses an adaptive training scheduling method for multimodal large language models based on selective co-location. It achieves this by constructing a closed-loop framework consisting of an analyzer, a planner, and an executor, encompassing detection / perception, decision-making / planning, and dynamic execution. The analyzer acquires the resolution of the input image and the computation time, memory usage, and communication traffic parameters of each model layer. The planner inputs these parameters into a cost model based on mixed-integer linear programming. While ensuring that memory usage does not exceed hardware capacity and strictly maintaining the original hierarchical order of the model, a selective co-location strategy is used to reduce communication overhead across modal boundaries, solving for the relationship matrix between each model layer and the device. The executor reallocates model layers and switches loads across the GPU cluster based on this relationship matrix. This invention solves the problem of load imbalance caused by structural differences between different modalities during multimodal large language model training, improving training throughput while ensuring memory safety.
Need to check novelty before this filing date? Find Prior Art