Large language model multi-stage simulation training method, system, equipment and medium

By employing a multi-stage simulation training method, utilizing multi-dimensional difficulty features and real-time performance monitoring, and dynamically adjusting training sample weights and parameter processing, the problems of uneven difficulty distribution and insufficient stage transitions in large language model training are solved, thereby improving training efficiency and model capabilities.

CN122088705AActive Publication Date: 2026-05-26XIAMEN YUANTING INFORMATION TECH CO LTD
5 Cites 0 Cited by

Patent Information

Application Number
CN202610459385.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-04-09
Publication Date
2026-05-26
Estimated Expiration
2046-04-09

AI Technical Summary

Technical Problem

Existing large language model training methods suffer from uneven distribution of training difficulty, insufficient stage transitions, and inadequate utilization of feedback, resulting in low training efficiency and limited model generalization ability.

Method used

A multi-stage simulation training method is adopted. Through multi-dimensional difficulty feature extraction and sample difficulty labeling system, progressive training stages are set, model performance is monitored in real time, training sample sampling weights are dynamically adjusted, and parameter freezing and fine-tuning are performed when switching stages to form a closed-loop optimization mechanism.

Benefits of technology

This approach enables the model to be trained within a suitable learning difficulty range, gradually improving its capabilities. It also addresses knowledge gaps between training stages and the adaptive optimization of training strategies, thereby enhancing training stability and efficiency.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a large language model multi-stage simulation training method, system and device and a medium. The method comprises the following steps: performing multi-dimensional difficulty feature extraction on a training sample, and dividing difficulty levels to construct a sample difficulty label system; a plurality of sequentially progressive training stages are set, matched preset sample difficulty distribution is configured in each stage, and a training data set of each stage is sampled and constructed; the accuracy of the model on each difficulty level verification set is monitored in real time in training; dynamically adjusting the sampling weight based on the accuracy rate: reducing the weight of the grade of which the accuracy rate continuously exceeds a first threshold value and allocating to a higher grade, increasing the weight of the grade of which the accuracy rate is continuously lower than a second threshold value, and performing normalization; after the stage is finished, low-difficulty samples are extracted and added into a next-stage data set, and parameter freezing and fine tuning are executed; and the difficulty feature weight, the stage division threshold and the regulation and control judgment threshold are adjusted by using the training process data. According to the invention, training difficulty self-adaptive regulation and control and multi-stage progressive learning are realized.
Need to check novelty before this filing date? Find Prior Art