Model training parameter storage method and device

By obtaining model training parameters and gradient values ​​in large model training, determining and adjusting attributes and determining the target storage address, the problem of parameter preservation in the prior art affecting training efficiency and interrupts resulting in parameters being unable to be read, and efficient parameter preservation and reading are achieved.

CN119598203APending Publication Date: 2025-03-11LENOVO (BEIJING) LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411752452.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-30
Publication Date
2025-03-11

AI Technical Summary

Technical Problem

During the training process of large-scale model, the existing technology will affect the training efficiency when saving model training parameters. If an interrupt occurs during the storage process, the parameters will not be read, resulting in wasting training time.

Method used

By obtaining the model training parameters to be updated in the target model and their gradient values, the adjustment attributes of each parameter are determined, and the target storage address is determined based on the attribute, so as to achieve efficient parameter storage and reading.

Benefits of technology

It improves the storage and reading speed of model training parameters, reduces the memory space usage, saves data transmission bandwidth, and avoids the waste of iteration steps caused by interruption of training tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119598203A_ABST
    Figure CN119598203A_ABST
Patent Text Reader

Abstract

The invention provides a model training parameter storage method. The method comprises the steps of obtaining a to-be-updated model training parameter in a target model; in response to updating of at least one model training parameter of the target model, acquiring a gradient value of the model training parameter; according to the gradient value corresponding to each model training parameter, determining an adjustment attribute of each model training parameter, the adjustment attribute representing whether the model training parameter is adjusted based on the gradient value; and determining a target storage address of the model training parameter based on the adjustment attribute. The invention further provides a model training parameter storage device.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Cited By

  • Memory resource allocation method and device, electronic equipment, medium and product

    CN120407201A