Video Memory Allocation for Large-Model Backward Passes

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Insufficient video memory during large model training leads to performance degradation due to recalculation strategies, which introduce unnecessary computational overhead.

Innovation Solution

Reconstruct video memory space based on parameter gradient information for backward calculations, store results, and release the space after calculations are completed to reduce occupancy and recalculation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If video memory is allocated for storing parameter gradient information during backward calculations, then calculation speed is improved, but video memory occupancy increases

Engineering Contradiction:

Improvecalculation speedVSAvoidvideo memory occupancy

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The patent implements dynamic allocation and release of video memory space for parameter gradient information. The video memory space is allocated before backward calculations and released after calculations complete, making the memory management adaptive to different training stages rather than static. This dynamic approach allows the system to optimize between calculation speed and memory occupancy by allocating memory only when needed for computations.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent performs preliminary allocation of video memory space for parameter gradient information before the backward calculation process begins. By pre-allocating the necessary memory space, the system avoids memory allocation overhead during actual calculations, thereby improving calculation speed while controlling peak memory occupancy through planned resource reservation.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If recalculation strategy is used to save video memory, then video memory occupancy is reduced, but training performance degrades

Engineering Contradiction:

Improvevideo memory occupancyVSAvoidtraining performance

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The patent extracts only the necessary parameter gradient information required for current backward calculations into video memory, rather than storing all model parameters. By selectively extracting and storing only the gradient data needed for immediate computation, the system reduces overall video memory occupancy while maintaining sufficient data for accurate backward propagation, thus avoiding the need for recalculation strategies.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS20260073223A1Method for controlling video memory for model training, electronic device and storage medium

Publication Date: 2026.03.12 BEIJING BAIDU NETCOM SCI & TECH CO LTD

AI summary

A method for controlling a video memory for model training, an electronic device and a storage medium are provided, relating to the field of artificial intelligence technology, and in particular to the fields of neural network, large model, training optimization and other technologies. The method includes: reconstructing a video memory space for one or more backward calculations during model training according to grouping information of parameter gradient information required for the one or more backward calculations; performing the one or more backward calculations to obtain one or more backward calculation results; storing the one or more backward calculation results into the video memory space reconstructed for the one or more backward calculations; and releasing the video memory space reconstructed for the one or more backward calculations.