CDVFT fine tuning method with efficient memory and based on cyclic matrix inversion
By saving only the forward output in the CDVFT fine-tuning method and utilizing the inversion property of circular matrices, the problem of high peak memory consumption in deep learning training is solved, achieving a balance between efficient memory utilization and training performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUN YAT SEN UNIV
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-21
AI Technical Summary
Existing CDVFT fine-tuning methods suffer from high peak memory usage during deep learning training, especially when batch size and model depth are large. This is mainly due to the long lifecycle of intermediate state tensors, which leads to excessive memory consumption.
By using a method based on circular matrix inversion, only the forward output is saved during forward propagation. Taking advantage of the fast inversion property of circular matrices, tensor recovery is performed during backpropagation, reducing tensor storage during forward propagation. Inverse operations are also performed during backpropagation to reduce memory usage.
It significantly reduces peak memory usage during model training while maintaining training performance. For example, in RoBERTa-base full model training, peak memory usage is reduced from 11.96 GB to 8.68 GB, and downstream task accuracy is not affected.
Smart Images
Figure CN121900960A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of memory data adjustment technology, and specifically to a memory-efficient CDVFT fine-tuning method based on circular matrix inversion. Background Technology
[0002] As the size of deep learning model parameters continues to grow, memory consumption during model training has become a critical bottleneck. The Cyclic and Diagonal Vector-Based Fine-Tuning (CDVFT) method effectively reduces the memory and computational costs of model fine-tuning by representing weight increments as the product of interleaved cyclic and diagonal matrices.
[0003] However, the CDVFT fine-tuning method still has room for improvement in terms of memory usage during deep learning training. Traditional deep learning models, by default, store all tensors required for gradient calculation during the forward propagation phase of training, including a large number of intermediate state tensors. The memory occupied by these tensors accumulates as forward propagation progresses until they are used up in backpropagation. This long lifespan of tensors leads to high peak memory usage during model training, which is extremely detrimental in memory-constrained training environments. Furthermore, this limitation is particularly pronounced when the batch size of input data is large and the model depth is significant. Summary of the Invention
[0004] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a memory-efficient CDVFT fine-tuning method based on circular matrix inversion, which solves the technical problem that a large number of intermediate state tensors occupy memory space for a long period in the prior art, affecting the training environment.
[0005] To achieve the above-mentioned technical objectives, in a first aspect, the present invention provides a memory-efficient CDVFT fine-tuning method based on circular matrix inversion, comprising the following steps: Obtain the input column vector and weight increments; The intermediate calculation results and forward output are obtained by forward processing based on the input column vector and the weight increment. Only the forward output is stored in memory during the model training phase. During backpropagation in training, the tensor needed for gradient calculation is obtained by performing the inverse operation of the forward propagation on the forward output: The element-wise multiplication of the intermediate state vector and the diagonal matrix vector product is inversely performed using element-wise division; The calculation of intermediate state vectors and cyclic matrix vectors is based on the inverse operation of the inverse of the cyclic matrix.
[0006] Compared with the prior art, the beneficial effects of the present invention include: When calculating element-wise multiplication and DFT operations during forward propagation, existing CDVFT fine-tuning methods save a portion of the input tensors to meet the needs of gradient calculation. These tensors are stored in the system until they are used in gradient calculation. Because deep learning models require backpropagation layer by layer, a large peak memory usage is reached during the first few backpropagation layers, and the memory usage continues to increase with batch size and model depth.
[0007] This invention proposes a memory-efficient CDVFT fine-tuning method based on circulant matrix inversion. Addressing the high peak memory usage in existing CDVFT implementations due to the storage of tensors for gradient calculation, the core of this embodiment lies in utilizing the fast inversion property of circulant matrices to reduce the tensors stored during forward propagation of fine-tuning and to restore the tensors during backward propagation, thereby reducing the peak memory usage during the fine-tuning process.
[0008] According to some embodiments of the present invention, forward processing based on the input column vector and the weight increment is performed to obtain intermediate calculation results and forward output, including the following steps: For the input column vector and weight increment ,use The intermediate calculation results of matrix-vector multiplication are represented by the forward processing written as: , in, , ,and These represent constructing a diagonal matrix and a circular matrix, respectively. This represents element-wise multiplication; after the forward processing is complete, only the forward output is stored in memory. Abandon saving the previous input And other intermediate calculation results.
[0009] According to some embodiments of the present invention, the element-wise multiplication of the intermediate state vector and the product of the diagonal matrix vectors is inversely performed using element-wise division, including the following steps: Regarding circular matrix vectors and the product of diagonal matrices and vectors The gradient calculation is expressed as: Wherein, the objective function is denoted as , and They are respectively and The form after shifting.
[0010] According to some embodiments of the present invention, CDVFT takes the form of interleaved cyclic and diagonal matrix multiplication, and the backpropagation process includes four computations: (1) Use element-wise division to multiply the diagonal matrix vectors Restore to the input vector : (2) Calculation and gradient: After the calculation is complete, reclaim the tensors that are no longer needed. ; (3) By using the discrete Fourier transform and element-wise multiplication, the product of the circular matrix and vector is obtained. Restore to the input vector : in, yes The discrete Fourier transform matrix of order 1; (4) Calculation and gradient: in, This represents taking the conjugate of the input vector and recycling tensors that are no longer needed after computation. .
[0011] Secondly, the present invention provides a memory-efficient CDVFT fine-tuning system based on circular matrix inversion, applied to the memory-efficient CDVFT fine-tuning method based on circular matrix inversion as described in any one of the first aspects, comprising the following steps: The data acquisition module acquires the input column vector and weight increments; The forward processing module performs forward processing based on the input column vector and the weight increment to obtain intermediate calculation results and forward output. Only the forward output is stored in memory during the model training phase. The backpropagation module, during backpropagation in training, obtains the tensor needed for gradient calculation by performing the inverse operation of the forward propagation on the forward output: The element-wise multiplication of the intermediate state vector and the diagonal matrix vector product is inversely performed using element-wise division; The calculation of intermediate state vectors and cyclic matrix vectors is based on the inverse operation of the inverse of the cyclic matrix.
[0012] Thirdly, the present invention provides a deep learning model training method that applies the memory-efficient CDVFT fine-tuning method based on circular matrix inversion as described in any one of the first aspects.
[0013] According to some embodiments of the present invention, the deep learning model is a large-scale language model or a computer vision model.
[0014] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0015] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein the abstract drawings are to be completely consistent with one of the drawings in the specification: Figure 1 This is an overview diagram of a memory-efficient CDVFT fine-tuning method based on circular matrix inversion provided in one embodiment of the present invention. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0017] It should be noted that although functional modules are divided in the system diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0018] Reference Figure 1 , Figure 1 This is an overview diagram of a memory-efficient CDVFT fine-tuning method based on circular matrix inversion provided in one embodiment of the present invention.
[0019] In one embodiment, a memory-efficient CDVFT fine-tuning method based on circulant matrix inversion includes the following steps: obtaining input column vectors and weight increments; performing forward processing based on the input column vectors and weight increments to obtain intermediate calculation results and forward outputs, storing only the forward outputs in memory during the model training phase; obtaining the tensor required for gradient calculation by performing the inverse operation of forward propagation on the forward outputs during backpropagation of training; performing the element-wise multiplication of the intermediate state vector with the diagonal matrix vector product inverse operation using element-wise division; and performing the calculation of the intermediate state vector with the circulant matrix vector based on the inverse operation of the circulant matrix inversion.
[0020] The first step is to obtain the core input data: First, identify the core data required for model fine-tuning, including the input column vector and weight increments. The input column vector is the original input data for model training, and the weight increments are key parameters used for fine-tuning in the CDVFT method. They are in the form of the product of an interleaved circular matrix and a diagonal matrix, used to achieve efficient updates of the model parameters.
[0021] The second step, forward processing and simplified storage: Based on the acquired input column vector and weight increments, forward propagation calculations are performed. During forward processing, interleaved operations of diagonal matrix multiplication and cyclic matrix multiplication are performed sequentially: first, element-wise multiplication of the diagonal matrix with the current vector is performed to obtain an intermediate result, and then multiplication of this intermediate result with the cyclic matrix is performed. This alternating iterative process completes the entire calculation process, ultimately yielding the forward output.
[0022] Unlike the traditional CDVFT method, the key optimization in this step lies in the tensor storage strategy: after the forward propagation is completed, the system only retains the final forward output in the memory during the training phase, and no longer stores the input column vectors and all intermediate state tensors generated during the forward computation process. This reduces memory usage from the source and avoids the problem of high peak memory caused by the accumulation of intermediate tensors.
[0023] The third step is the inverse operation tensor recovery during backpropagation: When training enters the backpropagation phase, in order to complete the gradient calculation, it is necessary to recover the required tensors from the saved forward output through inverse operations. This is specifically divided into two types of inverse operation processing: Inverse operations related to diagonal matrices: For the element-wise multiplication of diagonal matrices and vectors in forward propagation, the inverse operation is performed by element-wise division in backward propagation, which restores the relevant intermediate results to the previous level input vector in forward propagation, providing basic data for gradient calculation.
[0024] Inverse operations related to circular matrices: For the "multiplication of a circular matrix and a vector" operation in forward propagation, the inverse operation is achieved by leveraging the fast inversion property of circular matrices and combining it with relevant operations of the Discrete Fourier Transform, thus restoring the corresponding intermediate result to the previous level input vector. Simultaneously, the computation process simplifies the calculation by using conjugate transformation (when the input is a real vector, the Fourier transform of the shift operation is equivalent to the conjugate of the Fourier transform of the original vector), eliminating the need for additional shift operations and improving computational efficiency.
[0025] Through the aforementioned inverse operations, all intermediate state tensors and initial input column vectors not saved during forward propagation can be gradually restored. Simultaneously, gradient calculations and propagation of the diagonal and circumcircle matrix weights are completed, ensuring the normal progress of model fine-tuning. This entire process avoids storing a large number of intermediate tensors during forward propagation and guarantees training effectiveness through efficient inverse operations, achieving a balance between memory usage and training performance.
[0026] This embodiment uses, as follows: Figure 1 The forward processing flow is shown on the left under "Forward Process". For the input column vector... and weight increment ,use The intermediate calculation results of matrix-vector multiplication are represented by the forward processing written as: in, , ,and These represent constructing a diagonal matrix and a circular matrix, respectively. This indicates element-wise multiplication. After the forward processing is complete, the system only saves the forward output. Do not save the previous input Including other intermediate calculation results, thereby achieving the minimum peak memory usage without affecting the training process.
[0027] During backpropagation during training, this embodiment obtains the tensor required for gradient calculation by performing the inverse operation of the forward propagation on the forward output. For example... Figure 1 As shown in the "Backward Process" on the right, the element-wise multiplication of the intermediate state vector and the diagonal vector is inversely performed using element-wise division; while the calculation of the intermediate state vector and the cyclic matrix vector is based on the inverse operation of the cyclic matrix.
[0028] Furthermore, regarding circular matrix vectors and the product of diagonal matrices and vectors The gradient calculation is expressed as: in, and They are respectively and The form after shifting.
[0029] However, when the input to the Fast Fourier Transform (FFT) is a real vector, the Fourier transform of the shifted vector has been proven to be equivalent to the conjugate of the result of the Fourier transform of the original vector. Therefore, in actual computation, the shift operation is no longer needed; instead, the conjugate operation is used.
[0030] Let the objective function (such as the loss function) be denoted as Since CDVFT uses interleaved cyclic and diagonal matrix multiplication, the backpropagation process mainly involves four types of calculations: (1) Use element-wise division to multiply the diagonal matrix vectors Restore to the input vector : (2) Calculation and gradient: After the calculation is complete, reclaim the tensors that are no longer needed. .
[0031] (3) By using the discrete Fourier transform and element-wise multiplication, the product of the circular matrix and vector is obtained. Restore to the input vector : in, yes The discrete Fourier transform matrix of order 1.
[0032] (4) Calculation and gradient:
[0033] in, This represents taking the conjugate of the input vector. After computation, discard any tensors that are no longer needed. .
[0034] The above four-step calculation package covers the product of diagonal matrices and vectors. After circular matrix-vector product Restore to the next diagonal matrix vector product The process is as follows. Therefore, by repeating the above process sequentially, a forward-output process can be achieved. Restore all unsaved intermediate calculation results and inputs sequentially. (Right now ), and simultaneously complete all the necessary weights. and Gradient calculation and gradient propagation.
[0035] The results achieved were validated on a real dataset. 1. Since only the forward output is saved after the forward propagation of the model training is completed, the intermediate state tensors used for gradient calculation saved during the forward propagation of the existing CDVFT are eliminated, thereby significantly reducing the peak memory usage during model training.
[0036] 2. Since the model no longer saves the forward input and intermediate state tensors when it passes through each CDVFT layer during forward propagation, the peak memory usage during model training is significantly reduced (for example, experiments have shown that in RoBERTa-base full model training, this method reduces the peak memory to 8.68 GB, which is better than the existing CDVFT method of 11.96 GB, while maintaining a comparable accuracy for downstream tasks).
[0037] In one embodiment, a memory-efficient CDVFT fine-tuning system based on circulant matrix inversion includes: a data acquisition module for acquiring input column vectors and weight increments; a forward processing module for performing forward processing based on the input column vectors and weight increments to obtain intermediate calculation results and forward outputs, storing only the forward outputs in memory during model training; and a backpropagation module for obtaining the tensors required for gradient calculation by performing the inverse operation of the forward propagation on the forward outputs during backpropagation in training: the element-wise multiplication of the intermediate state vector with the diagonal matrix vector product is inversely performed using element-wise division; and the calculation of the intermediate state vector with the circulant matrix vector is based on the inverse operation of the circulant matrix inversion.
[0038] In one embodiment, the deep learning model training method employs the aforementioned memory-efficient CDVFT fine-tuning method based on circulant matrix inversion. The deep learning model is a large-scale language model or a computer vision model.
[0039] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0040] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0041] Furthermore, one embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by a processor in the above-described terminal embodiment, such that the processor executes the memory-efficient CDVFT fine-tuning method based on cyclic matrix inversion in the above-described embodiment.
[0042] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.
[0043] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.
[0044] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A memory-efficient CDVFT fine-tuning method based on circular matrix inversion, characterized in that, Including the following steps: Obtain the input column vector and weight increments; The intermediate calculation results and forward output are obtained by forward processing based on the input column vector and the weight increment. Only the forward output is stored in memory during the model training phase. During backpropagation in training, the tensor needed for gradient calculation is obtained by performing the inverse operation of the forward propagation on the forward output: The element-wise multiplication of the intermediate state vector and the diagonal matrix vector product is inversely performed using element-wise division; The calculation of intermediate state vectors and cyclic matrix vectors is based on the inverse operation of the inverse of the cyclic matrix.
2. The memory-efficient CDVFT fine-tuning method based on circular matrix inversion according to claim 1, characterized in that, The forward processing based on the input column vector and the weight increment yields intermediate calculation results and forward output, including the following steps: For the input column vector and weight increment ,use The intermediate calculation results of matrix-vector multiplication are represented by the forward processing written as: ， in, , ,and These represent constructing a diagonal matrix and a circular matrix, respectively. This represents element-wise multiplication; after the forward processing is complete, only the forward output is stored in memory. Abandon saving forward input And other intermediate calculation results.
3. The memory-efficient CDVFT fine-tuning method based on circular matrix inversion according to claim 2, characterized in that, The element-wise multiplication of the intermediate state vector and the diagonal matrix vector product is inversely performed using element-wise division, including the following steps: Regarding circular matrix vectors and the product of diagonal matrices and vectors The gradient calculation is expressed as: Wherein, the objective function is denoted as , and They are respectively and The form after shifting.
4. The memory-efficient CDVFT fine-tuning method based on circular matrix inversion according to claim 3, characterized in that, CDVFT uses an interleaved product of cyclic and diagonal matrices, and the backpropagation process involves four computations: (1) Use element-wise division to multiply the diagonal matrix vectors Restore to the input vector : (2) Calculation and gradient: After the calculation is complete, reclaim the tensors that are no longer needed. ; (3) By using the discrete Fourier transform and element-wise multiplication, the product of the circular matrix and vector is obtained. Restore to the input vector : in, yes The discrete Fourier transform matrix of order 1; (4) Calculation and gradient: in, This represents taking the conjugate of the input vector and recycling tensors that are no longer needed after computation. .
5. A memory-efficient CDVFT fine-tuning system based on circular matrix inversion, applied to the memory-efficient CDVFT fine-tuning method based on circular matrix inversion as described in any one of claims 1 to 4, characterized in that, include: The data acquisition module acquires the input column vector and weight increments; The forward processing module performs forward processing based on the input column vector and the weight increment to obtain intermediate calculation results and forward output. Only the forward output is stored in memory during the model training phase. The backpropagation module, during backpropagation in training, obtains the tensor needed for gradient calculation by performing the inverse operation of the forward propagation on the forward output: The element-wise multiplication of the intermediate state vector and the diagonal matrix vector product is inversely performed using element-wise division; The calculation of intermediate state vectors and cyclic matrix vectors is based on the inverse operation of the inverse of the cyclic matrix.
6. A deep learning model training method, characterized in that, The memory-efficient CDVFT fine-tuning method based on circular matrix inversion as described in any one of claims 1 to 4 is applied.
7. The deep learning model training method according to claim 6, characterized in that, The deep learning model is a large-scale language model or a computer vision model.