Fine-tuning method and device for pre-trained large language model

Abstract

The present disclosure relates to a fine-tuning method and device for pre-training a large language model, which determines at least one iterative processing process with a corresponding operator in the training process of the large language model, receives the parameters of each corresponding operator stored in other accelerators at the beginning of each iterative process, completes the first iterative processing, and generates / consumes the activation vector corresponding to each operator. Then, the memory information corresponding to the first iterative processing is determined to adjust the memory space on the accelerator corresponding to the large language model, including the activation vector space and the parameter space outside the activation vector space. The activation vector corresponding to each operator is stored in the activation vector space, the corresponding parameters are stored in the parameter space, and the subsequent iterative processing process is completed based on the parameters in the parameter space. The present disclosure automatically adjusts the memory usage of the accelerator and automatically caches the parameters needed during the training process of the large language model, so as to reduce communication overhead through parameter reuse and improve the overall throughput.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for fine-tuning a pre-trained large language model. Background Art

[0002] Large language models are widely used. To achieve personalization and allow each user to have their own unique large model, fine-tuning technology is crucial. Fine-tuning involves further updating the parameters of the pre-trained large model based on user-provided data, allowing the large model to learn the characteristics of the newly introduced data. To prevent the fine-tuning process from damaging the original capabilities of the pre-trained model and to reduce the computational complexity of fine-tuning, existing fine-tuning solutions generally adopt parameter-efficient fine-tuning methods: keeping most parameters frozen and only updating a small number of parameters through training. This method maintains the original capabilities of the pre-trained model while increasing its capabilities in the vertical domain corresponding to the newly introduced data. Fine-tuning requires efficient computing because the model needs to be continuously fine-tuned on constantly generated data, and there will also be fine-tuning requests from many users. During the fine-tuning process, it is necessary to try multiple pre-trained models and hyperparameters to improve the quality of the fine-tuning results. The performance bottleneck of fine-tuning on accelerators (such as GPUs) lies in parameter acquisition. To train large models, parameters are distributed across multiple accelerators due to the limited memory capacity of a single accelerator. During the computation process, each accelerator must call an All-Gather operation to obtain all model parameters from other accelerators. Due to the limited communication bandwidth between accelerators, the communication operations required to obtain model parameters become a performance bottleneck. Summary of the Invention

[0003] In view of this, the present disclosure proposes a fine-tuning method and device for a pre-trained large language model, aiming to improve the communication efficiency during the training process of the large language model.

[0004] According to a first aspect of the present disclosure, a method for fine-tuning a pre-trained large language model is provided, and an accelerator for fine-tuning the large language model is provided. The method comprises:

[0005] Determining at least one iterative processing process in a large language model training process, each of the iterative processing processes corresponds to at least one operator, and the large language model is a pre-trained model;

[0006] For each of the iterative processing processes, receiving parameters of each corresponding operator of the iterative processing process stored in other accelerators and completing the first iterative processing, generating / consuming activation vectors corresponding to each of the operators;

[0007] After completing the first iterative process, determining memory information corresponding to the first iterative process;

[0008] adjusting a memory space on the accelerator for fine-tuning the large language model according to the memory information, wherein the memory space of the accelerator includes an activation vector space and a parameter space other than the activation vector space;

[0009] Storing the activation vector corresponding to each of the operators in the activation vector space;

[0010] The parameters of each operator are stored in the parameter space, and the iterative processing process is completed based on the parameters in the parameter space.

[0011] In a possible implementation, determining the memory information corresponding to the first iterative processing includes:

[0012] Determining the amount of memory corresponding to the activation vector generated by each of the operators;

[0013] Determining a change trend of memory usage corresponding to the activation vector;

[0014] Corresponding memory information is determined according to the memory amount and the memory change trend.

[0015] In a possible implementation, adjusting the memory space on the accelerator for fine-tuning the large language model according to the memory information includes:

[0016] Adjusting the amount of memory corresponding to the activation vector of each operator by adjusting the batch size dimension of the activation vector stored in the memory space, wherein the adjustment trend is determined by the current total amount of memory in the accelerator;

[0017] It is determined whether the activation vector corresponding to each of the operators needs to be saved, and the activation vector space in the memory space is adjusted according to the amount of memory corresponding to the activation vectors that need to be saved.

[0018] In one possible implementation, the parameters include local parameters and remote parameters sent by other accelerators, and the parameter space includes a static cache space and a dynamic cache space. The static cache space is used to store local parameters, and the dynamic cache space is used to store remote parameters sent by other accelerators.

[0019] In a possible implementation, storing the parameters of each operator in the parameter space includes:

[0020] At least part of the remote parameters sent by other accelerators is stored in the dynamic cache space.

[0021] In a possible implementation, adjusting the memory space on the accelerator for fine-tuning the large language model according to the memory information includes:

[0022] In response to the change trend being an increase in memory usage, maintaining the allocation of static cache space in the memory space, reducing the allocation of dynamic cache space in the memory space, and increasing the allocation of activation vector space in the memory space;

[0023] In response to the change trend being a decrease in memory usage, the allocation of static cache space in the memory space is maintained, the allocation of activation vector space in the memory space is reduced, and the allocation of dynamic cache space in the memory space is increased.

[0024] In a possible implementation, the method further includes:

[0025] In response to reducing the allocation of the dynamic cache space in the memory space, part of the remote parameters stored in the dynamic cache space is released.

[0026] According to a second aspect of the present disclosure, a fine-tuning device for a pre-trained large language model is provided, and an accelerator for fine-tuning a large language model is provided, the device comprising:

[0027] an information determination module, configured to determine at least one iterative processing process in a large language model training process, each of the iterative processing processes corresponding to at least one operator, wherein the large language model is a pre-trained model;

[0028] A data processing module is configured to receive, for each iterative processing process, parameters of each corresponding operator of the iterative processing process stored in other accelerators, complete the first iterative processing, and generate / consume an activation vector corresponding to each of the operators;

[0029] A memory detection module, configured to determine memory information corresponding to the first iterative processing after the first iterative processing is completed;

[0030] a memory transformation module, configured to adjust a memory space on the accelerator used for fine-tuning the large language model according to the memory information, wherein the memory space of the accelerator includes an activation vector space and a parameter space other than the activation vector space;

[0031] A vector storage module, configured to store the activation vector corresponding to each of the operators in the activation vector space;

[0032] A parameter storage module is used to store the parameters of each operator in the parameter space and complete the iterative processing process based on the parameters in the parameter space.

[0033] In a possible implementation, the memory detection module is further configured to:

[0034] Determining the amount of memory corresponding to the activation vector generated by each of the operators;

[0035] Determining a change trend of memory usage corresponding to the activation vector;

[0036] Corresponding memory information is determined according to the memory amount and the memory change trend.

[0037] In a possible implementation, the memory transformation module is further configured to:

[0038] Adjusting the amount of memory corresponding to the activation vector of each operator by adjusting the batch size dimension of the activation vector stored in the memory space, wherein the adjustment trend is determined by the current total amount of memory in the accelerator;

[0039] It is determined whether the activation vector corresponding to each of the operators needs to be saved, and the activation vector space in the memory space is adjusted according to the amount of memory corresponding to the activation vectors that need to be saved.

[0040] In one possible implementation, the parameters include local parameters and remote parameters sent by other accelerators, and the parameter space includes a static cache space and a dynamic cache space. The static cache space is used to store local parameters, and the dynamic cache space is used to store remote parameters sent by other accelerators.

[0041] In a possible implementation, the parameter storage module is further configured to:

[0042] At least part of the remote parameters sent by other accelerators is stored in the dynamic cache space.

[0043] In a possible implementation, the memory transformation module is further configured to:

[0044] In response to the change trend being an increase in memory usage, maintaining the allocation of static cache space in the memory space, reducing the allocation of dynamic cache space in the memory space, and increasing the allocation of activation vector space in the memory space;

[0045] In response to the change trend being a decrease in memory usage, the allocation of static cache space in the memory space is maintained, the allocation of activation vector space in the memory space is reduced, and the allocation of dynamic cache space in the memory space is increased.

[0046] In a possible implementation, the apparatus further includes:

[0047] The parameter releasing module is configured to release part of the remote parameters stored in the dynamic cache space in response to reducing the allocation of the dynamic cache space in the memory space.

[0048] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0049] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0050] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0051] In an embodiment of the present disclosure, at least one iterative processing process with a corresponding operator in the training process of a large language model is determined, and at the beginning of each iterative process, the parameters of each corresponding operator stored in other accelerators are received and the first iterative processing is completed, and the activation vector corresponding to each operator is generated / consumed. Then the memory information corresponding to the first iterative processing is determined to adjust the memory space including the activation vector space and the parameter space outside the activation vector space on the accelerator corresponding to the large language model. The activation vector corresponding to each operator is stored in the activation vector space, the corresponding parameters are stored in the parameter space, and the subsequent iterative processing process is completed based on the parameters in the parameter space. The present disclosure automatically adjusts the memory usage of the accelerator and automatically caches the parameters needed during the training process of the large language model, so as to reduce communication overhead through parameter reuse and improve the overall throughput. Other features and aspects of the present disclosure will become clear based on the detailed description of exemplary embodiments with reference to the accompanying drawings below. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0053] Figure 1 A flowchart illustrating a method for fine-tuning a pre-trained large language model according to an embodiment of the present disclosure is shown;

[0054] Figure 2 A schematic diagram illustrating a fine-tuning process of a pre-trained large language model according to an embodiment of the present disclosure is shown;

[0055] Figure 3 A diagram showing the application effect of a fine-tuning method for a pre-trained large language model according to an embodiment of the present disclosure is shown;

[0056] Figure 4A schematic diagram illustrating a fine-tuning apparatus for a pre-trained large language model according to an embodiment of the present disclosure is shown;

[0057] Figure 5 A schematic diagram illustrating an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0059] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0060] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0061] The fine-tuning method for a pre-trained large language model according to the embodiments of the present disclosure can be performed by an electronic device such as a GPU server. The server can be a single server or a server cluster consisting of multiple servers. Any electronic device can implement the fine-tuning method for a pre-trained large language model according to the embodiments of the present disclosure by invoking computer-readable instructions stored in a memory via a processor.

[0062] Figure 1 FIG. 1 is a flow chart showing a method for fine-tuning a pre-trained large language model according to an embodiment of the present disclosure. Figure 1 As shown, the fine-tuning method of the pre-trained large language model in the embodiment of the present disclosure may include the following steps S10-S60.

[0063] Step S10: Determine at least one iterative processing process in the large language model training process.

[0064] In one possible implementation, the parameter caching method for accelerating the fine-tuning of a pre-trained model according to an embodiment of the present disclosure can be used to accelerate the accelerator corresponding to the large language model during the training process of a large language model. The large language model according to an embodiment of the present disclosure is a large language model that needs to be fine-tuned after pre-training, and the training process of the large language model is implemented by multiple corresponding accelerators, each accelerator stores part of the parameters of each operator in the large language model, and this method can be applied to the accelerator that is currently fine-tuning the large language model. Furthermore, the training process of the large language model includes multiple iterative processing processes, and each iterative processing process includes multiple operators, that is, an iterative processing process can be jointly executed by multiple operators. During the training process of the large language model, the parameters of some operators are stored on each accelerator, and the operators on each accelerator need to obtain the parameters of the operators from other accelerators through communication transmission before performing the corresponding calculations.

[0065] For example, when a large language model has D corresponding accelerators, each accelerator can locally store 1 / D of the operator parameters corresponding to the large language model. The storage of these local operator parameters can be determined before training the large language model by comparing the model parameter size with the accelerator memory size. During training, each accelerator communicates with other accelerators in real time, receives the locally stored parameters sent by other accelerators, and performs calculations based on the received parameters to complete the corresponding iterative processing process.

[0066] Step S20: For each of the iterative processing processes, receive the parameters of each corresponding operator of the iterative processing process stored in other accelerators and complete the first iterative processing, and generate / consume the activation vector corresponding to each of the operators.

[0067] In one possible implementation, after determining multiple iterative processing steps in the large language model training process, the electronic device receives the parameters of the operators required for the iterative processing steps stored in other accelerators before the start of each iterative processing step, and completes the first iterative processing step based on the locally stored operator parameters and the received operator parameters. During the iterative processing step, each operator generates / consumes a corresponding activation vector. The large language model training process includes two propagation processes: forward propagation and backward propagation. During the forward propagation iterative processing step, each operator generates a corresponding activation vector, and during the backward propagation iterative processing step, each operator consumes a corresponding activation vector.

[0068] Step S30: After completing the first iterative process, determine the memory information corresponding to the first iterative process.

[0069] In one possible implementation, any iterative processing process in the large language model training process can be completed through multiple iterative processing, wherein, after the first iterative processing of a certain iterative processing process is completed, the memory information corresponding to the first iterative processing can be determined through memory detection. The memory information can be determined by detecting the usage of the memory space in the accelerator before and after the execution of each operator, and is used to provide the necessary information for adjusting the accelerator memory space. The memory space of the accelerator includes an activation vector space and a parameter space other than the activation vector space. The activation vector space is used to store the activation vector generated during the operator calculation process, and the parameter space is used to store the parameters of the operator.

[0070] Optionally, the memory information may include the amount of memory corresponding to the activation vectors generated / consumed by each operator during the first iterative processing, as well as the changing trend of the memory usage of the activation vectors in the accelerator's memory space. In other words, the process of determining the memory information may include determining the amount of memory corresponding to the activation vectors generated by each operator, as well as determining the changing trend of the memory usage corresponding to the activation vectors. The corresponding memory information is determined based on the memory amount and the memory change trend. Memory usage refers to the amount of space required to store all activation vectors in the accelerator's memory space.

[0071] Furthermore, the amount of memory for the activation vector generated by each operator can be obtained by calling the accelerator's memory usage detection instruction and comparing the accelerator's memory usage before and after the operator is executed. The changing trend of the memory usage corresponding to the activation vector can be determined by judging the type of the current training phase. Among them, when the type of the current training phase is forward propagation, it can be determined that the changing trend of the memory usage corresponding to the activation vector is an increasing trend. When the type of the current training phase is backward propagation, it can be determined that the changing trend of the memory usage corresponding to the activation vector is a decreasing trend. Optionally, since the fine-tuning training of the large language model is a periodically repeated load, the determined memory information can be reused in subsequent repeated cycles to prevent the introduction of additional overhead.

[0072] Step S40: Adjust the memory space on the accelerator for fine-tuning the large language model according to the memory information.

[0073] In one possible implementation, after determining the memory information, the electronic device can adjust the activation vector space and the parameter space in the memory space according to the change trend therein, wherein the adjustment of the activation vector space can adjust the activation vector space according to the amount of memory included therein. Specifically, the parameters of the operator include local parameters and remote parameters sent by other accelerators, and the parameter space may include a static cache space and a dynamic cache space, the static cache space is used to store local parameters, and the dynamic cache space is used to store remote parameters sent by other accelerators. The local parameters cached in the static cache space can determine the parameter ratio in each accelerator by comparing the parameter size of the large language model and the memory size of the accelerator before the training starts. This part of the local parameters will always be stored in the static cache space of the accelerator. The remote parameters cached in the dynamic cache space are the parameters transmitted by other accelerators stored in the dynamic cache space when the activation vector occupies less memory during the training process of the large language model, and are used for reuse in subsequent use.

[0074] Optionally, during the memory space adjustment process, the static cache space remains unchanged. That is, if the trend is an increase in memory usage, the allocation of static cache space in the memory space can be maintained, the allocation of dynamic cache space in the memory space can be reduced, and the allocation of activation vector space in the memory space can be increased. If the trend is a decrease in memory usage, the allocation of static cache space in the memory space can be maintained, the allocation of activation vector space in the memory space can be reduced, and the allocation of dynamic cache space in the memory space can be increased.

[0075] Furthermore, in cases where the activation vector space needs to be adjusted in the embodiments of the present disclosure, this can be achieved by adjusting the dimensions of the batch size and changing the memory usage of individual operators. The batch size refers to the size of the data set input for each iteration. The memory usage of the activation vectors corresponding to each operator can be adjusted by first adjusting the dimensions of the batch size used to store the activation vectors in the memory space. The adjustment trend can be determined based on the current total memory volume of the memory space in the accelerator (i.e., the remaining memory volume in the current memory space). For example, if the total memory volume is large, the dimensions of the batch size can be lowered; if the total memory volume is small, the dimensions of the batch size can be increased. It is then determined whether the activation vectors corresponding to each operator need to be saved. Based on the memory volume corresponding to the activation vectors that need to be saved, the activation vector space in the memory space is adjusted so that the adjusted activation vector space can store the memory volume corresponding to the activation vectors that need to be saved, for example, the minimum space that can store the corresponding memory volume. The overall memory usage can be increased or decreased by changing the dimensions of the batch size of the activation vectors corresponding to some operators. The activation vectors that are not saved can be recalculated later when needed. The rules for whether to save the activation vectors can be selected as needed, and this application does not impose any restrictions on this.

[0076] Step S50: Store the activation vector corresponding to each operator in the activation vector space.

[0077] In one possible implementation, after adjusting the storage space of the accelerator, the electronic device stores the activation vectors generated by each operator during training in the activation vector space within the adjusted storage space. Each activation vector can be stored in the adjusted batch size dimension.

[0078] Step S60: Store the parameters of each operator in the parameter space, and complete the iterative processing based on the parameters in the parameter space.

[0079] In one possible implementation, after adjusting the accelerator's storage space, the electronic device can also store the parameters of each operator sent by other accelerators in the parameter space, so that in subsequent iterations, the iterative processing process can be completed based on the parameters in the parameter space. That is, for a certain iterative processing process, the steps of this embodiment can be performed during the first iteration to adjust the memory space on the accelerator, store the activation vector generated by the first iteration process, and the operator parameters used. In subsequent iterations, the iterative processing is performed using the stored operator parameters, and the generated activation vectors are continued to be stored using the adjusted memory space until the iterative processing process is completed.

[0080] Among them, due to the limitation of the parameter space size, the remote parameters sent by other accelerators may not be completely stored in the dynamic cache space in the parameter space. Therefore, the electronic device can store at least part of the remote parameters sent by other accelerators in the dynamic cache space. Exemplarily, when the remaining space of the dynamic cache space is greater than or equal to the memory usage of the remote parameters, all the remote parameters can be stored in the dynamic cache space. When the remaining space of the dynamic cache space is less than the memory usage of the remote parameters, part of the remote parameters can be stored in the dynamic cache space. The remote parameters that cannot be stored in the dynamic cache space can be obtained through communication with other accelerators. Compared with obtaining all remote parameters through communication with other accelerators, this method reduces the transmission amount of remote parameters between accelerators. That is, by storing all or part of the remote parameters in the local dynamic cache space of the accelerator and directly using the remote parameters in the local dynamic cache space in the subsequent iteration process, the communication overhead between accelerators is reduced and the overall throughput is improved.

[0081] Optionally, when the electronic device subsequently reduces the allocation of dynamic cache space in the memory space, the electronic device can release some of the remote parameters stored in the dynamic cache space. In other words, during the training of a large language model, when the activation vector occupies less memory, the electronic device can use the free dynamic cache space in the accelerator to store the received remote parameters, and reuse them in subsequent iterations to reduce the overall communication volume. At the same time, when the memory occupancy of the activation vector becomes larger, this part of the cache can be dynamically released. Based on the above technical features, the embodiment of the present disclosure can adaptively cache parameters according to the activation vector and the memory situation in the accelerator.

[0082] Figure 2 A schematic diagram showing a fine-tuning process of a pre-trained large language model according to an embodiment of the present disclosure is shown. Figure 2 As shown,

[0083] The disclosed embodiments can be implemented using a specific system comprising a memory detection module, a memory transformation module, and a parameter cache module. This system can operate on hardware consisting of several computers, a GPU computing accelerator, and a computer network connecting them. Before training a large language model, the system is fed with pre-trained base model parameters, adapter model parameters to be trained, and the data the user wishes to train. During the training of the large language model, after the first iteration of each iterative processing step, the memory detection module detects the memory size occupied by the activation data generated by each operator to determine memory information. The memory transformation module is then used to determine the specific execution plan for each operator, including setting the batch size and determining whether to retain or discard computation results. The accelerator's memory space is then adjusted. Finally, the parameter cache module caches model parameters both statically and dynamically, reusing them in subsequent iterative processes to reduce communication traffic and improve model throughput. The strategies in the memory transformation and parameter cache modules search for changes during execution, ultimately achieving efficient fine-tuning.

[0084] Figure 3 FIG2 shows an application effect diagram of a fine-tuning method for a pre-trained large language model according to an embodiment of the present disclosure. Figure 3 As shown, the fine-tuning method of the pre-trained large language model of the embodiment of the present disclosure is used on the NVIDIAA100 GPU and V100 GPU platform accelerators respectively. Compared with other methods, the processing speed of the embodiment of the present disclosure when fine-tuning different pre-trained large language models is faster, achieving an acceleration effect.

[0085] Based on the above technical features, the embodiment of the present disclosure can automatically adjust the memory usage of the accelerator to allocate free dynamic storage space for the received parameters by real-time detection of the memory usage of the activation vector during the training process of the large language model, so as to automatically cache the required parameters, reduce communication overhead through parameter reuse in subsequent repetition cycles, and improve overall throughput.

[0086] Figure 4 A schematic diagram of a fine-tuning device for a pre-trained large language model according to an embodiment of the present disclosure is shown. Figure 4 As shown, the fine-tuning device for the pre-trained large language model according to the embodiment of the present disclosure may include:

[0087] An information determination module 40 is configured to determine at least one iterative process in a large language model training process, each of the iterative processes corresponding to at least one operator, wherein the large language model is a pre-trained model;

[0088] The data processing module 41 is configured to receive, for each iterative processing process, parameters of each corresponding operator in the iterative processing process stored in other accelerators, complete the first iterative processing, and generate / consume the activation vector corresponding to each operator;

[0089] A memory detection module 42 is configured to determine memory information corresponding to the first iterative processing after the first iterative processing is completed;

[0090] a memory transformation module 43, configured to adjust a memory space on the accelerator used for fine-tuning the large language model according to the memory information, wherein the memory space of the accelerator includes an activation vector space and a parameter space other than the activation vector space;

[0091] A vector storage module 44 is configured to store the activation vector corresponding to each operator in the activation vector space;

[0092] The parameter storage module 45 is used to store the parameters of each operator in the parameter space and complete the iterative processing based on the parameters in the parameter space.

[0093] In a possible implementation, the memory detection module 42 is further configured to:

[0094] Determining the amount of memory corresponding to the activation vector generated by each of the operators;

[0095] Determining a change trend of memory usage corresponding to the activation vector;

[0096] Corresponding memory information is determined according to the memory amount and the memory change trend.

[0097] In a possible implementation, the memory transformation module 43 is further configured to:

[0098] Adjusting the amount of memory corresponding to the activation vector of each operator by adjusting the batch size dimension of the activation vector stored in the memory space, wherein the adjustment trend is determined by the current total amount of memory in the accelerator;

[0099] It is determined whether the activation vector corresponding to each of the operators needs to be saved, and the activation vector space in the memory space is adjusted according to the amount of memory corresponding to the activation vectors that need to be saved.

[0100] In one possible implementation, the parameters include local parameters and remote parameters sent by other accelerators, and the parameter space includes a static cache space and a dynamic cache space. The static cache space is used to store local parameters, and the dynamic cache space is used to store remote parameters sent by other accelerators.

[0101] In a possible implementation, the parameter storage module 45 is further configured to:

[0102] At least part of the remote parameters sent by other accelerators is stored in the dynamic cache space.

[0103] In a possible implementation, the memory transformation module 43 is further configured to:

[0104] In response to the change trend being an increase in memory usage, maintaining the allocation of static cache space in the memory space, reducing the allocation of dynamic cache space in the memory space, and increasing the allocation of activation vector space in the memory space;

[0105] In response to the change trend being a decrease in memory usage, the allocation of static cache space in the memory space is maintained, the allocation of activation vector space in the memory space is reduced, and the allocation of dynamic cache space in the memory space is increased.

[0106] In a possible implementation, the apparatus further includes:

[0107] The parameter releasing module is configured to release part of the remote parameters stored in the dynamic cache space in response to reducing the allocation of the dynamic cache space in the memory space.

[0108] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0109] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0110] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0111] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0112] Figure 51 shows a schematic diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. Figure 5 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0113] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0114] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0115] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0116] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0117] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0118] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0119] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0120] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0121] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0122] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0123] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A fine-tuning method for a pre-trained large language model, characterized in that: An accelerator for fine-tuning a large language model, the method comprising: Determining at least one iterative processing process in a large language model training process, each of the iterative processing processes corresponds to at least one operator, and the large language model is a pre-trained model; For each of the iterative processing processes, receiving parameters of each corresponding operator of the iterative processing process stored in other accelerators and completing the first iterative processing, generating / consuming activation vectors corresponding to each of the operators; After completing the first iterative process, determining memory information corresponding to the first iterative process; adjusting a memory space on the accelerator for fine-tuning the large language model according to the memory information, wherein the memory space of the accelerator includes an activation vector space and a parameter space other than the activation vector space; Storing the activation vector corresponding to each of the operators in the activation vector space; The parameters of each operator are stored in the parameter space, and the iterative processing process is completed based on the parameters in the parameter space.

2. The method according to claim 1, characterized in that The determining of memory information corresponding to the first iterative processing includes: Determining the amount of memory corresponding to the activation vector generated by each of the operators; Determining a change trend of memory usage corresponding to the activation vector; Corresponding memory information is determined according to the memory amount and the memory change trend.

3. The method according to claim 1 or 2, characterized in that The adjusting, according to the memory information, the memory space on the accelerator for fine-tuning the large language model comprises: Adjusting the amount of memory corresponding to the activation vector of each operator by adjusting the batch size dimension of the activation vector stored in the memory space, wherein the adjustment trend is determined by the current total amount of memory in the accelerator; It is determined whether the activation vector corresponding to each of the operators needs to be saved, and the activation vector space in the memory space is adjusted according to the amount of memory corresponding to the activation vectors that need to be saved.

4. The method according to claim 3, characterized in that The parameters include local parameters and remote parameters sent by other accelerators. The parameter space includes a static cache space and a dynamic cache space. The static cache space is used to store local parameters, and the dynamic cache space is used to store remote parameters sent by other accelerators.

5. The method according to claim 4, characterized in that Storing the parameters of each operator in the parameter space includes: At least part of the remote parameters sent by other accelerators is stored in the dynamic cache space.

6. The method according to claim 2, characterized in that The adjusting, according to the memory information, the memory space on the accelerator for fine-tuning the large language model comprises: In response to the change trend being an increase in memory usage, maintaining the allocation of static cache space in the memory space, reducing the allocation of dynamic cache space in the memory space, and increasing the allocation of activation vector space in the memory space; In response to the change trend being a decrease in memory usage, the allocation of static cache space in the memory space is maintained, the allocation of activation vector space in the memory space is reduced, and the allocation of dynamic cache space in the memory space is increased.

7. The method according to any one of claims 4, characterized in that The method further comprises: In response to reducing the allocation of the dynamic cache space in the memory space, part of the remote parameters stored in the dynamic cache space is released.

8. A fine-tuning device for pre-training a large language model, characterized in that: An accelerator for fine-tuning a large language model, the apparatus comprising: an information determination module, configured to determine at least one iterative processing process in a large language model training process, each of the iterative processing processes corresponding to at least one operator, wherein the large language model is a pre-trained model; A data processing module is configured to receive, for each iterative processing process, parameters of each corresponding operator of the iterative processing process stored in other accelerators, complete the first iterative processing, and generate / consume an activation vector corresponding to each of the operators; A memory detection module, configured to determine memory information corresponding to the first iterative processing after the first iterative processing is completed; a memory transformation module, configured to adjust a memory space on the accelerator used for fine-tuning the large language model according to the memory information, wherein the memory space of the accelerator includes an activation vector space and a parameter space other than the activation vector space; A vector storage module, configured to store the activation vector corresponding to each of the operators in the activation vector space; A parameter storage module is used to store the parameters of each operator in the parameter space and complete the iterative processing process based on the parameters in the parameter space.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images