Large model fine-tuning training method and device, equipment, storage medium and product

Abstract

The invention discloses a large model fine tuning training method, device and equipment, a storage medium and a product, and relates to the technical field of network intelligence, and the method comprises the steps: inputting a label data set into a large model through end side equipment for forward propagation calculation, and transmitting an output activation value between each Trm layer to auxiliary computing power node equipment; and the auxiliary computing power node equipment fuses the output activation value with the output of each layer in the bypass network adapter to execute forward propagation calculation, and the end side equipment fuses the target output with the output of the last Trm layer to obtain model output. And determining a loss value according to the model output and the local tag data set, so that the auxiliary computing power node equipment performs back propagation parameter updating according to the loss value. Therefore, a backbone network passing through the end-side device is avoided, the total memory overhead in the training process is reduced, the memory pressure of the end-side device is effectively migrated to the auxiliary computing power node, and the end-side device with the limited memory efficiently executes a large-model fine tuning task.

Description

Technical Field

[0001] The present application relates to the field of network intelligence technology, and in particular to a large-model fine-tuning training method, device, equipment, storage medium and product. Background Art

[0002] In recent years, general large model technology based on Transformer structure has made significant progress in the field of artificial intelligence and has demonstrated excellent performance in a variety of tasks such as natural language processing and computer vision. These large models have acquired powerful generalization representation capabilities through pre-training on large-scale data sets, and adapted to specific tasks in the fine-tuning stage, achieving excellent performance in multiple benchmarks. Among them, the large model fine-tuning stage relies on sample data in a specific field. This data is widely distributed on end-side devices (such as smartphones, IoT devices, etc.) and is continuously generated in the interactive response process with users. In order to effectively utilize distributed data while protecting data privacy, using end-side devices to perform large model fine-tuning intelligent computing tasks has become a current hot technology.

[0003] However, the existing methods for performing large model fine-tuning on end-side devices require a large amount of memory to store intermediate results such as activation values, gradients, and optimizer states. This poses a huge challenge to the memory capacity of the end-side devices, making it easy for memory overflow problems to occur during model loading and data processing, resulting in abnormal interruption of the fine-tuning task. Summary of the invention

[0004] The main purpose of this application is to provide a large model fine-tuning training method, device, equipment, storage medium and product, aiming to solve the technical problem that the existing large model local fine-tuning tasks cannot be performed on terminal devices with limited memory.

[0005] To achieve the above objectives, the present application proposes a large model fine-tuning training method, which is applied to a terminal device, in which a quantized and compressed pre-trained large model is deployed, and the method includes:

[0006] Input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in a manner of fusing the output activation values ​​with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device;

[0007] Receive the target output fed back by the auxiliary computing node device, and fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output;

[0008] A loss value is determined based on the model output and the local label data set, and the loss value is transmitted to the auxiliary computing power node device, so that the auxiliary computing power node device performs back-propagation parameter update on the bypass network adapter based on the loss value.

[0009] In one embodiment, the step of transmitting the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing node device includes:

[0010] Iteratively determine the target quantization bit width based on the total amount of data that can be transmitted, the output activation values ​​between each Transformer layer, and the quantization error caused by the gradual increase in the quantization level of the output activation value;

[0011] Performing low-bit-width quantization on the output activation values ​​between each Transformer layer in the pre-trained large model according to the target quantization bit width to obtain quantized output activation values;

[0012] The quantized output activation value is transmitted to the auxiliary computing power node device.

[0013] In one embodiment, the step of iteratively determining the target quantization bit width according to the total amount of data that can be transmitted, the output activation values ​​between each Transformer layer, and the quantization error caused by the gradual increase in the quantization level of the output activation value includes:

[0014] Determine the total amount of data that can be transmitted according to the uplink rate from the terminal device to the auxiliary computing power node device at the current iteration time and the preset transmission delay threshold;

[0015] Determine the trace of the average Fisher information matrix of each layer of adapters in the bypass network adapter according to the gradient information of the parameters to be updated in the bypass network adapter corresponding to each Transformer layer in the pre-trained large model;

[0016] Initialize the quantization level of the quantization bit width of the output activation value between each Transformer layer to the lowest level, and determine the remaining transmittable data amount according to the output activation value corresponding to the current number of Transformer layers at the current iteration number, the quantization level and the total transmittable data amount;

[0017] Determine the utility increment according to the product of the quantization error caused by increasing the quantization level by one level and the trace of the average Fisher information matrix;

[0018] Iteratively determine the target number of Transformer layers that maximizes the utility increment, and update the remaining transmittable data amount according to the target number of layers and the quantization level after increasing by one level, until the remaining transmittable data amount is less than or equal to 0, and obtain the target quantization bit width.

[0019] In one embodiment, after the step of inputting the sampled label data set into the pre-trained large model for forward propagation calculation, and transmitting the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, it also includes:

[0020] The local label data is sampled, and the sample serial number corresponding to the sampled label data set is transmitted to the auxiliary computing node device, so that the auxiliary computing node device queries the database whether there is an activation value corresponding to the sample serial number. If the sample serial number does not exist, the sample serial number is cached in the database.

[0021] In one embodiment, after the step of caching the sample serial number in the database if the sample serial number does not exist, the method further includes:

[0022] If the sample sequence number already exists, the average amount of data that can be transmitted per layer of a single sample is determined according to the uplink rate from the end-side device to the auxiliary computing power node device at the current iteration time and the preset transmission delay threshold;

[0023] Determine the average quantization bit width of a single intermediate activation value according to the intermediate activation value dimension corresponding to the pre-trained large model and the average transmittable data amount;

[0024] Determine whether a condition that a difference between the average quantization bit width and the cached average quantization bit width of activation values ​​of each layer corresponding to the sample sequence number is greater than or equal to a preset quantization bit width difference threshold is met;

[0025] If the condition is met, the activation value corresponding to the sample number in the cache is cleared, and the activation value corresponding to the sample number is updated.

[0026] In one embodiment, after the step of clearing the activation value corresponding to the sample number in the cache and updating the activation value corresponding to the sample number if the condition is met, the method further includes:

[0027] If the condition is not met, the sample number and the activation value corresponding to the sample number in the cache are cleared, and the terminal device is instructed to skip the forward propagation calculation and activation value transmission operations on the sampled label data set corresponding to the sample number.

[0028] In addition, to achieve the above-mentioned purpose, the present application also proposes a large model fine-tuning training device, wherein a pre-trained large model after quantization and compression is deployed in the end-side device, and the device is applied to the end-side device, and the large model fine-tuning training device includes:

[0029] A forward propagation calculation module, used to input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in a manner of fusing the output activation values ​​with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device;

[0030] A model output fusion module, used to receive the target output fed back by the auxiliary computing node device, and fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output;

[0031] A back propagation parameter update module is used to determine a loss value based on the model output and a local label data set, and transmit the loss value to the auxiliary computing power node device, so that the auxiliary computing power node device performs a back propagation parameter update on the bypass network adapter according to the loss value.

[0032] In addition, to achieve the above-mentioned objectives, the present application also proposes a large model fine-tuning training device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the large model fine-tuning training method as described above.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the large model fine-tuning training method described above are implemented.

[0034] In addition, to achieve the above-mentioned objectives, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the large model fine-tuning training method described above.

[0035] The present application provides a large model fine-tuning training method, which inputs the sampled label data set into the pre-trained large model for forward propagation calculation by deploying a terminal device with a quantized and compressed pre-trained large model, and transmits the output activation value between each Trm layer in the pre-trained large model to the auxiliary computing power node device. The auxiliary computing power node device fuses the output activation value with the output of each layer in the bypass network adapter, performs forward propagation calculation, and feeds back the target output. The terminal device fuses the target output with the output of the last layer of the Trm layer to obtain the model output, determines the loss value based on the model output and the local label data set, so that the auxiliary computing power node device performs back-propagation parameter updates on the bypass network adapter based on the loss value. This avoids parameter updates passing through the backbone network of the terminal device, reduces the total memory overhead of the training process, and at the same time realizes the effective migration of the terminal device memory and computing pressure to the auxiliary computing power node, so that the memory-constrained terminal device can efficiently perform large model fine-tuning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0038] Figure 1 A flowchart of the first embodiment of the large model fine-tuning training method of the present application is provided;

[0039] Figure 2 A schematic diagram of the interactive process of the large model fine-tuning training method of this application;

[0040] Figure 3 This is a schematic diagram of the structure of the bypass network adapter in the auxiliary computing node device of this application;

[0041] Figure 4 A flowchart diagram of the second embodiment of the large model fine-tuning training method of the present application;

[0042] Figure 5 A flowchart diagram of the third embodiment of the large model fine-tuning training method of the present application;

[0043] Figure 6 This is a schematic diagram of the module structure of the large model fine-tuning training device according to an embodiment of the present application;

[0044] Figure 7This is a schematic diagram of the device structure of the hardware operating environment involved in the large model fine-tuning training method in the embodiment of the present application.

[0045] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0047] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0048] The main solution of the embodiment of the present application is: the end-side device inputs the sampled label data set into the pre-trained large model for forward propagation calculation, and transmits the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in a manner of fusing the output activation value with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device; receives the target output fed back by the auxiliary computing power node device, and fuses the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output; determines the loss value according to the model output and the local label data set, and transmits the loss value to the auxiliary computing power node device, so that the auxiliary computing power node device updates the back-propagation parameters of the bypass network adapter according to the loss value.

[0049] The existing general large model technology based on the Transformer structure has made significant progress in the field of artificial intelligence and has demonstrated excellent performance in a variety of tasks such as natural language processing and computer vision. These large models have obtained powerful generalization representation capabilities through pre-training on large-scale data sets, and are adapted to specific tasks in the fine-tuning stage, achieving excellent performance in multiple benchmarks. Among them, the large model fine-tuning stage relies on sample data in a specific field, which is widely distributed on end-side devices (such as smartphones, IoT devices, etc.) and continuously generated in the interactive response process with users. In order to effectively utilize distributed data while protecting data privacy, using end-side devices to perform large model fine-tuning intelligent computing tasks has become a current hot technology. However, the existing methods for end-side devices to perform large model fine-tuning require a large amount of memory to store intermediate results such as activation values, gradients, and optimizer states, which will pose a huge challenge to the memory capacity of the end-side device, making it easy to have memory overflow problems during model loading and data processing, resulting in abnormal interruption of fine-tuning tasks.

[0050] The present application provides a solution, which is to input the sampled label data set into the pre-trained large model for forward propagation calculation by deploying a terminal device with a quantized and compressed pre-trained large model, and transmit the output activation value between each Trm layer in the pre-trained large model to the auxiliary computing power node device. The auxiliary computing power node device fuses the output activation value with the output of each layer in the bypass network adapter, performs forward propagation calculation, and feeds back the target output. The terminal device fuses the target output with the output of the last layer of the Trm layer to obtain the model output, determines the loss value based on the model output and the local label data set, so that the auxiliary computing power node device performs back-propagation parameter updates on the bypass network adapter based on the loss value. This avoids the parameter update passing through the backbone network of the terminal device, reduces the total memory overhead of the training process, and realizes the effective migration of the terminal device memory and computing pressure to the auxiliary computing power node, so that the memory-constrained terminal device can efficiently perform large model fine-tuning tasks.

[0051] It should be noted that the execution subject of the method of this embodiment can be a memory-constrained terminal device deployed with a quantized and compressed pre-trained large model, such as a smart phone, an Internet of Things device, etc.; it can also be a large model fine-tuning training device with the same or similar functions. This embodiment and the following embodiments will be described using a large model fine-tuning training device as an example.

[0052] Based on this, the present application embodiment provides a large model fine-tuning training method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the large model fine-tuning training method of the present application.

[0053] In this embodiment, the method is applied to a terminal side device, in which a pre-trained large model after quantization and compression is deployed. The large model fine-tuning training method includes steps S10 to S30:

[0054] Step S10, input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in a manner of fusing the output activation values ​​with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device.

[0055] It can be understood that the end-side device refers to a terminal or node close to the data source and the user. These devices have certain computing power and data processing functions, and can quickly respond to and process locally generated data, thereby reducing network load and improving system efficiency. This embodiment provides an end-side memory-friendly large model fine-tuning method, which can be applied to edge computing networks such as smart homes. Based on the collaborative computing of neighboring high-performance computing nodes, the end-side device that supports loading a large model performs Transformer large model fine-tuning training tasks based on its locally continuously generated user label data.

[0056] Before starting the fine-tuning task, combine Figure 2 The structure of the end-side device, the structure of the auxiliary computing node device, and the interactive training process between the two are explained. The end-side device determines the quantization level of the large model based on its own hardware capabilities such as computing power and memory capacity, and locally loads the complete backbone network of the pre-trained large model that has been quantized and compressed with a specific bit width, including an input embedding layer (Embedding Layer, Embed), a group of serially connected Transformer layers (Trm), and an output head layer (HeadLayer, Hed). The auxiliary computing node device (with higher memory and computing power resources) initializes a bypass network, which consists of a group of stacked parallel adapter modules. The input of each adapter module is obtained by fusion of the output of the previous layer adapter module and the activation value output of the corresponding layer of the end-side backbone network. Each bypass network adapter module adopts the same structure, including a dimensionality reduction mapping layer, a nonlinear layer, a dimensionality increase mapping layer, and an addition layer. The structure of the adapter can refer to Figure 3 .

[0057] It should be noted that during the fine-tuning training process, the end-side device samples the local labeled data set to obtain a representative data subset. This data subset is then used as input and input into the backbone network of the local pre-trained large model to start an iterative forward propagation calculation. During the fine-tuning process, user label data is allowed to be maintained locally on the end-side device without sharing between devices, thereby improving data privacy protection capabilities. At the same time, the activation values ​​between each Transformer layer are quantized and encoded with low bit width and transmitted to the computing power node, and respectively input into the adapters of the corresponding layers of the stacked structure for forward propagation calculation, and the target output of the last layer of adapters is returned to the end-side device.

[0058] Step S20, receiving the target output fed back by the auxiliary computing node device, and fusing the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output.

[0059] It can be understood that the terminal device receives the target output fed back by the auxiliary computing node device, fuses it with the output of the last layer Transformer of the backbone network, and then inputs the fusion into the head layer to obtain the model output.

[0060] Step S30, determining a loss value based on the model output and the local label data set, and transmitting the loss value to the auxiliary computing power node device, so that the auxiliary computing power node device performs back propagation parameter update on the bypass network adapter according to the loss value.

[0061] It should be understood that the end-side device determines the loss value based on the model output obtained by the head layer combined with the local label data set, and transmits the loss value back to the computing power node, and back-propagates the parameters of the bypass network adapter to update the parameters, thereby avoiding the backbone network of the end-side device, reducing the total memory overhead of the training process, and effectively migrating the end-side device memory and computing pressure to the auxiliary computing power node. The large model fine-tuning network is effectively divided into a parallel frozen state backbone network and an activated state bypass network. During the fine-tuning process, only the bypass parameters in the activated state are trained and updated, while the original Trm module is always kept in an inactive state, and no parameter update calculation is performed, which shortens the calculation time and reduces the memory overhead introduced by storing intermediate parameters.

[0062] This embodiment provides a large model fine-tuning training method, which inputs the sampled label data set into the pre-trained large model for forward propagation calculation by deploying a terminal device with a quantized and compressed pre-trained large model, and transmits the output activation value between each Trm layer in the pre-trained large model to the auxiliary computing power node device. The auxiliary computing power node device fuses the output activation value with the output of each layer in the bypass network adapter, performs forward propagation calculation, and feeds back the target output. The terminal device fuses the target output with the output of the last layer of the Trm layer to obtain the model output, determines the loss value based on the model output and the local label data set, so that the auxiliary computing power node device performs back-propagation parameter update on the bypass network adapter based on the loss value. This avoids the parameter update passing through the backbone network of the terminal device, reduces the total memory overhead of the training process, and realizes the effective migration of the terminal device memory and computing pressure to the auxiliary computing power node, so that the memory-constrained terminal device can efficiently perform large model fine-tuning tasks.

[0063] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 4 , step S10, the large model fine-tuning training method further includes steps S101 to S103:

[0064] Step S101: iteratively determine the target quantization bit width according to the total amount of data that can be transmitted, the output activation values ​​between each Transformer layer, and the quantization error caused by the gradual increase in the quantization level of the output activation value.

[0065] It can be understood that an importance-aware adaptive activation value quantization strategy is proposed in this embodiment. During the execution of the fine-tuning task, the auxiliary computing power node performs an activation value quantization bit width decision every k training iterations, that is, a suitable quantization bit width is selected for each layer of Trm output activation value, and the activation value is quantized by the quantization bit width and transmitted to the auxiliary computing power node device. After the quantization level of the output activation value can be initialized, the quantization level can be gradually increased iteratively, and the optimal target quantization bit width is iteratively determined according to the quantization error caused by the increase in the quantization level and the total amount of data that can be transmitted.

[0066] In a feasible implementation, step S101 may include steps S1011 to S1015:

[0067] Step S1011, determining the total amount of data that can be transmitted based on the uplink rate from the terminal device to the auxiliary computing power node device at the current iteration moment and the preset transmission delay threshold.

[0068] It is worth noting that at the tth iteration, the output activation value of the lth layer Trm is , measure the uplink rate R from the end-side device to the auxiliary computing node at the current iteration time t t ; Combined with the uplink rate R t and the preset transmission delay threshold T th Calculate the total amount of data that can be transmitted R max , where T th The value setting of should take into account the update calculation time of the auxiliary computing node for the bypass network, and try to shorten its computing idle time.

[0069] Step S1012, determining the trace of the average Fisher information matrix of each layer of adapters in the bypass network adapter according to the gradient information of the parameters to be updated in the bypass network adapter corresponding to each Transformer layer in the pre-trained large model.

[0070] It can be understood that based on the gradient information of the parameters to be updated in the bypass network corresponding to each Trm layer, the trace of the average Fisher information matrix of each layer adapter in the bypass network adapter is calculated (denoted as tr(F)). The larger the tr(F) value, the more important the adapter is in the current training stage.

[0071] Step S1013, initialize the quantization level of the quantization bit width of the output activation value between each Transformer layer to the lowest level, and determine the remaining transmittable data amount according to the output activation value corresponding to the current number of Transformer layers at the current iteration number, the quantization level and the total transmittable data amount.

[0072] It should be understood that the quantization bit width of the output activation value of each layer Trm is initialized to the lowest level, that is, the quantization level Calculate the remaining amount of data that can be transmitted R res :

[0073]

[0074] Step S1014: determining the utility increment according to the product of the quantization error caused by increasing the quantization level by one level and the trace of the average Fisher information matrix.

[0075] It can be understood that, for any Trm layer, the utility increment function is defined as the product of the reduction in quantization error caused by increasing its quantization bit width by one level and the trace of its average Fisher information matrix. The utility increment function is as follows:

[0076]

[0077] in, Represents the activation value Use quantization level Quantization error introduced by quantization.

[0078] Step S1015, iteratively determine the target number of Transformer layers that maximizes the utility increment, and update the remaining transmittable data amount according to the target number of layers and the quantization level after increasing by one level, until the remaining transmittable data amount is less than or equal to 0, and obtain the target quantization bit width.

[0079] It can be understood that the search for the Trm layer l that maximizes the utility increment function value is to quantize its bit width level from Increase to And update the remaining available transmission data amount. Iterate and repeat the above operations in this step until the remaining transmittable data amount is not greater than 0, or the quantization bit width of the output activation values ​​of all layers Trm reaches the highest level, and output the target quantization bit width of the output activation value of each layer Trm at this time.

[0080] Step S102: performing low-bit-width quantization on the output activation values ​​between the Transformer layers in the pre-trained large model according to the target quantization bit width to obtain quantized output activation values.

[0081] Step S103: Transmit the quantized output activation value to the auxiliary computing power node device.

[0082] In this embodiment, the target quantization bit width is iteratively determined according to the total amount of data that can be transmitted, the output activation values ​​between each Transformer layer, and the quantization error caused by the gradual increase in the quantization level of the output activation value. The output activation values ​​between each Transformer layer in the pre-trained large model are quantized at a low bit width according to the target quantization bit width to obtain the quantized output activation value, and the quantized output activation value is transmitted to the auxiliary computing power node device. In the above manner, a high-precision quantization strategy can be selected for more important output activation values ​​under the transmission time constraint to alleviate the quantization loss introduced by it, thereby helping to improve the training convergence performance. The end-side device adopts an activation value adaptive quantization mechanism, which captures the importance of different Trm layers to the convergence process by online sensing of the training state and channel conditions, and reasonably allocates the quantization bit width of the output activation value of each layer, so as to save the device transmission resource overhead without sacrificing the convergence speed and fine-tuning performance, and accelerate the fine-tuning process.

[0083] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 5 After step S10, the large model fine-tuning training method further includes step A10:

[0084] Step A10: Sample the local label data, and transmit the sample serial number corresponding to the sampled label data set to the auxiliary computing node device, so that the auxiliary computing node device queries the database whether there is an activation value corresponding to the sample serial number. If the sample serial number does not exist, the sample serial number is cached in the database.

[0085] It is worth noting that, in view of the fixed parameters of the end-side pre-trained backbone model during the fine-tuning training process, this embodiment caches the output activation values ​​of each Trm layer with a specific quantization bit width received by the auxiliary computing node, together with its corresponding data sample number, thereby effectively reducing the computational and transmission overhead introduced by the repeated forward propagation and activation value transmission of the same extracted samples by the end-side device. Taking into account the node-side storage capacity constraints and the intermediate activation value dimensions of the large model, the cache refresh cycle is manually set, such as a training round (epoch), that is, after each cache refresh cycle, the auxiliary computing node clears the cached activation values ​​in the storage space and restarts the cache and update process.

[0086] It is understandable that before each iteration starts, the auxiliary computing node preloads the cached activation values ​​of the samples required for this iteration from the hard disk to the memory for access and use. The auxiliary computing node receives the sample serial numbers of N groups of data batches randomly generated by the end-side device, and queries the storage database for each sample serial number to see whether there is a corresponding activation value. For samples that do not exist, the sample serial number is cached in the current sample sequence. In each iteration, the auxiliary computing node receives the output activation values ​​of each Trm layer with a specific quantization bit width related to the valid sample sequence from the end-side device, and stores them in the hard disk together with their corresponding data sample serial numbers. The storage format example is shown in the following table.

[0087] Sample ID Layer number Quantized intermediate activation values xx xx xx

[0088] In a feasible implementation manner, after step A10, steps A20 to A50 may also be included:

[0089] Step A20: If the sample sequence number already exists, the average transmittable data volume of a single sample and a single layer is determined based on the uplink rate from the terminal device to the auxiliary computing power node device at the current iteration moment and the preset transmission delay threshold.

[0090] It should be noted that for the sample numbers that already exist in the database, further judgment is performed. First, the uplink rate R from the end-side device to the auxiliary computing power node at the current iteration time is measured. t , based on the preset transmission delay threshold T th , calculate the average amount of data that can be transmitted per sample and per layer:

[0091]

[0092] Where B is the data batch size.

[0093] Step A30, determining the average quantization bit width of a single intermediate activation value according to the intermediate activation value dimension corresponding to the pre-trained large model and the average transmittable data amount.

[0094] It can be understood that the average quantization bit width of a single intermediate activation value is determined according to the intermediate activation value dimension and the average amount of data transmitted corresponding to the pre-trained large model, and is recorded as

[0095] Step A40, determining whether the condition that the difference between the average quantization bit width and the cached average quantization bit width of activation values ​​of each layer corresponding to the sample sequence number is greater than or equal to a preset quantization bit width difference threshold is met.

[0096] It should be understood that the judgment condition Is it true, where b cache is the average quantization bit width of the cached activation values ​​of each layer for this sample, Δb is the preset quantization bit width difference threshold.

[0097] Step A50: if the condition is met, clear the activation value corresponding to the sample number in the cache, and update the activation value corresponding to the sample number.

[0098] It should be understood that if the condition is met, the sample entry corresponding to the sample serial number in the storage database and the activation value contained in the entry are cleared, and the sample serial number is retained in the current sample sequence.

[0099] In a feasible implementation manner, after step A50, step A60 may also be included:

[0100] Step A60: If the condition is not met, the sample number and the activation value corresponding to the sample number in the cache are cleared, and the terminal device is instructed to skip the forward propagation calculation and activation value transmission operation of the sampled label data set corresponding to the sample number.

[0101] It is worth noting that if the condition is not met, the sample number and the activation value corresponding to the sample number in the cache are cleared to receive the activation value of a higher quantization level for cache update. And instruct the end-side device to skip the forward propagation calculation and quantization activation value transmission operation of the sampled label data set corresponding to the sample number. The end-side device extracts the remaining samples that do not exist in the storage database and performs the forward calculation of this iteration. At the end of the current cache refresh cycle, the auxiliary computing node clears all cached activation values ​​to release storage space.

[0102] In this embodiment, by sampling the local label data, the sample serial number corresponding to the sampled label data set is transmitted to the auxiliary computing node device, so that the auxiliary computing node device queries whether there is an activation value corresponding to the sample serial number in the database, and adopts a corresponding strategy according to the query result, thereby effectively reducing the computational and transmission overhead introduced by the repeated forward propagation and activation value transmission of the same extracted sample by the end-side device, and at the same time, when the real-time transmission conditions are good, it can flexibly control the use of activation values ​​with higher quantization accuracy to replace and update the original low-precision activation values, which helps to further improve the training convergence performance. Based on the evaluation of real-time transmission conditions, the auxiliary computing node flexibly controls the cache update of quantized activation values, and gradually improves the storage accuracy of cached activation values ​​when the transmission conditions are good, further improving the training convergence performance.

[0103] Exemplarily, in order to help understand the implementation process of the large model fine-tuning training method obtained by combining this embodiment with the above-mentioned embodiment 1, the collaborative fine-tuning training process between the terminal side device and the auxiliary computing power node device is described, specifically:

[0104] Before starting the fine-tuning task, the end-side device determines the quantization level of the large model based on its own hardware capabilities such as computing power and memory capacity, and locally loads the complete backbone network of the pre-trained large model that has been quantized to a specific bit width, including the input embedding layer, a set of serially connected Transformer layers, and the output header layer. The auxiliary computing node device initializes the parameters of each adapter in the bypass network with a specific structure. At the same time, the end-side device selects the necessary training setting parameters (including learning rate, local iteration number, batch size, etc.) to initialize training.

[0105] In any iteration of the fine-tuning task execution process, the client device and the auxiliary computing node device operate according to the following process:

[0106] i. Sample extraction (end side, auxiliary computing node side): The end-side device randomly generates sample numbers for group data batches in the local data set and reports them to the auxiliary computing node. The auxiliary computing node queries the storage database for each sample number to see whether its activation value exists, and determines whether a cache update is required based on the "activation value cache and update mechanism". If the associated activation value of a sample has been cached and no cache update is required, the end-side device is instructed to skip the forward propagation calculation and activation value quantization transmission operation for the sample. The end-side device extracts the remaining samples that do not exist in the storage database and performs the forward calculation for this iteration.

[0107] ii. Forward propagation (end side): The sample data is input into the local quantized large model on the end side, and the intermediate activation values ​​are transmitted between layers with full precision.

[0108] iii. Activation value quantization transmission: The activation value output by each Trm layer is quantized with a specific bit width according to the decision result of "importance-aware adaptive activation value quantization" and transmitted to the auxiliary computing node;

[0109] iv. Forward propagation (auxiliary computing node side): The auxiliary computing node obtains the activation value of the cached sample from its cache unit, and receives the quantized activation value output from the end-side device for the uncached sample, fuses the activation value with the bypass network layer by layer in order, performs related forward propagation calculations, and transmits the output of the last adapter back to the end-side device;

[0110] v. Loss calculation (end side): The end-side device receives the bypass network output of the auxiliary computing node, fuses it with the output of the last Trm layer of the local backbone network, and inputs it into the Hed layer. It calculates the loss value based on the sample label, updates the Hed layer parameters in reverse, and transmits the output intermediate gradient to the auxiliary computing node.

[0111] vi. Back propagation (on the auxiliary computing node side): Perform reverse gradient updates on the bypass network parameters to complete a training iteration for the current data batch, and repeat the above iterative process until the model converges.

[0112] In the solution of the present application, the end-side device always maintains a complete and available large model, and can implement local service response to randomly requested large model inference tasks. Its local large model can be periodically updated by integrating with the bypass network adjusted by the auxiliary computing power node. In addition, the auxiliary computing power node can synchronously adjust multiple sets of bypass networks to adapt to multiple target tasks, and send bypass networks to the end-side device on demand, so as to realize flexible switching of multiple tasks without adjusting the local trunk large model of the end-side device.

[0113] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the large model fine-tuning training method of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0114] This application also provides a large model fine-tuning training device, please refer to Figure 6 The device is applied to a terminal device, in which a large pre-trained model after quantization and compression is deployed, and the large model fine-tuning training device includes:

[0115] A forward propagation calculation module 10 is used to input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in a manner of fusing the output activation values ​​with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device;

[0116] The model output fusion module 20 is used to receive the target output fed back by the auxiliary computing node device, and fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output;

[0117] The back propagation parameter updating module 30 is used to determine the loss value according to the model output and the local label data set, and transmit the loss value to the auxiliary computing power node device, so that the auxiliary computing power node device updates the back propagation parameters of the bypass network adapter according to the loss value.

[0118] The large model fine-tuning training device provided by the present application adopts the large model fine-tuning training method in the above embodiment to solve the technical problem. Compared with the prior art, the beneficial effects of the large model fine-tuning training device provided by the present application are the same as the beneficial effects of the large model fine-tuning training method provided by the above embodiment, and the other technical features in the large model fine-tuning training device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0119] The present application provides a large model fine-tuning training device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the large model fine-tuning training method in the above-mentioned embodiment one.

[0120] Reference below Figure 7 , which shows a schematic diagram of the structure of a large model fine-tuning training device suitable for implementing the embodiment of the present application. The large model fine-tuning training device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The large model fine-tuning training device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0121] like Figure 7As shown, the large model fine-tuning training device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the large model fine-tuning training device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the large model fine-tuning training device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a large model fine-tuning training device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0122] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0123] The large model fine-tuning training device provided by the present application adopts the large model fine-tuning training method in the above embodiment, which can solve the technical problems of large model fine-tuning training. Compared with the prior art, the beneficial effects of the large model fine-tuning training device provided by the present application are the same as the beneficial effects of the large model fine-tuning training method provided by the above embodiment, and the other technical features in the large model fine-tuning training device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0124] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0125] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0126] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the large model fine-tuning training method in the above-mentioned embodiment.

[0127] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0128] The above-mentioned computer-readable storage medium may be included in the large model fine-tuning training device; or it may exist independently without being assembled into the large model fine-tuning training device.

[0129] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0132] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned large model fine-tuning training method, and can solve technical problems. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the large model fine-tuning training method provided in the above-mentioned embodiment, and will not be repeated here.

[0133] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the large model fine-tuning training method as described above.

[0134] The computer program product provided in this application can solve the technical problem. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the large model fine-tuning training method provided in the above embodiment, which will not be repeated here.

[0135] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A large model fine-tuning training method, characterized in that: The method is applied to a terminal side device, in which a large pre-trained model after quantization and compression is deployed, and the method includes: Input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in a manner of fusing the output activation values ​​with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device; Receive the target output fed back by the auxiliary computing node device, and fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output; A loss value is determined based on the model output and the local label data set, and the loss value is transmitted to the auxiliary computing power node device, so that the auxiliary computing power node device performs back-propagation parameter update on the bypass network adapter based on the loss value.

2. The method according to claim 1, characterized in that The step of transmitting the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device includes: Iteratively determine the target quantization bit width based on the total amount of data that can be transmitted, the output activation values ​​between each Transformer layer, and the quantization error caused by the gradual increase in the quantization level of the output activation value; Performing low-bit-width quantization on the output activation values ​​between each Transformer layer in the pre-trained large model according to the target quantization bit width to obtain quantized output activation values; The quantized output activation value is transmitted to the auxiliary computing power node device.

3. The method according to claim 2, characterized in that The step of iteratively determining the target quantization bit width according to the total amount of data that can be transmitted, the output activation values ​​between each Transformer layer, and the quantization error caused by the gradual increase in the quantization level of the output activation value includes: Determine the total amount of data that can be transmitted according to the uplink rate from the terminal device to the auxiliary computing power node device at the current iteration time and the preset transmission delay threshold; Determine the trace of the average Fisher information matrix of each layer of adapters in the bypass network adapter according to the gradient information of the parameters to be updated in the bypass network adapter corresponding to each Transformer layer in the pre-trained large model; Initialize the quantization level of the quantization bit width of the output activation value between each Transformer layer to the lowest level, and determine the remaining transmittable data amount according to the output activation value corresponding to the current number of Transformer layers at the current iteration number, the quantization level and the total transmittable data amount; Determine the utility increment according to the product of the quantization error caused by increasing the quantization level by one level and the trace of the average Fisher information matrix; Iteratively determine the target number of Transformer layers that maximizes the utility increment, and update the remaining transmittable data amount according to the target number of layers and the quantization level after increasing by one level, until the remaining transmittable data amount is less than or equal to 0, and obtain the target quantization bit width.

4. The method according to claim 1, characterized in that After the step of inputting the sampled label data set into the pre-trained large model for forward propagation calculation, and transmitting the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, the method further includes: The local label data is sampled, and the sample serial number corresponding to the sampled label data set is transmitted to the auxiliary computing node device, so that the auxiliary computing node device queries the database whether there is an activation value corresponding to the sample serial number. If the sample serial number does not exist, the sample serial number is cached in the database.

5. The method according to claim 4, characterized in that After the step of caching the sample serial number in the database if the sample serial number does not exist, the method further includes: If the sample sequence number already exists, the average amount of data that can be transmitted per layer of a single sample is determined according to the uplink rate from the end-side device to the auxiliary computing power node device at the current iteration time and the preset transmission delay threshold; Determine the average quantization bit width of a single intermediate activation value according to the intermediate activation value dimension corresponding to the pre-trained large model and the average transmittable data amount; Determine whether a condition that a difference between the average quantization bit width and the cached average quantization bit width of activation values ​​of each layer corresponding to the sample sequence number is greater than or equal to a preset quantization bit width difference threshold is met; If the condition is met, the activation value corresponding to the sample number in the cache is cleared, and the activation value corresponding to the sample number is updated.

6. The method according to claim 5, characterized in that After the step of clearing the activation value corresponding to the sample number in the cache if the condition is met and updating the activation value corresponding to the sample number, the method further includes: If the condition is not met, the sample number and the activation value corresponding to the sample number in the cache are cleared, and the terminal device is instructed to skip the forward propagation calculation and activation value transmission operations on the sampled label data set corresponding to the sample number.

7. A large model fine-tuning training device, characterized in that: The device is applied to a terminal side device, in which a pre-trained large model after quantization and compression is deployed, and the large model fine-tuning training device includes: A forward propagation calculation module, used to input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in a manner of fusing the output activation values ​​with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device; A model output fusion module, used to receive the target output fed back by the auxiliary computing node device, and fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output; A back propagation parameter update module is used to determine a loss value based on the model output and a local label data set, and transmit the loss value to the auxiliary computing power node device, so that the auxiliary computing power node device performs a back propagation parameter update on the bypass network adapter according to the loss value.

8. A large model fine-tuning training device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the large model fine-tuning training method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the large model fine-tuning training method as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the steps of the large model fine-tuning training method as described in any one of claims 1 to 6.

Images