Large model fine-tuning training method, device, equipment, storage medium and product

Abstract

The application discloses a large model fine-tuning training method and device, equipment, storage medium and product, relates to the network intelligent technical field, and the method comprises the following steps: inputting a label data set into a large model through an end-side equipment to perform forward propagation calculation, and transmitting output activation values between each Trm layer to an auxiliary computing power node equipment. The auxiliary computing power node equipment performs forward propagation calculation by fusing the output activation values and the output of each layer in the bypass network adapter, and the end-side equipment fuses the target output and the output of the last Trm layer to obtain a model output. According to the model output and the local label data set, a loss value is determined, so that the auxiliary computing power node equipment performs backward propagation parameter updating according to the loss value. Thus, the main network passing through the end-side equipment is avoided, the total memory overhead of the training process is reduced, the memory pressure of the end-side equipment is effectively migrated to the auxiliary computing power node, and the end-side equipment with limited memory can efficiently perform a large model fine-tuning task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network intelligence, and in particular to a large model fine-tuning training method, device, equipment, storage medium and product. BACKGROUND

[0002] In recent years, general large model technology based on the Transformer structure has made significant progress in the field of artificial intelligence and has shown excellent performance in natural language processing, computer vision and other tasks. These large models have obtained strong generalization representation ability through pre-training on large-scale data sets, and have adapted to specific tasks in the fine-tuning stage, achieving excellent performance in multiple benchmark tests. Among them, the fine-tuning stage of the large model relies on sample data in a specific field, which is widely distributed in end-side devices (such as smart phones, Internet of Things devices, etc.) and continuously generated in the interaction response process with users. In order to effectively utilize distributed data while protecting data privacy, using end-side devices to perform large model fine-tuning intelligence tasks has become a current hot technology.

[0003] However, the existing method of performing large model fine-tuning on end-side devices requires a large amount of memory to store intermediate results such as activation values, gradients and optimizer states, which will pose a huge challenge to the memory capacity of end-side devices, making it prone to memory overflow problems during model loading and data processing, resulting in abnormal interruption of the fine-tuning task. SUMMARY

[0004] The main purpose of the present application is to provide a large model fine-tuning training method, device, equipment, storage medium and product, which aims to solve the technical problem that the existing large model local fine-tuning task cannot be performed on memory-constrained end-side devices.

[0005] To achieve the above purpose, the present application provides a large model fine-tuning training method, which is applied to an end-side device, wherein a pre-trained large model compressed by quantization is deployed in the end-side device, and the method comprises:

[0006] inputting the sampled label data set into the pre-trained large model for forward propagation calculation, and transmitting the output activation values between each Transformer layer in the pre-trained large model to an auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in a manner of fusing the output activation values with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device;

[0007] receiving the target output fed back by the auxiliary computing power node device, and fusing the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain a model output;

[0008] determine a loss value according to the model output and the local label data set, and transmit the loss value to the auxiliary computing power node device to enable the auxiliary computing power node device to perform back propagation parameter updating on the bypass network adapter according to the loss value.

[0009] In an embodiment, the step of transmitting the output activation value between each Transformer layer in the pre-trained large model to the auxiliary computing power node device comprises:

[0010] determining a target quantization bit width according to the total transmittable data amount, the output activation value between each Transformer layer, and the quantization error caused by gradually increasing the quantization level of the output activation value;

[0011] performing low-bit-width quantization on the output activation value between each Transformer layer in the pre-trained large model according to the target quantization bit width to obtain quantized output activation value;

[0012] transmitting the quantized output activation value to the auxiliary computing power node device.

[0013] In an embodiment, the step of determining a target quantization bit width according to the total transmittable data amount, the output activation value between each Transformer layer, and the quantization error caused by gradually increasing the quantization level of the output activation value comprises:

[0014] determining the total transmittable data amount according to the uplink rate of the end-side device to the auxiliary computing power node device at the current iteration time and a preset transmission delay threshold;

[0015] determining the trace of the average Fisher information matrix of each layer adapter in the bypass network adapter according to the gradient information of the parameters to be updated in the bypass network adapter corresponding to each Transformer layer in the pre-trained large model;

[0016] initializing the quantization level of the quantization bit width of the output activation value between each Transformer layer to the lowest level, and determining the remaining transmittable data amount according to the output activation value corresponding to the current layer of Transformer at the current iteration number, the quantization level, and the total transmittable data amount;

[0017] determining the utility increment according to the product between the quantization error caused by increasing the quantization level by one level and the trace of the average Fisher information matrix;

[0018] iteratively determine a target number of layers of the Transformer layer that maximizes the utility increment, and update the remaining transmissible data amount according to the target number of layers and a quantization level increased by one level until the remaining transmissible data amount is less than or equal to 0, to obtain a target quantization bit width.

[0019] In an embodiment, after the step of inputting the sampled label data set into the pre-trained large model for forward propagation calculation and transmitting the output activation values between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, the method further comprises:

[0020] sampling the local label data, and transmitting a sample serial number corresponding to the sampled label data set to the auxiliary computing power node device, so that the auxiliary computing power node device queries whether the activation value corresponding to the sample serial number exists in the database, and if the sample serial number does not exist, caching the sample serial number in the database.

[0021] In an embodiment, after the step of if the sample serial number does not exist, caching the sample serial number in the database, the method further comprises:

[0022] If the sample serial number exists, determining a single-sample single-layer average transmissible data amount according to an uplink rate of the end-side device to the auxiliary computing power node device at the current iteration time and a preset transmission delay threshold;

[0023] determining an average quantization bit width of a single intermediate activation value according to an intermediate activation value dimension corresponding to the pre-trained large model and the average transmissible data amount;

[0024] determining whether a condition that a difference between the average quantization bit width and a cached average quantization bit width of the activation value corresponding to the sample serial number is greater than or equal to a preset quantization bit width difference threshold is met;

[0025] If the condition is met, clearing the activation value corresponding to the sample serial number in the cache and updating the activation value corresponding to the sample serial number.

[0026] In an embodiment, after the step of if the condition is met, clearing the activation value corresponding to the sample serial number in the cache and updating the activation value corresponding to the sample serial number, the method further comprises:

[0027] If the condition is not met, clearing the sample serial number and the activation value corresponding to the sample serial number in the cache, and instructing the end-side device to skip the forward propagation calculation and activation value transmission operation on the sampled label data set corresponding to the sample serial number.

[0028] In addition, to achieve the above object, the application further provides a large model fine-tuning training device, the end-side device is deployed with a pre-trained large model after quantization compression, the device is applied to the end-side device, and the large model fine-tuning training device comprises:

[0029] A forward propagation calculation module is configured to input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in the manner of fusing the output activation values with each layer output in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device.

[0030] A model output fusion module is configured to receive the target output fed back by the auxiliary computing power node device, fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model, and obtain a model output.

[0031] A back propagation parameter updating module is configured to determine a loss value according to the model output and the local label data set, and transmit the loss value to the auxiliary computing power node device, so that the auxiliary computing power node device performs back propagation parameter updating on the bypass network adapter according to the loss value.

[0032] In addition, to achieve the above object, the application further provides a large model fine-tuning training device, the device comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the large model fine-tuning training method as described above.

[0033] In addition, to achieve the above object, the application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the large model fine-tuning training method as described above.

[0034] In addition, to achieve the above object, the application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the large model fine-tuning training method as described above.

[0035] The application provides a large model fine-tuning training method. A pre-trained large model after quantization and compression is deployed on an end-side device. The end-side device inputs a sampled label data set into the pre-trained large model for forward propagation calculation, and transmits output activation values between each Trm layer in the pre-trained large model to an auxiliary computing node device. The auxiliary computing node device fuses the output activation values with each layer output in a bypass network adapter, performs forward propagation calculation, and feeds back a target output. The end-side device fuses the target output with the output of the last layer of the Trm layer, obtains a model output, determines a loss value according to the model output and a local label data set, so that the auxiliary computing node device performs backward propagation parameter update on the bypass network adapter according to the loss value. Thus, the parameter update is prevented from passing through the main network of the end-side device, the total memory overhead of the training process is reduced, the memory of the end-side device and the calculation pressure are effectively migrated to the auxiliary computing node, and the end-side device with limited memory can efficiently perform a large model fine-tuning task. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application together with the specification.

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0038] Figure 1 A flowchart is provided for the large model fine-tuning training method embodiment one of the application;

[0039] Figure 2 An interaction flowchart is provided for the large model fine-tuning training method of the application;

[0040] Figure 3 A structure diagram is provided for the bypass network adapter in the auxiliary computing node device of the application;

[0041] Figure 4 A flowchart is provided for the large model fine-tuning training method embodiment two of the application;

[0042] Figure 5 A flowchart is provided for the large model fine-tuning training method embodiment three of the application;

[0043] Figure 6 A module structure diagram is provided for the large model fine-tuning training device of the application embodiment;

[0044] Figure 7A device structure schematic diagram of a hardware running environment involved in the large model fine-tuning training method in the embodiments of the present application.

[0045] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments in conjunction with the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0047] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below in conjunction with the drawings and the specific embodiments.

[0048] The main solution of the embodiments of the present application is that the end-side device inputs the sampled label data set into the pre-trained large model for forward propagation calculation, and transmits the output activation values between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in the manner of fusing the output activation values with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device; receiving the target output fed back by the auxiliary computing power node device, and fusing the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain a model output; determining a loss value according to the model output and the local label data set, and transmitting the loss value to the auxiliary computing power node device, so that the auxiliary computing power node device performs backward propagation parameter update on the bypass network adapter according to the loss value.

[0049] Due to the significant progress of the existing technology based on the general large model technology of the Transformer structure in the field of artificial intelligence, and the excellent performance in natural language processing, computer vision and other tasks. These large models have obtained strong generalization representation ability through pre-training on large-scale data sets, and have adapted to specific tasks in the fine-tuning stage, realizing excellent performance in multiple benchmark tests. Among them, the fine-tuning stage of the large model relies on sample data in specific fields, which are widely distributed in end-side devices (such as smart phones, Internet of Things devices, etc.) and continuously generated in the interaction response process with users. In order to effectively utilize distributed data while protecting data privacy, using end-side devices to perform large model fine-tuning intelligent computing tasks has become a current hot technology. However, the existing method of performing large model fine-tuning by end-side devices needs a large amount of memory to store intermediate results such as activation values, gradients and optimizer states, which will pose a huge challenge to the memory capacity of end-side devices, making it easy to cause memory overflow problems in the model loading and data processing process, resulting in abnormal interruption of the fine-tuning task.

[0050] The application provides a solution. A terminal device with a pre-trained large model after quantization and compression inputs a sampled label data set into the pre-trained large model for forward propagation calculation, and transmits output activation values between each Trm layer in the pre-trained large model to an auxiliary computing power node device. The auxiliary computing power node device fuses the output activation values with the output of each layer in the bypass network adapter, performs forward propagation calculation, and feeds back the target output. The terminal device fuses the target output with the output of the last layer of the Trm layer, obtains a model output, determines a loss value according to the model output and the local label data set, so that the auxiliary computing power node device performs backward propagation parameter updating on the bypass network adapter according to the loss value. Thus, the parameter updating is avoided through the main network of the terminal device, the total memory overhead of the training process is reduced, the memory of the terminal device and the computing pressure are effectively migrated to the auxiliary computing power node, and the memory-limited terminal device can efficiently perform a large model fine-tuning task.

[0051] It should be noted that the execution subject of the method of the embodiment can be a memory-limited terminal device with a pre-trained large model after quantization and compression, for example, a smart phone, an Internet of Things device, etc. The execution subject can also be a large model fine-tuning training device with the same or similar function. The embodiment and each of the following embodiments will be described by taking the large model fine-tuning training device as an example.

[0052] Based on this, the application embodiment provides a large model fine-tuning training method, which refers to Figure 1 , Figure 1 The figure is a flowchart of the first embodiment of the large model fine-tuning training method of the application.

[0053] In the embodiment, the method is applied to a terminal device, the terminal device is deployed with a pre-trained large model after quantization and compression, and the large model fine-tuning training method includes steps S10-S30.

[0054] Step S10, input a sampled label data set into the pre-trained large model for forward propagation calculation, and transmit output activation values between each Transformer layer in the pre-trained large model to an auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation in the manner of fusing the output activation values with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the terminal device.

[0055] It can be understood that the end-side device refers to a terminal or node close to the data source and the user, which has certain computing power and data processing function, can quickly respond and process locally generated data, thereby reducing network load and improving system efficiency. The embodiment provides an end-side memory-friendly large model fine-tuning method, which can be applied to an edge computing network such as a smart home, and based on the cooperative calculation of adjacent high-performance computing nodes, supports the end-side device loaded with a large model to perform a Transformer large model fine-tuning training task based on the local user label data generated continuously.

[0056] Before starting the fine-tuning task, in combination with Figure 2 The structure of the end-side device, the structure of the auxiliary computing node device, and the interactive training process between the two are described. The end-side device determines the quantization level of the large model according to its own computing power and memory capacity and the like hardware capabilities, and locally loads the complete backbone network of the pre-trained large model compressed by specific bit-width quantization, including the input embedding layer (Embedding Layer, Emb), a group of cascaded Transformer layers (Trm), and the output head layer (HeadLayer, Hed). The auxiliary computing node device (with higher memory and computing power resources) initializes a bypass network, which is composed of a group of stacked parallel adapter modules, and the input of each adapter module is obtained by fusing the output of the previous layer adapter module and the activation value output of the corresponding layer of the end-side backbone network. Each bypass network adapter module adopts the same structure, including a dimension reduction mapping layer, a nonlinear layer, a dimension increasing mapping layer, and a normalization layer, and the structure of the adapter can refer to Figure 3 .

[0057] It should be noted that during the fine-tuning training process, the end-side device samples the local label data set to obtain a representative data subset. This data subset is then used as input to start an iterative forward propagation calculation in the backbone network of the local pre-trained large model. During the fine-tuning process, the user label data is maintained locally on the end-side device without the need for inter-device sharing, improving data privacy protection capabilities. At the same time, the activation values between the Transformer layers are low-bit-width quantization encoded and transmitted to the computing node, respectively input into the adapter of the corresponding layer of the stacked structure for forward propagation calculation, and the target output of the last layer adapter is returned to the end-side device.

[0058] Step S20, receiving the target output fed back by the auxiliary computing node device, and fusing the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain a model output.

[0059] It can be understood that the end-side device receives the target output fed back by the auxiliary computing power node device, fuses the target output with the output of the last layer of the backbone network, inputs the fused output into the head layer to obtain the model output.

[0060] In step S30, a loss value is determined according to the model output and the local label data set, and the loss value is transmitted to the auxiliary computing power node device to enable the auxiliary computing power node device to perform back propagation parameter update on the bypass network adapter according to the loss value.

[0061] It should be understood that the end-side device determines a loss value according to the model output obtained by the head layer in combination with the local label data set, and transmits the loss value back to the computing power node in reverse to perform back propagation parameter update on the bypass network adapter, thereby avoiding the backbone network passing through the end-side device, reducing the total memory overhead of the training process, and effectively migrating the memory and computing pressure of the end-side device to the auxiliary computing power node. The large model fine-tuning network is effectively divided into a parallel frozen state backbone network and an activated state bypass network, and only the bypass parameters in the activated state are trained and updated during the fine-tuning process, while the original Trm module always remains in a non-activated state and does not perform parameter update calculation, thereby shortening the calculation time and reducing the memory overhead introduced by storing intermediate parameters.

[0062] The embodiment provides a large model fine-tuning training method. An end-side device deployed with a pre-trained large model after quantization and compression inputs a sampled label data set into the pre-trained large model to perform forward propagation calculation, and transmits output activation values between each Trm layer in the pre-trained large model to an auxiliary computing power node device. The auxiliary computing power node device fuses the output activation values with the output of each layer in the bypass network adapter, performs forward propagation calculation, and feeds back a target output. The end-side device fuses the target output with the output of the last layer of the Trm layer to obtain a model output, determines a loss value according to the model output and the local label data set, to enable the auxiliary computing power node device to perform back propagation parameter update on the bypass network adapter according to the loss value. Thus, the parameter update is avoided to pass through the backbone network of the end-side device, the total memory overhead of the training process is reduced, the memory and computing pressure of the end-side device are effectively migrated to the auxiliary computing power node, and the end-side device with limited memory can efficiently perform a large model fine-tuning task.

[0063] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above-mentioned first embodiment can be referred to the above introduction, and will not be described in detail. On this basis, please refer to Figure 4 , in step S10, the large model fine-tuning training method further includes steps S101-S103:

[0064] Step S101: iteratively determining the target quantization bit-width according to the total transmittable data amount, the output activation value between each Transformer layer, and the quantization error caused by gradually increasing the quantization level of the output activation value.

[0065] It can be understood that the adaptive activation value quantization strategy with importance awareness is proposed in the embodiment. During the fine-tuning task execution process, the auxiliary computing node performs activation value quantization bit-width decision once every k training iterations, that is, a suitable quantization bit-width is selected for the output activation value of each layer Trm, and the activation value is quantized by the quantization bit-width and then transmitted to the auxiliary computing node device. After initializing the quantization level of the output activation value, the quantization level is gradually increased iteratively, and the optimal target quantization bit-width is determined according to the quantization error caused by increasing the quantization level and the total transmittable data amount.

[0066] In a feasible implementation, step S101 can include steps S1011-S1015:

[0067] Step S1011: determining the total transmittable data amount according to the uplink rate of the end-side device to the auxiliary computing node device at the current iteration time and the preset transmission delay threshold.

[0068] It is worth noting that the output activation value of the lth layer Trm at the tth iteration is , the uplink rate R t of the end-side device to the auxiliary computing node at the current iteration time t is measured. t The total transmittable data amount R max is calculated in combination with the uplink rate R t and the preset transmission delay threshold T th , wherein the value of T th should be set considering the update calculation time of the auxiliary computing node side to the bypass network, and the idle calculation time is shortened as much as possible.

[0069] Step S1012: determining the trace of the average Fisher information matrix of each layer adapter in the bypass network adapter according to the gradient information of the to-be-updated parameters of the bypass network adapter corresponding to each Transformer layer in the pre-trained large model.

[0070] It can be understood that the trace of the average Fisher information matrix of each layer adapter in the bypass network adapter (denoted as tr(F)) is calculated based on the gradient information of the to-be-updated parameters of the bypass network corresponding to each Trm layer. The greater the value of tr(F) is, the more important the adapter is in the current training stage.

[0071] Step S1013, initialize the quantization level of the quantization bit width of the output activation value between each Transformer layer to the lowest level, and determine the remaining transmissible data amount according to the output activation value corresponding to the Transformer layer of the current layer at the current iteration number, the quantization level, and the transmissible total data amount.

[0072] It should be understood that the quantization bit width of the output activation value of each layer Trm is initialized to the lowest level, that is, the quantization level Calculate the remaining transmissible data amount R res :

[0073]

[0074] Step S1014, determine the utility increment according to the product between the quantization error caused by increasing the quantization level by one level and the trace of the average Fisher information matrix.

[0075] It can be understood that for any Trm layer, the utility increment function is defined as the product of the decrease in quantization error caused by increasing the quantization bit width by one level and the trace of the average Fisher information matrix, and the utility increment function is as follows:

[0076]

[0077] wherein, represents the activation value The quantization error introduced by quantizing with the quantization level .

[0078] Step S1015, iteratively determine the target number of layers of the Transformer layer that maximizes the utility increment, and update the remaining transmissible data amount according to the target number of layers and the quantization level increased by one level, until the remaining transmissible data amount is less than or equal to 0, and the target quantization bit width is obtained.

[0079] It can be understood that the Trm layer l that maximizes the utility increment function value is searched, and the quantization bit width level of the Trm layer l is increased from to and the remaining transmissible data amount is updated. The above operations in this step are iteratively repeated until the remaining transmissible data amount is not greater than 0, or the quantization bit width of the output activation value of all layers Trm reaches the highest level, and the target quantization bit width of the output activation value of each layer Trm at this time is output.

[0080] Step S102: quantize the output activation value between each Transformer layer in the pre-trained large model according to the target quantization bit width, and obtain the quantized output activation value.

[0081] Step S103: transmitting the quantized output activation value to the auxiliary computing power node device.

[0082] In this embodiment, the target quantization bit width is determined according to the total transmittable data amount, the output activation value between each Transformer layer, and the quantization error caused by the gradual increase of the quantization level of the output activation value. The output activation value between each Transformer layer in the pre-trained large model is quantized at a low bit width according to the target quantization bit width, and the quantized output activation value is obtained. The quantized output activation value is transmitted to the auxiliary computing power node device. Through the above-mentioned manner, a high-precision quantization strategy can be selected for more important output activation values under the transmission time constraint to alleviate the quantization loss introduced thereby, thereby helping to improve the training convergence performance. The end-side device adopts an activation value adaptive quantization mechanism, captures the importance of different Trm layers to the convergence process by online sensing the training state and channel conditions, and reasonably allocates the quantization bit width of the output activation value of each layer, so as to save the device transmission resource overhead without sacrificing the convergence speed and fine-tuning performance, and to realize the acceleration of the fine-tuning process.

[0083] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as the above-mentioned first embodiment can be referred to the above introduction, and will not be described in detail. On this basis, please refer to Figure 5 , after step S10, the large model fine-tuning training method further includes step A10:

[0084] Step A10: sampling the local label data, transmitting the sample serial number corresponding to the sampled label data set to the auxiliary computing power node device, so that the auxiliary computing power node device queries whether the activation value corresponding to the sample serial number exists in the database, and if the sample serial number does not exist, the sample serial number is cached in the database.

[0085] It is worth noting that, in view of the characteristics of the fixed pre-training backbone model parameters in the fine-tuning training process, the auxiliary computing power node receives the output activation value of each Trm layer using a specific quantization bit width, and the corresponding data sample serial number is cached together, thereby effectively reducing the calculation and transmission overhead introduced by the repeated forward propagation and activation value transmission of the end-side device for the same extracted sample. Considering the node-side storage capacity constraint and the dimension of the intermediate activation value in the large model, the cache refresh period is artificially set, such as one training epoch, that is, after the end of each cache refresh period, the auxiliary computing power node clears the cached activation value in the storage space and reopens the cache and update process.

[0086] It can be understood that before the start of each iteration, the auxiliary computing power node preloads the cached activation values of the required samples of this iteration from the hard disk to the memory for access. The auxiliary computing power node receives the sample sequence number of the N groups of data batches randomly generated by the end-side device, queries whether the corresponding activation value exists in the storage database for each sample sequence number, and caches the sample sequence number in the current sample sequence for the sample that does not exist. In each iteration, the auxiliary computing power node receives the Trm layer output activation value with a specific quantization bit width related to the valid sample sequence from the end-side device, together with its corresponding data sample sequence number, and stores it in the hard disk. The storage format is shown in the following table.

[0087] Sample ID Layer number Quantized intermediate activation value xx xx xx

[0088] In a feasible implementation, after step A10, steps A20-A50 can also be included:

[0089] Step A20, if the sample sequence number already exists, determining the average transmissible data amount of a single sample and a single layer according to the uplink rate of the end-side device to the auxiliary computing power node device at the current iteration time and the preset transmission delay threshold.

[0090] It should be noted that for the sample sequence number that already exists in the database, further judgment is performed. First, the uplink rate R t of the end-side device to the auxiliary computing power node at the current iteration time is measured, and based on the preset transmission delay threshold T th , the average transmissible data amount of a single sample and a single layer is calculated:

[0091]

[0092] Where B is the data batch size.

[0093] Step A30, determining the average quantization bit width of a single intermediate activation value according to the intermediate activation value dimension corresponding to the pre-trained large model and the average transmissible data amount.

[0094] It can be understood that the average quantization bit width of a single intermediate activation value is further determined according to the intermediate activation value dimension corresponding to the pre-trained large model and the average transmissible data amount, denoted as

[0095] Step A40, judging whether the condition that the difference between the average quantization bit width and the cached average quantization bit width of the layer activation value corresponding to the sample sequence number is greater than or equal to the preset quantization bit width difference threshold is established.

[0096] It should be understood that the judgment condition is established, where b cache is the average quantization bit width of the cached layer activation value for the sample, and Δb a preset quantization bit width difference threshold value.

[0097] Step A50, if the condition is met, clear the activation value corresponding to the sample sequence number in the cache, and update the activation value corresponding to the sample sequence number.

[0098] It should be understood that if the condition is met, the sample entry corresponding to the sample sequence number in the storage database and the activation value contained in the entry are cleared, and the sample sequence number is retained in the current sample sequence.

[0099] In a possible implementation, after step A50, step A60 can also be included:

[0100] Step A60, if the condition is not met, clear the sample sequence number and the activation value corresponding to the sample sequence number in the cache, and instruct the end-side device to skip the forward propagation calculation and activation value transmission operation on the sampled label data set corresponding to the sample sequence number.

[0101] It is worth noting that if the condition is not met, the sample sequence number and the activation value corresponding to the sample sequence number in the cache are cleared to receive higher quantization level activation values for cache updating. And instruct the end-side device to skip the forward propagation calculation and quantized activation value transmission operation on the sampled label data set corresponding to the sample sequence number. The end-side device extracts the remaining samples that do not exist in the storage database and performs the forward calculation of this iteration. At the end of the current cache refresh period, the auxiliary computing power node clears all cached activation values and releases the storage space.

[0102] In this embodiment, by sampling the local label data, the sample sequence number corresponding to the sampled label data set is transmitted to the auxiliary computing power node device, so that the auxiliary computing power node device queries whether the activation value corresponding to the sample sequence number exists in the database, and takes appropriate strategies according to the query result, thereby effectively reducing the calculation and transmission overhead introduced by repeated forward propagation and activation value transmission of the same extracted sample by the end-side device. At the same time, it can flexibly control the replacement and update of the original low-precision activation value with higher-precision activation value when the real-time transmission condition is good, which helps to further improve the training convergence performance. Based on the evaluation of the real-time transmission condition, the auxiliary computing power node flexibly controls the cache updating of the quantized activation value, gradually improves the storage precision of the cached activation value when the transmission condition is good, and further improves the training convergence performance.

[0103] Exemplarily, in order to help understand the implementation process of the large model fine-tuning training method obtained after combining the above-mentioned embodiment one with the present embodiment, the cooperative fine-tuning training process between the end-side device and the auxiliary computing power node device is described, specifically:

[0104] Before starting the fine-tuning task, the end-side device determines the quantization level of the large model according to its own hardware capabilities such as computing power and memory capacity, and locally loads the complete backbone network of the pre-trained large model with specific bit-width quantization, including the input embedding layer, a series of concatenated Transformer layers and the output head layer. The auxiliary computing node device initializes the parameters of each adapter in the bypass network with a specific structure. At the same time, the end-side device selects the necessary training setting parameters (including learning rate, local iteration number and batch size, etc.) to initialize the training.

[0105] At any iteration in the fine-tuning task execution process, the end-side device and the auxiliary computing node device run according to the following process:

[0106] i. Sample extraction (end-side, auxiliary computing node side): The end-side device randomly generates sample sequence numbers for a group of data batches in the local data set and reports them to the auxiliary computing node. The auxiliary computing node queries whether the activation values of each sample sequence number exist in the storage database, and determines whether the cache needs to be updated according to the "activation value cache and update mechanism". If the associated activation values of a sample have been cached and do not need to be cached, the end-side device is instructed to skip the forward propagation calculation and activation value quantization transmission operation for that sample. The end-side device extracts the remaining samples that do not exist in the storage database and performs forward calculation for this iteration.

[0107] ii. Forward propagation (end-side): input the sample data into the quantized large model locally on the end-side, and maintain the intermediate activation values in full precision for inter-layer transmission;

[0108] iii. Activation value quantization transmission: the output activation values of each Trm layer are subjected to specific bit-width quantization processing according to the decision results of the "importance-aware adaptive activation value quantization" and are transmitted to the auxiliary computing node;

[0109] iv. Forward propagation (auxiliary computing node side): the auxiliary computing node obtains the activation values of the cached samples from their cache units, and receives the quantized activation value output from the end-side device for the uncached samples. The activation values are fused with the bypass network layer by layer in sequence, the relevant forward propagation calculation is performed, and the output of the last adapter is transmitted back to the end-side device;

[0110] v. Loss calculation (end-side): the end-side device receives the output of the bypass network of the auxiliary computing node, fuses it with the output of the last Trm layer of the local backbone network to input the Hed layer, calculates the loss value based on the sample label, and updates the parameters of the Hed layer in reverse, and transmits the output intermediate gradient to the auxiliary computing node;

[0111] vi. Backpropagation (auxiliary computing node side): the parameters of the bypass network are updated in reverse with the gradient, thereby completing one training iteration for the current data batch. Repeat the above iteration process until the model converges.

[0112] The end-side device in the scheme of the application always maintains a complete available large model, and can realize local service response to a randomly requested large model inference task. The local large model can be periodically updated by fusion with the bypass network adjusted by the assisted computing node. In addition, the assisted computing node can synchronously adjust multiple bypass networks to adapt to multiple target tasks, and deliver the bypass network to the end-side device on demand, so as to realize flexible switching of multiple tasks without adjusting the local backbone large model of the end-side device.

[0113] It should be noted that the above examples are only used for understanding the application and do not constitute a limitation on the large model fine-tuning training method of the application. More forms of simple transformation based on this technical concept are within the protection scope of the application.

[0114] The application also provides a large model fine-tuning training device, which is described in detail in the following Figure 6 The device is applied to an end-side device, a pre-trained large model quantized and compressed is deployed in the end-side device, and the large model fine-tuning training device comprises:

[0115] A forward propagation calculation module 10 is configured to input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values between each Transformer layer in the pre-trained large model to the assisted computing node device, so that the assisted computing node device performs forward propagation calculation in the manner of fusion of the output activation values and each layer output in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device.

[0116] A model output fusion module 20 is configured to receive the target output fed back by the assisted computing node device, and fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain a model output.

[0117] A backward propagation parameter updating module 30 is configured to determine a loss value according to the model output and the local label data set, and transmit the loss value to the assisted computing node device, so that the assisted computing node device performs backward propagation parameter updating on the bypass network adapter according to the loss value.

[0118] The large model fine-tuning training device provided by the application adopts the large model fine-tuning training method in the above embodiments, and can solve the technical problems. Compared with the prior art, the large model fine-tuning training device provided by the application has the same beneficial effects as the large model fine-tuning training method provided by the above embodiments, and other technical features in the large model fine-tuning training device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0119] The application provides a large model fine-tuning training device, which comprises at least one processor and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the large model fine-tuning training method in the above-mentioned embodiment one.

[0120] Reference will be made to the accompanying drawings Figure 7 which shows a structural schematic diagram of a large model fine-tuning training device suitable for being used to implement the embodiments of the application. The large model fine-tuning training device in the embodiments of the application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 7 The large model fine-tuning training device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the application.

[0121] As Figure 7As shown, the large model fine-tuning training device can include a processing apparatus 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or loaded from a storage apparatus 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the large model fine-tuning training device to operate are also stored. The processing apparatus 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input apparatuses 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output apparatuses 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the large model fine-tuning training device to communicate with other devices wirelessly or by wire to exchange data. Although the large model fine-tuning training device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0122] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication apparatus, or installed from the storage apparatus 1003, or installed from the ROM 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0123] The large model fine-tuning training device provided by the present disclosure adopts the large model fine-tuning training method in the above-mentioned embodiments, and can solve the technical problem of large model fine-tuning training. Compared with the prior art, the large model fine-tuning training device provided by the present disclosure has the same beneficial effects as the large model fine-tuning training method provided by the above-mentioned embodiments, and other technical features in the large model fine-tuning training device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0124] It should be understood that various parts of the present application can be realized by hardware, software, firmware, or a combination thereof. In the above description of the embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0125] The above description is merely a specific implementation of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered by the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the scope of protection of the claims.

[0126] The present application provides a computer readable storage medium having computer readable program instructions (i.e., a computer program) stored thereon, the computer readable program instructions being used to perform the large model fine-tuning training method in the above-described embodiments.

[0127] The computer readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination thereof.

[0128] The above computer readable storage medium can be included in the large model fine-tuning training device; or can exist separately and not be assembled into the large model fine-tuning training device.

[0129] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0130] The flow diagrams and the block diagrams in the drawings are meant as methodological and functional description of implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0131] The modules involved in the embodiments of the present application can be implemented in software or hardware. In some cases, the names of the modules do not constitute a limitation on the modules themselves.

[0132] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the above-mentioned large model fine-tuning training method, and can solve the technical problems. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the large model fine-tuning training method provided by the above-mentioned embodiments, which will not be repeated here.

[0133] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the large model fine-tuning training method as described above.

[0134] The computer program product provided by the application can solve the technical problems. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the large model fine-tuning training method provided by the above-mentioned embodiments, which will not be repeated here.

[0135] The above-mentioned is only part of the embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation, direct / indirect application in other related technical fields, or direct / indirect application in other related technical fields under the technical concept of the application, including in the patent protection scope of the application.

Claims

1. A large model fine-tuning training method, characterized in that: The method is applied to a terminal-side device, where a large pre-trained model that has been quantized and compressed is deployed. The method includes: Input the sampled label data set into the pre-trained large model for forward propagation calculation, and transmit the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculation by fusing the output activation values ​​with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device; Receive the target output fed back by the auxiliary computing node device, and fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output; A loss value is determined based on the model output and the local label data set, and the loss value is transmitted to the auxiliary computing power node device, so that the auxiliary computing power node device performs backpropagation parameter update on the bypass network adapter based on the loss value.

2. The method according to claim 1, wherein The step of transmitting the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device includes: Iteratively determining a target quantization bit width based on the total amount of data that can be transmitted, the output activation values ​​between each Transformer layer, and the quantization error caused by the gradual increase in the quantization level of the output activation values; Performing low-bit-width quantization on the output activation values ​​between each Transformer layer in the pre-trained large model according to the target quantization bit width to obtain quantized output activation values; The quantized output activation value is transmitted to the auxiliary computing power node device.

3. The method according to claim 2, wherein The step of iteratively determining the target quantization bit width based on the total amount of transmittable data, the output activation values ​​between each Transformer layer, and the quantization error caused by the gradual increase in the quantization level of the output activation value comprises: Determine the total amount of data that can be transmitted based on the uplink rate from the end-side device to the auxiliary computing power node device at the current iteration time and a preset transmission delay threshold; Determine the trace of the average Fisher information matrix of each layer adapter in the bypass network adapter according to the gradient information of the parameters to be updated in the bypass network adapter corresponding to each Transformer layer in the pre-trained large model; Initialize the quantization level of the quantization bit width of the output activation value between each Transformer layer to the lowest level, and determine the remaining transmittable data amount according to the output activation value corresponding to the current number of Transformer layers at the current iteration number, the quantization level, and the total transmittable data amount; determining a utility increment based on a product of a quantization error caused by increasing the quantization level by one level and a trace of the average Fisher information matrix; Iteratively determine the target number of Transformer layers that maximizes the utility increment, and update the remaining transmittable data amount according to the target number of layers and the quantization level after increasing by one level, until the remaining transmittable data amount is less than or equal to 0, and obtain the target quantization bit width.

4. The method according to claim 1, wherein After the step of inputting the sampled label data set into the pre-trained large model for forward propagation calculation, and transmitting the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, the method further includes: The local label data is sampled, and the sample serial number corresponding to the sampled label data set is transmitted to the auxiliary computing power node device, so that the auxiliary computing power node device queries the database whether there is an activation value corresponding to the sample serial number. If the sample serial number does not exist, the sample serial number is cached in the database.

5. The method according to claim 4, wherein After the step of caching the sample serial number in the database if the sample serial number does not exist, the method further includes: If the sample sequence number already exists, the average transmittable data volume of a single sample per layer is determined based on the uplink rate from the client device to the auxiliary computing node device at the current iteration time and the preset transmission delay threshold; Determine the average quantization bit width of a single intermediate activation value according to the intermediate activation value dimension corresponding to the pre-trained large model and the average transmittable data amount; Determine whether a condition is met where a difference between the average quantization bit width and the cached average quantization bit width of activation values ​​of each layer corresponding to the sample sequence number is greater than or equal to a preset quantization bit width difference threshold; If the condition is met, the activation value corresponding to the sample number in the cache is cleared, and the activation value corresponding to the sample number is updated.

6. The method according to claim 5, wherein After the step of clearing the activation value corresponding to the sample number in the cache and updating the activation value corresponding to the sample number if the condition is met, the method further includes: If the condition is not met, the sample number and the activation value corresponding to the sample number in the cache are cleared, and the end-side device is instructed to skip the forward propagation calculation and activation value transmission operations on the sampled label data set corresponding to the sample number.

7. A large model fine-tuning training device, characterized in that: The device is applied to a terminal-side device, in which a large pre-trained model that has been quantized and compressed is deployed. The large model fine-tuning training device includes: A forward propagation calculation module is used to input the sampled label data set into the pre-trained large model to perform forward propagation calculations, and transmit the output activation values ​​between each Transformer layer in the pre-trained large model to the auxiliary computing power node device, so that the auxiliary computing power node device performs forward propagation calculations by fusing the output activation values ​​with the output of each layer in the bypass network adapter, and feeds back the target output of the last layer adapter to the end-side device; A model output fusion module is used to receive the target output fed back by the auxiliary computing node device and fuse the target output with the output of the last layer of the Transformer layer in the pre-trained large model to obtain the model output; A back propagation parameter update module is used to determine a loss value based on the model output and the local label data set, and transmit the loss value to the auxiliary computing power node device, so that the auxiliary computing power node device updates the back propagation parameters of the bypass network adapter according to the loss value.

8. A large model fine-tuning training device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the large model fine-tuning training method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the large model fine-tuning training method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product includes a computer program, which, when executed by a processor, implements the steps of the large model fine-tuning training method according to any one of claims 1 to 6.

Images