Large model adjustment method and device, equipment and storage medium

Abstract

The invention relates to the technical field of large models, and discloses a large model adjustment method, device and equipment and a storage medium, the method comprises the following steps: carrying out segmentation deployment on a to-be-adjusted large model to obtain sub-models deployed at different end side equipment, each sub-model comprising different numbers of coding and decoding layers and adjustable adapters, a corresponding adjustable adapter is embedded behind each coding and decoding layer; updating the adjustable adapter layer by layer according to the training state of the to-be-adjusted large model to obtain a large model of the adjustable adapter embedded with updated configuration; and performing forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain a large model after parameter adjustment. The parameter adjustment efficiency of the large model can be effectively improved, and equipment resources are saved.

Description

Technical Field

[0001] The present application relates to the field of large model technology, and in particular to large model adjustment methods, devices, equipment and storage media. Background Art

[0002] In recent years, the general large model technology based on the Transformer structure has made significant progress in the field of artificial intelligence and has demonstrated excellent performance in a variety of tasks such as natural language processing and computer vision. These large models have acquired strong generalization representation capabilities through pre-training on large-scale data sets, and are adapted to specific tasks in the fine-tuning stage, achieving excellent performance in multiple benchmarks. Among them, the fine-tuning stage of the large model relies on sample data in a specific field, which is widely distributed on end-side devices (such as smartphones, IoT devices, etc.) and continuously generated in the interactive response process with users. In order to effectively utilize distributed data while protecting data privacy, using end-side devices to perform large model fine-tuning intelligent computing tasks has become a current hot technology. Mainstream general large models usually have multiple layers of Transformer modules, each layer of which contains multi-head self-attention and feedforward neural networks. These hierarchical structures enable Transformer to efficiently learn complex data patterns, but also introduce large-scale matrix operations. Performing such high-computation intelligent computing tasks on end-side devices faces significant computing resource bottlenecks, which can easily lead to processing delays and excessive energy consumption. In addition, the number of parameters of large models is usually in the hundreds of millions or even tens of billions. During the forward / backward propagation calculation process, a large amount of memory is required to store intermediate results such as activation values, gradients, and optimizer states. This poses a huge challenge to the memory capacity of the end-side device, making it easy to have memory overflow problems during model loading and data processing, resulting in abnormal interruption of fine-tuning tasks. By tapping into the communication interconnection and resource complementarity capabilities between connected end-side devices, using model segmentation and multi-device pipeline parallel computing technology, the large model is divided into multiple sub-models and assigned to different end-side devices for collaborative execution, which can significantly reduce the computing power and memory resource consumption of a single device. These end-side devices coexist in a local area network and form a pipeline topology based on the hierarchical order of the sub-models deployed in the original large model. The data samples required for fine-tuning training are stored in the "head" device of the pipeline, and their corresponding sample labels are passed to the "tail" device of the pipeline. During the forward propagation process, each device applies the sub-model layer it is responsible for to the input passed in the previous stage, and passes the output of the last layer to the next stage device. Backward propagation is performed in the opposite order, and the gradient information is passed back to each device in turn. To improve training efficiency, multiple devices can process different batches of data at the same time, and a copy of the sub-model weights used by each device when processing the forward pass will be saved for subsequent backward propagation of the corresponding batch.

[0003] However, the existing collaborative fine-tuning technology based on pipeline parallelism has the following two main problems: 1) The current collaborative training design based on pipeline parallelism generally follows a chain-like sequential training mechanism, which requires the first device in the chain to hold all input data and share relevant labels with the last device, which raises privacy issues and increases interaction overhead. In addition, only the data samples of the "head" device of the pipeline are allowed to be trained during the entire process, while other devices only serve as auxiliary computing nodes. Therefore, this pipeline parallel training framework cannot use the data of multiple devices to fine-tune a unified model. In practical applications, due to storage limitations, the data samples on a single device may be quite limited, while the multi-devices in a local area network can often collect different types of task data. The inefficient use of these data will damage the performance of the fine-tuning model. 2) Pipeline parallel training allows the use of unupdated model parameters for forward propagation calculations. In order to unify the model update direction, each device is required to store different versions of model parameters for average update, which will introduce significant storage overhead, making the end-side devices with insufficient memory and storage capacity face more severe resource challenges, which can easily lead to the failure of fine-tuning tasks.

[0004] Therefore, how to effectively improve the efficiency of large model parameter adjustment and save equipment resources is a problem that needs to be solved urgently.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a large model adjustment method, device, equipment and storage medium, aiming to solve the technical problem of how to effectively improve the efficiency of large model parameter adjustment and save equipment resources.

[0007] To achieve the above objectives, the present application proposes a large model adjustment method, the method comprising:

[0008] The large model to be adjusted is split and deployed to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is embedded with a corresponding adjustable adapter;

[0009] updating the adjustable adapter layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model in which the adjustable adapter with the updated configuration is embedded;

[0010] The large model of the adjustable adapter embedded with the updated configuration is subjected to forward propagation calculation and backward propagation calculation to obtain the large model with adjusted parameters.

[0011] In one embodiment, the large model to be adjusted is divided and deployed to obtain multiple sub-models and corresponding terminal-side devices, including:

[0012] Determine the number of codec layers to be allocated to each end-side device according to the memory capacity of each end-side device;

[0013] Dividing the large model to be adjusted into a plurality of sub-models according to the number of allocated coding and decoding layers;

[0014] Get the target Hamiltonian circuit;

[0015] Based on the target Hamiltonian circuit, the plurality of sub-models are sequentially deployed on corresponding end-side devices to obtain sub-models deployed on different end-side devices.

[0016] In one embodiment, determining the number of codec layers allocated to each end-side device according to the memory capacity of each end-side device includes:

[0017] Acquire multiple end-side devices and corresponding forward propagation calculation delays, and sort the end-side devices based on the forward propagation calculation delays to obtain sorted end-side devices;

[0018] Determine the threshold of the number of codec layers of each end-side device according to the memory capacity of each end-side device;

[0019] The number of codec layers to be allocated to each end-side device is determined according to the codec layer number threshold and the sorted end-side devices.

[0020] In one embodiment, obtaining a target Hamiltonian circuit includes:

[0021] Acquire transmission links between the end-side devices, and construct a directed graph according to the end-side devices and the transmission links, wherein the directed graph includes a plurality of original nodes;

[0022] Adding a virtual node to the directed graph, and constructing directed edges based on the virtual node and each original node;

[0023] Obtaining the weight of each directed edge, and generating a directed weighted graph based on the weight of each directed edge;

[0024] A target Hamiltonian circuit is determined based on the directed weighted graph.

[0025] In one embodiment, determining a target Hamiltonian circuit based on the directed weighted graph comprises:

[0026] Taking the virtual node in the directed weighted graph as a starting point and performing path planning to generate a path table;

[0027] Determine the path length from each visited node in the visited node set to an unvisited node based on the path table;

[0028] determining whether there is a reference Hamiltonian circuit according to the path length;

[0029] When the reference Hamiltonian circuit exists, the virtual node is removed and the path is reconstructed through the backtracking table to generate a target Hamiltonian circuit.

[0030] In one embodiment, the step of sequentially deploying the plurality of sub-models on corresponding end-side devices based on the target Hamiltonian circuit to obtain sub-models deployed on different end-side devices includes:

[0031] Determining an arrangement order of the end-side devices based on the target Hamiltonian circuit;

[0032] Obtain model parameters of each sub-model;

[0033] The model parameters are sent to the corresponding end-side devices in sequence according to the arrangement order of the end-side devices to obtain sub-models deployed on different end-side devices.

[0034] In one embodiment, updating the adjustable adapter layer by layer according to the training state of the large model to be adjusted to obtain a large model in which the adjustable adapter after the updated configuration is embedded includes:

[0035] Obtaining the connection order of the codec layers in each sub-model in the large model to be adjusted;

[0036] Activating the corresponding adjustable adapters layer by layer according to the training state of the large model to be adjusted and the connection sequence to obtain an adjustable adapter in an activated state;

[0037] Adjusting the width configuration of the adjustable adapter in the activated state to obtain an adjustable adapter with an updated configuration;

[0038] The adjustable adapter after the updated configuration is embedded into the large model to be adjusted to obtain the large model in which the adjustable adapter after the updated configuration is embedded.

[0039] In one embodiment, the adjusting the width configuration of the adjustable adapter in the activated state to obtain the adjustable adapter with updated configuration includes:

[0040] Acquiring gradient information of current adjustable parameters of the adjustable adapter in the activated state;

[0041] Determining a trace of a Fisher information matrix based on the gradient information of the current adjustable parameter;

[0042] sorting the adjustable adapters in the activation state based on the trace of the Fisher information matrix to obtain a target adapter;

[0043] The bottleneck dimension of the target adapter is updated based on a preset bottleneck dimension growth step to obtain an adjustable adapter after the updated configuration.

[0044] In one embodiment, the step of performing forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain the large model with adjusted parameters includes:

[0045] Based on the task data samples, forward propagation calculation and backward propagation calculation are performed on the sub-models of each end-side device to achieve pipeline parallel training and obtain the trained large model;

[0046] Evaluate the training convergence according to the loss value of the trained large model to obtain an evaluation result;

[0047] When the evaluation result is that the training has converged, the trained large model is used as the large model after parameter adjustment;

[0048] When the evaluation result is that the training has not converged, the trained large model is subjected to polling training until the training converges, thereby obtaining the large model with adjusted parameters.

[0049] In one embodiment, the forward propagation calculation and the backward propagation calculation are performed on the sub-models of each end-side device based on the task data sample to implement pipeline parallel training to obtain a trained large model, including:

[0050] Determining a forward propagation path and a backward propagation path based on the terminal side device;

[0051] Based on the task data samples, forward propagation training is performed on the sub-model of each end-side device along the forward propagation path, and backward propagation training is performed on the sub-model of each end-side device along the backward propagation path to realize pipeline parallel training and obtain a trained large model.

[0052] In addition, to achieve the above purpose, the present application also proposes a large model adjustment device, the large model adjustment device comprising:

[0053] A deployment module is used to split and deploy the large model to be adjusted to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is embedded with a corresponding adjustable adapter;

[0054] An updating module, used for updating the adjustable adapter layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model of the adjustable adapter embedded with the updated configuration;

[0055] The propagation module is used to perform forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain the large model with adjusted parameters.

[0056] In addition, to achieve the above-mentioned purpose, the present application also proposes a large model adjustment device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the large model adjustment method described above.

[0057] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the large model adjustment method described above are implemented.

[0058] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the large model adjustment method described above are implemented.

[0059] The present application provides a large model adjustment method. The present application first divides and deploys the large model to be adjusted to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and a corresponding adjustable adapter is embedded after each codec layer; the adjustable adapter is updated layer by layer according to the training state of the large model to be adjusted to obtain a large model with the adjustable adapter embedded in the updated configuration; forward propagation calculation and backward propagation calculation are performed on the large model with the adjustable adapter embedded in the updated configuration to obtain a large model with adjusted parameters, which can effectively improve the efficiency of large model parameter adjustment and save device resources.

[0060] In summary, the present application can accurately adapt to the corresponding end-side devices by splitting and deploying the large model to be adjusted, obtain sub-models deployed on different end-side devices, and update the adjustable adapters in each sub-model layer by layer, effectively shortening the back propagation path, thereby performing forward propagation and backward propagation, and realizing fine-tuning of the large model through multiple devices, and quickly and efficiently obtaining the adjusted large model, overcoming the technical defects of being unable to use multiple devices for model fine-tuning and the failure of model fine-tuning due to limited device resources, and can effectively improve the efficiency of large model parameter adjustment and save device resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0063] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the large model adjustment method of the present application;

[0064] Figure 2 A schematic diagram of pipeline parallel training provided in Example 1 of the large model adjustment method of this application;

[0065] Figure 3 The adapter online configuration decision flow chart provided in Example 1 of the large model adjustment method of this application;

[0066] Figure 4 A schematic diagram of the structure of a Transformer large model fused with a serial adapter provided in Example 1 of the large model adjustment method of the present application;

[0067] Figure 5 A schematic diagram of multi-device collaborative fine-tuning training provided in Example 1 of the large model adjustment method of this application;

[0068] Figure 6 A flow chart of the second embodiment of the large model adjustment method of the present application;

[0069] Figure 7 A model segmentation deployment decision flow chart provided for the second embodiment of the large model adjustment method of this application;

[0070] Figure 8 This is a schematic diagram of the module structure of the large model adjustment device of the embodiment of the present application;

[0071] Fig. 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the large model adjustment method of the embodiment of the present application.

[0072] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0073] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0074] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0075] The main solution of the embodiment of the present application is: to split the large model to be adjusted and deploy it, to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and a corresponding adjustable adapter is embedded after each codec layer; the adjustable adapter is updated layer by layer according to the training status of the large model to be adjusted, to obtain a large model with the adjustable adapter embedded in the updated configuration; forward propagation calculation and backward propagation calculation are performed on the large model with the adjustable adapter embedded in the updated configuration, to obtain a large model with adjusted parameters.

[0076] The existing collaborative fine-tuning technology based on pipeline parallelism has the following two main problems: 1) The current collaborative training design based on pipeline parallelism generally follows a chain-like sequential training mechanism, which requires the first device in the chain to hold all input data and share relevant labels with the last device, which raises privacy issues and increases interaction overhead. In addition, only the data samples of the "head" device of the pipeline are allowed to be trained during the entire process, while other devices only serve as auxiliary computing nodes. Therefore, this pipeline parallel training framework cannot use the data of multiple devices to fine-tune a unified model. In practical applications, due to storage limitations, the data samples on a single device may be quite limited, while the multi-devices in a local area network can often collect different types of task data. The inefficient use of these data will damage the performance of the fine-tuning model. 2) Pipeline parallel training allows the use of unupdated model parameters for forward propagation calculations. In order to unify the model update direction, each device is required to store different versions of model parameters for average update, which will introduce significant storage overhead, making the end-side devices with insufficient memory and storage capacity face more severe resource challenges, which can easily lead to the failure of fine-tuning tasks. Therefore, how to effectively improve the efficiency of large model parameter adjustment and save equipment resources is a problem that needs to be solved urgently.

[0077] The present application can accurately adapt to the corresponding end-side devices by splitting and deploying the large model to be adjusted, obtain sub-models deployed on different end-side devices, and update the adjustable adapters in each sub-model layer by layer, effectively shortening the back propagation path, thereby performing forward propagation and backward propagation, and realizing fine-tuning of the large model through multiple devices, and quickly and efficiently obtaining the adjusted large model, overcoming the technical defects of being unable to use multiple devices for model fine-tuning and the failure of model fine-tuning due to limited device resources, and can effectively improve the efficiency of large model parameter adjustment and save device resources.

[0078] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a large model adjustment device, etc. The following takes the large model adjustment device as an example to illustrate this embodiment and the following embodiments.

[0079] Based on this, the embodiment of the present application provides a large model adjustment method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the large model adjustment method of the present application. In this embodiment, the large model adjustment method includes steps S10 to S30:

[0080] Step S10, splitting and deploying the large model to be adjusted to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is embedded with a corresponding adjustable adapter.

[0081] It should be noted that a large model is a machine learning model containing a large number of parameters, such as a deep learning network, which is usually used to process complex tasks such as natural language processing, image recognition, etc. In this embodiment, the large model to be adjusted is a pre-trained model, which already has the basic ability to process specific tasks, but in order to adapt to specific application scenarios, further adjustment or fine-tuning is required. For example, in a distributed computing network such as a smart home, it supports several network-connected end-side devices with limited computing power and memory to use their own local label data to collaboratively perform complex Transformer large model fine-tuning training.

[0082] It is understandable that the large model to be adjusted includes multiple codec layers, and the codec layer, namely the transformer layer, is a codec structure for processing the conversion between input data and output data. In an implementation method, the codec layer of the large model to be adjusted is split and deployed on different end-side devices, and the sub-model deployed on each device contains a different number of codec layers to adapt to the computing power and storage resources of the device.

[0083] It is worth noting that the adjustable adapter is a mechanism for adjusting model parameters. It can fine-tune the model according to the specific needs and environmental conditions of the end-side device. In this embodiment, a corresponding adjustable adapter is embedded after each codec layer, so that each sub-model can adjust parameters independently to adapt to different application scenarios and data distributions.

[0084] It is worth noting that splitting and deploying the large model to be adjusted on a group of end-side devices can adapt to the differentiated computing speed, memory capacity and link rate of the end-side devices, and can solve the computing power and memory resource limitations of a single device to support large model fine-tuning tasks. An adjustable adapter layer is embedded after each Transformer layer carried by each end-side device. During the fine-tuning process, only the parameters of the adapter layer in the active state are trained, while the original Transformer layer is always kept in an inactive state, and no parameter update calculation is performed, which improves the computing efficiency of the end-side device and reduces the memory overhead introduced by storing intermediate parameters.

[0085] In the specific implementation, the collaborative fine-tuning system adopts a hierarchical model segmentation strategy, with the Transformer module as the basic segmentation granularity and deployed on a group of end-side devices according to rules. At the same time, it maintains parameter copies of the input embedding layer (Embedding Layer, Embed) and the output head layer (Head Layer, Hed) on each device, thereby constructing a ring topology with any data source device as the start and end nodes, and switching the data source node in a polling manner in different training rounds to achieve effective utilization of the local data of all participating devices.

[0086] like Figure 2 As shown, Figure 2 The figure is a schematic diagram of pipeline parallel training, including the end-side devices u1, u2, u3, and u4. The data is forward propagated during the startup period. Fi is the forward calculation of data batch i, and Bi is the backward calculation of data batch i. Before the backward propagation (B1) of the first data batch is completed, the forward propagation of the first, second, third, and fourth data batches has been started, and they are all based on the same version of the model weights as the first data batch. This may cause the first four data batches to have different model update directions, which may prolong the convergence time. At the same time, in order to unify the model update direction of the four data batches, each device is required to store four versions of model parameters for average update, which will introduce significant storage overhead, making the end-side devices with insufficient memory and storage capacity face more severe resource challenges, which may easily lead to the failure of fine-tuning tasks.

[0087] Step S20, updating the adjustable adapter layer by layer according to the training state of the large model to be adjusted, to obtain a large model in which the adjustable adapter with updated configuration is embedded.

[0088] It should be noted that in this implementation, the adjustable adapter is dynamically configured during the task running phase, the model training status and device resource changes are perceived online, the underlying adapters are activated layer by layer as needed, and the adjustable parameters of the high-importance adapters in the activated state are adaptively increased, so as to save the device resource overhead in the early stage of training without sacrificing the convergence speed and fine-tuning performance.

[0089] In a feasible implementation, step S20 may include: obtaining the connection order of the codec layers in each sub-model in the large model to be adjusted; activating the corresponding adjustable adapter layer by layer according to the training state of the large model to be adjusted and the connection order to obtain an adjustable adapter in an activated state; adjusting the width configuration of the adjustable adapter in the activated state to obtain an adjustable adapter with an updated configuration; embedding the adjustable adapter with the updated configuration into the large model to be adjusted to obtain a large model with the adjustable adapter embedded with the updated configuration.

[0090] It should be noted that the level of the adjustable adapter embedded behind each codec layer can be determined according to the connection order of the codec layer in the large model to be adjusted. In view of the progressive learning characteristics of "from easy to difficult" in the fine-tuning training process, a strategy of dynamically expanding the amount of adjustable parameters in the activated state is adopted, that is, taking a certain number of training rounds as a cycle, by judging that the training process encounters a temporary stagnation point (that is, further training under the current adapter configuration will no longer bring significant accuracy improvement), the adapter parameters of the lower layer are gradually unfrozen from top to bottom, and the bottleneck dimensions of some important activated adapters are increased. In this implementation, a bottom-up layer-by-layer parameter solidification strategy can also be used to shorten the back propagation path. This embodiment does not make specific restrictions on this. This embodiment takes the top-down layer-by-layer parameter unfreezing strategy as an example for explanation.

[0091] It can be understood that activating the adjustable adapters of each layer layer by layer is the deep expansion of the adapter, which means that the adapter layer parameters of the top layer (that is, the layer closest to the model output) are unfrozen first at the beginning of fine-tuning training, and after every k training rounds, the adapter parameters of the next layer are unfrozen to participate in the update calculation. Once any parameter position of any adapter layer is unfrozen, it will remain activated until the training converges.

[0092] In a feasible implementation manner, the adjusting the width configuration of the adjustable adapter in the activated state to obtain the adjustable adapter with updated configuration includes: obtaining the gradient information of the current adjustable parameters of the adjustable adapter in the activated state; determining the trace of the Fisher information matrix based on the gradient information of the current adjustable parameters; sorting the adjustable adapters in the activated state based on the trace of the Fisher information matrix to obtain a target adapter; and updating the bottleneck dimension of the target adapter based on a preset bottleneck dimension growth step to obtain the adjustable adapter with updated configuration.

[0093] It should be noted that in the fine-tuning training process, the Fisher Information Matrix is ​​a key tool for measuring the importance of parameters. The Fisher Information Matrix is ​​obtained based on the gradient information of the model parameters and can provide quantitative indicators of the degree of influence of the parameters on the model output. By calculating the trace of the Fisher Information Matrix (that is, the sum of the diagonal elements of the matrix), the importance of the adjustable adapter can be determined.

[0094] It can be understood that the gradient information of all parameters in the currently activated adjustable adapter layer is calculated. These gradient information reflect the contribution of each parameter to the improvement of model performance in the current training stage. Based on these gradient information, the Fisher information matrix can be constructed and its trace can be calculated. The size of the trace can be used as a measure of the importance of the adapter. The adapter with a larger trace indicates that it has a more significant impact on the model performance. The activated adjustable adapters are sorted according to the trace of the Fisher information matrix to determine the target adapter. The target adapter refers to the important adapter in the current training stage. The bottleneck dimension of the target adapter is updated based on the preset bottleneck dimension growth step. The bottleneck dimension refers to the limitation or adjustment of the number of parameters in the adjustable adapter layer. By gradually increasing the bottleneck dimension, the important parameters can be finely adjusted, thereby further improving the performance of the model without sacrificing the convergence speed and fine-tuning performance. By guiding the parameter optimization of the adjustable adapter layer through the trace of the Fisher information matrix, the efficiency and performance of model training can be effectively balanced, ensuring efficient and accurate model fine-tuning on the end-side device.

[0095] It is worth noting that in this embodiment, the trace of the Fisher information matrix (denoted as tr(F)) can be used to evaluate the training status to guide the configuration decision of the adapter. The approximation of the Fisher information or other metrics can also be used to evaluate the training status and the real-time importance of different model parameters or modules to the training performance. This embodiment does not impose specific restrictions on this. This embodiment is illustrated by taking the trace of the Fisher information matrix as an example.

[0096] It is worth noting that the width expansion of the adapter means that for the activated adapter layer, the trace of its Fisher information matrix (i.e., tr(F) value) is calculated based on the gradient information of its current adjustable parameters, and all activated adapters are sorted in descending order according to their tr(F) values. The adapters ranked in the top α% are regarded as important adapters for the current training stage. Under the memory constraints of the end-side devices to which they belong, the bottleneck dimensions of the top α% important adapters are linearly increased, and the newly added adjustable parameters are initialized according to the zero-mean Gaussian distribution, as shown in the following formula:

[0097]

[0098] Among them, γ is the growth step of the bottleneck dimension, is the maximum allowed bottleneck dimension of the jth adapter under the memory constraint of the end-side device u.

[0099] It is worth noting that the configuration update interval is calculated by calculating the average tr(F) of all adjustable parameters and the average duration of a single training round The ratio of device calculation delay and point-to-point communication delay is used to evaluate the performance of model convergence and system timeliness under the current adapter configuration strategy, as shown in the following formula:

[0100]

[0101] Among them, TE is the training efficiency value, k is the configuration update interval, is the average duration of a single training round, and r is the number of training rounds.

[0102] When the training efficiency value of the current k rounds decreases compared with the previous 2k to k+1 rounds, that is, TE(r-2k:rk)>TE(rk:r), the configuration update interval k is linearly increased to k+Δ, so that the system continues to use the current adapter depth and width configuration, avoiding aggressive increase in the amount of adjustable parameters.

[0103] like Figure 3 As shown, Figure 3 It is the adapter online configuration decision flow chart. When the training round r=1, determine whether r%k=0 is satisfied. If not, perform the rth round of training and r=r+1. If satisfied, evaluate the training efficiency TE(rk:r), sort the importance of the adapter layers, increase the width of the first α% important adapter layers, increase the adapter activation depth, initialize the newly added model parameters, and determine whether TE(r-2k:rk)>TE(rk:r). If satisfied, configure the update interval k to k+Δ. If not satisfied, maintain the configuration update interval k.

[0104] like Figure 4 As shown, Figure 4This is a schematic diagram of the Transformer large model structure that integrates serial adapters. The large model consists of an input embedding layer (Emb), an output head layer (Hed), and multiple Transformer layers, with forward and backward propagation (Forward) and backward propagation (Backward) between the Transformer layers. Each Transformer layer consists of three residual connections and layer normalization (Add&Norm), an adapter (Adapter), a feed-forward layer (Feed-forward), and a multi-head attention mechanism (Multi-head Attention). Each adapter (Adapter) module is mainly composed of two feed-forward (Feedforward) sublayers. The first feed-forward sublayer (down-project) takes the output of the Transformer block as input and projects the original input dimension d (high-dimensional features) to m (low-dimensional features). The number of parameters of the Adapter module is limited by controlling the size of m. Then, a nonlinear layer (Nonlinearity) is passed in the middle. At the output stage, the second feed-forward sublayer (up-project) is used to restore the input dimension, and m (low-dimensional features) is remapped back to d (original high-dimensional features) as the output of the Adapter module.

[0105] In the specific implementation, during the fine-tuning training process, a top-down adapter layer-by-layer unfreezing strategy is adopted, in which only the adapter parameters in the activated state participate in the update calculation of the backward propagation. The central controller makes online decisions on the adapter configuration strategy based on the real-time observation of the computing power, memory, and link status of each end-side device during the training process, as well as the evaluation of the model convergence state, including configuring the adapter top-down unfreezing depth and the structural width of each activated adapter. Based on this configuration, for any training iteration, its forward propagation calculation starts from the end-side device holding the data sample processed by the current iteration, and the feature representation of the data is generated through the Emb layer of the device, and then passes through each collaborative end-side device in turn according to the Transformer layer deployment order, and finally transmits it to the original input device end-side device to generate the model output result, and calculates the loss value based on the corresponding data label. The propagation process forms a ring topology structure, and the adjacent end-side devices can transmit the required inter-layer parameters for calculation through point-to-point communication methods such as direct communication between devices. Similarly, the backpropagation calculation of this training iteration is performed in reverse along the edge devices in this ring topology, and the inter-layer gradient values ​​are transmitted between adjacent devices in the topology. The backpropagation process stops at the deepest adapter under the current unfreeze depth configuration.

[0106] Step S30, performing forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain the large model with adjusted parameters.

[0107] It should be noted that forward propagation and backward propagation are two key steps that are performed alternately during the training process. Forward propagation is responsible for processing the input data through the various hierarchical structures of the model and finally generating prediction results. This process involves the collaborative work of the input embedding layer, the Transformer layer, and the output head layer. In each Transformer layer, the data is first processed by residual connection and layer normalization, and then the feature dimension reduction and dimension increase operations are performed through the adapter module. Finally, it is further processed by the feedforward layer and the multi-head attention mechanism until the final feature representation is output. Backward propagation is based on the forward propagation, and according to the error calculated by the loss function, the error signal is back-propagated to update the parameters in the model. This process also needs to go through the various submodules of the Transformer layer, including the adapter module, as well as the residual connection and layer normalization. Through the gradient descent algorithm, the model parameters are adjusted to reduce the difference between the predicted results and the true values.

[0108] In a feasible implementation, step S20 may include: performing forward propagation calculations and backward propagation calculations on the sub-models of each end-side device based on the task data sample to implement pipeline parallel training and obtain a trained large model; evaluating the training convergence according to the loss value of the trained large model to obtain an evaluation result; when the evaluation result is that the training converges, using the trained large model as the large model after parameter adjustment; when the evaluation result is that the training does not converge, performing polling training on the trained large model until the training converges to obtain a large model after parameter adjustment.

[0109] It should be noted that in pipeline parallel training, each end-side device is responsible for different parts of the model, so that the entire training process can be carried out in parallel. After receiving the data sample, each end-side device first performs the forward propagation calculation and processes the data through the part of the model it is responsible for. After the processing is completed, the data will be passed to the next end-side device until all end-side devices complete the calculation of the part of the model they are responsible for in turn. After each end-side device completes its task, it passes the result to the next device, forming a pipeline operation.

[0110] It is understandable that in pipeline parallel training, since each end-side device is executed sequentially, it is necessary to ensure the efficiency and accuracy of data transmission between each device. To achieve this, high-speed network connections are usually used between end-side devices to reduce data transmission time. In addition, in order to ensure the stability of training, each end-side device needs to be synchronized to ensure that when each device is processing data, other devices have completed the corresponding computing tasks.

[0111] It is worth noting that during the implementation of pipeline parallel training, the central controller is responsible for coordinating the work of each end-side device. It reasonably allocates tasks according to the computing power, memory and link status of each device to ensure that each device can work efficiently during the training process. In addition, the central controller will dynamically adjust the pipeline parallel training strategy based on the evaluation of the model convergence state, such as adjusting task allocation, update interval, etc., to improve training efficiency and model performance. Through pipeline parallel training, the training speed of large-scale models can be significantly improved, and the time from model training to deployment can be shortened. At the same time, since each end-side device can process data in parallel, it can make full use of distributed computing resources and improve computing efficiency.

[0112] It is worth noting that during the training process, the calculation of the loss value reflects the degree of difference between the model prediction result and the true label. When the loss value reaches a low level, it indicates that the model has fitted the training data well, so it can be considered that the training process has converged. In order to accurately evaluate the training convergence, the central controller regularly calculates the loss value of the entire model and compares it with the preset threshold. If the loss value is lower than the threshold, the model is considered to have converged. At this time, the training can be stopped and the current model can be used as the final large model after parameter adjustment. On the contrary, if the loss value is still high, it indicates that the model has not fully learned the data features and needs to continue training. In this case, the central controller will decide whether to perform polling training based on the current training status and resource conditions, that is, to perform training in a loop between each end-side device until the model converges. Polling training helps ensure that each end-side device can fully participate in the training process, while avoiding the overall training efficiency reduction due to the computing bottleneck of a single device. In this way, the model can achieve the best training effect with limited computing resources.

[0113] It is worth noting that during the fine-tuning training process, the top-down adapter layer-by-layer unfreezing strategy ensures the training efficiency and performance of the model. In order to further optimize this process, a training method based on gradient accumulation can be introduced, which allows the gradient to be accumulated in each training batch instead of updating the model parameters immediately. In this way, the model can observe the changing trend of the gradient over a longer period of time, so as to adjust the parameters more accurately. Specifically, when the training batch begins, the gradient is initialized to zero. With the forward and backward propagation of each training sample, the gradient is accumulated to the initial value. Only when all samples of a batch are processed, the model parameters are updated according to the accumulated gradient. This method is particularly suitable for processing small batches of data, which can effectively reduce the variance of gradient estimation and improve the stability and convergence speed of the model.

[0114] It can be understood that in this embodiment, the adapter fine-tuning technology is used for fusion, and the adapter layer is embedded after each Transformer layer. The underlying adapter is kept in an inactive state in the initial stage of fine-tuning. Since only the adapter parameters in the thawed state are updated by gradient descent during the fine-tuning process, the characteristic of stopping the backward propagation calculation in advance is utilized, and multiple batches of data are input in advance to perform forward propagation calculations, so as to realize pipeline parallel calculations among multiple devices. Other efficient parameter fine-tuning technologies may also be used, such as LoRA, prompt fine-tuning, etc. This embodiment does not make specific restrictions on this. This embodiment takes the adapter fine-tuning technology as an example for illustration.

[0115] It is understandable that the on-demand layer-by-layer parameter activation strategy shortens the back-propagation calculation path and uses the fixed characteristics of the underlying model parameters to start the forward propagation calculation of a new round of training iterations in advance. Without introducing model obsolescence and increasing the storage overhead of parameter copies, it promotes parallel computing between devices and can significantly accelerate the fine-tuning process.

[0116] In a feasible implementation manner, forward propagation calculation and backward propagation calculation are performed on the sub-model of each end-side device based on the task data sample to realize pipeline parallel training and obtain a trained large model, including: determining a forward propagation path and a backward propagation path based on the end-side device; performing forward propagation training on the sub-model of each end-side device along the forward propagation path based on the task data sample, and performing backward propagation training on the sub-model of each end-side device along the backward propagation path to realize pipeline parallel training and obtain a trained large model.

[0117] It should be noted that the forward propagation path and the backward propagation path define the order in which data flows between various end-side devices. The forward propagation path starts from the input data and passes through the sub-models of each end-side device in turn until the last device completes the forward propagation calculation. The backward propagation path starts from the last end-side device and passes through each sub-model in reverse until the first device completes the backward propagation calculation. This path setting ensures that data can be efficiently transmitted between various end-side devices while ensuring the synchronization of the training process.

[0118] It can be understood that for any training iteration, the sampling device of the sample data is called the iteration "initiating node", and the batch data of the current iteration is input into the input embedding layer model of this device; the end-side device with the lowest unfrozen adapter layer is called the iteration "termination node", and the backpropagation calculation of the current iteration stops at the activation adapter layer carried by this device.

[0119] In the specific implementation, during the initialization phase, each device uploads its status information to the central controller, including the communication rate of all connectable point-to-point links, the computing speed and memory limit of each device. The central controller determines the model segmentation allocation strategy based on the collected system status information, and distributes the pre-trained layer parameters to the corresponding end-side devices. The central controller selects and notifies the first "initiating node" in each training round. At the same time, the central controller also sends the necessary training settings (including learning rate, local iteration number and batch size) to the end-side device to initialize training. In each training iteration, the selected "initiating node" samples a small batch of data from its local data set, inputs its local input embedding layer, and sends the output intermediate parameters to the corresponding next end-side device in the order of the Transformer block arrangement. The intermediate results generated by each end-side device sub-model are transmitted along the ring topology, and the output of the last Transformer block is sent back to the "initiating node" device to calculate the training loss, thereby completing the forward propagation of the data batch. Then, the "initiating node" starts the back propagation calculation based on the locally calculated gradient. The "initiating node" updates its output header layer parameters and transfers the gradient between the end-side devices along the reverse ring topology. The end-side device that carries the activated adapter layer in the back propagation path updates its adapter layer parameters accordingly. The back propagation calculation stops at the "termination node".

[0120] It is worth noting that in this embodiment, since the underlying model does not contain activation parameters that need to be updated by backward propagation, the training process allows the "initiating node" device to continuously sample small batches of data and input them into the distributed large model before the forward and backward propagation calculations of the current iteration are completed, so as to start the forward propagation of a new training iteration. That is, the distributed training system can execute multiple iterations of forward / backward propagation calculations at the same time, and different devices may undertake the computing tasks of different training iterations at the same time, thereby realizing a pipelined parallel training process with batch data as the granularity between end-side devices.

[0121] It is worth noting that after completing a predetermined number of iterations, the "initiating node" device reports the local loss value to the central controller to evaluate the training convergence, and at the same time selects the next "initiating node" with the best channel quality with itself, and passes it the latest parameters of the local output header layer to update its header layer. When all end-side devices in the system have served as the "initiating node", the current training round ends, and it is determined whether the current round is an adapter configuration update round. If not, a new round of training begins; if so, the central controller obtains the current Fisher matrix and other training status information of its sub-model from each end-side device, as well as the device's real-time communication, computing power, memory and other system status information, determines the adapter update configuration, and sends the configuration decision to each end-side device. The end-side device initializes the newly added activation parameters based on the latest adapter configuration, and repeats the above process until convergence, that is, the model fine-tuning is completed.

[0122] like Figure 5 As shown, Figure 5 : This is a schematic diagram of multi-device collaborative fine-tuning training. In the figure, four devices u1, u2, u3, and u4 collaborate to support the fine-tuning task of a large model with a 12-layer Transformer module. The Transformer layer is loaded onto u1, u2, u3, and u4 in order according to the ratio of 4:5:2:3. At the same time, it is assumed that the adapter thawing depth output by the adapter online configuration solution is 3. It is assumed that sampling starts from u1's local sample data. The first training iteration performs forward propagation along the path of u1→u2→u3→u4→u1, and performs reverse propagation along the path of u1→u4 for training. The above iterative process is repeated. After completing h times of u1-side data training, u1 transmits its local latest Hed layer parameters to u2 to update the u2-side Hed layer. Subsequently, u2 samples a new batch of data from its local data set and starts u2-side data training. The above training process is completed once on all end-side devices, which is regarded as a training round. This polling collaborative training process is repeated until the model converges.

[0123] In the above-mentioned multi-device collaborative fine-tuning training process, two parallel processing operations are further incorporated: 1) In different iterations of data training on a device side, the backward propagation update calculation process of the model and the forward propagation calculation process before the adapter unfreeze layer are independent of each other and can be processed in parallel at the same time; 2) In the iterations of two consecutive device-side data training, the backward propagation update calculation process of the model Transformer layer and the parameter transmission process of the Hed layer are independent of each other and can be executed in parallel at the same time. The above two types of calculation-calculation and calculation-transmission parallel processing help to improve device resource utilization and accelerate the training process.

[0124] The present embodiment provides a large model adjustment method. The present embodiment first divides and deploys the large model to be adjusted to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and a corresponding adjustable adapter is embedded behind each codec layer; the adjustable adapter is updated layer by layer according to the training state of the large model to be adjusted to obtain a large model embedded with the adjustable adapter after the updated configuration; forward propagation calculation and backward propagation calculation are performed on the large model embedded with the adjustable adapter after the updated configuration to obtain a large model with adjusted parameters, which can effectively improve the efficiency of large model parameter adjustment and save device resources.

[0125] In summary, this embodiment can accurately adapt to the corresponding end-side devices by splitting and deploying the large model to be adjusted, obtain sub-models deployed on different end-side devices, and update the adjustable adapters in each sub-model layer by layer, thereby effectively shortening the back propagation path, thereby performing forward propagation and backward propagation, and realizing fine-tuning of the large model through multiple devices, and quickly and efficiently obtaining the adjusted large model, overcoming the technical defects of being unable to use multiple devices for model fine-tuning and the failure of model fine-tuning due to limited device resources, and can effectively improve the efficiency of large model parameter adjustment and save device resources.

[0126] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 6 , the step S10 further includes steps S101-S104:

[0127] Step S101: determining the number of codec layers to be allocated to each end-side device according to the memory capacity of each end-side device.

[0128] It should be noted that the memory capacity of the end-side device is one of the important factors affecting its processing capability. In this embodiment, by considering the memory capacity of the end-side device, the codec layers can be allocated more reasonably to ensure that each device can efficiently perform its assigned tasks and avoid performance bottlenecks or training failures caused by insufficient memory.

[0129] It is understandable that in the process of model segmentation and allocation, with the goal of minimizing training latency, the differentiated computing speed, memory capacity, and link rate between the collaborating end-side devices are taken into consideration, and the device ring topology construction decision is made based on graph representation and optimization technology. The allocation and deployment plan of the Transformer module on each device is determined to shorten the idle waiting time of heterogeneous devices during fine-tuning training and enhance the convergence acceleration advantage brought by pipeline parallel training.

[0130] In the specific implementation, before starting the fine-tuning task, the central controller first determines the large model segmentation deployment strategy based on the available computing power, memory capacity, point-to-point communication rate and other information of the end-side devices in the system, including after which Transformer layers the model is segmented to form multiple groups of sub-models containing continuous Transformer layers, and to which device each sub-model is assigned for processing. The central controller broadcasts the segmentation deployment strategy and distributes the parameters of each sub-model to the corresponding end-side device according to the strategy to perform the corresponding forward and backward propagation calculations. Among them, an adapter module with a specific initialization structure and parameters is connected in series after each Transformer layer, and the central controller sends it together with the associated Transformer layer parameters to the corresponding end-side device. In addition to the assigned sub-model, each end-side device also locally deploys copies of Emb and Hed to perform feature representation of its local sample data, generate model output results, calculate loss values, and update output header layer parameters.

[0131] In a feasible implementation, step S101 may include: obtaining multiple end-side devices and corresponding forward propagation calculation delays, and sorting each end-side device based on the forward propagation calculation delay to obtain sorted end-side devices; determining a threshold value for the number of codec layers of each end-side device according to the memory capacity of each end-side device; and determining the number of codec layer allocations of each end-side device according to the threshold value for the number of codec layers and the sorted end-side devices.

[0132] It should be noted that for any collaborative fine-tuning task, a model segmentation deployment is performed before the system is built, and the end-side devices and the direct transmission links between them are modeled as a directed graph G = (U, A), where U is the device vertex set, A is the directed edge set, and its element a = (u, u′) represents the directed transmission link from device u to device u′. The link transmission delay is defined as The forward propagation computation delay of executing a Transformer module on the device is τ tr (u, l), the forward propagation calculation delay of the end-side device is a reflection of the computing power of the end-side device. In this embodiment, by considering the forward propagation calculation delay of the end-side device, the encoding and decoding layers can be allocated more reasonably to ensure that each device can efficiently perform its assigned tasks and avoid performance bottlenecks or training failures caused by long calculation delays.

[0133] It can be understood that the number of Transformer blocks allocated to each device is optimized based on the available memory capacity of each device and the size of each Transformer block, with the goal of minimizing the total computation time of all devices.

[0134] In the specific implementation, the end-side devices are sorted in descending order according to their computing power (i.e., forward propagation calculation delay); for each sorted end-side device, the maximum number of Transformer blocks that can be allocated, that is, the threshold for the number of encoding and decoding layers, is calculated based on its memory capacity and the memory requirement of a single Transformer block; the number of unallocated Transformer blocks is updated, and the number of Transformer blocks to be allocated to the next device is determined based on the sorting until all devices are traversed; if there are still unallocated Transformer blocks after the traversal, it means that the edge devices participating in the collaborative fine-tuning cannot carry the target large model, and the model segmentation deployment algorithm is terminated.

[0135] It is worth noting that the Transformer block number allocation optimization process allows some terminals with limited computing power or memory to not be assigned Transformer blocks. They only carry the input embedding layer (Emb) and the output head layer (Hed), are responsible for processing input data and calculating losses, and do not participate in the forward and backward propagation calculations of the Transformer blocks.

[0136] Step S102, dividing the large model to be adjusted into a plurality of sub-models according to the number of allocated coding and decoding layers.

[0137] It should be noted that the size and complexity of each sub-model can be dynamically adjusted according to the number of Transformer blocks corresponding to each end-side device to adapt to the computing power of different devices. For example, for edge devices with strong computing power, more Transformer blocks can be assigned to enable them to undertake more complex tasks; while for devices with weaker computing power, fewer Transformer blocks are assigned to mainly be responsible for basic input and output processing.

[0138] It can be understood that if the large model to be adjusted in this embodiment has 12 layers of Transformer modules, and the number of Transformer blocks corresponding to the end-side device is 4, 5, 2, and 3 respectively, then the 12-layer Transformer module can be divided into four sub-models, namely, a sub-model containing 4 Transformer blocks, a sub-model containing 5 Transformer blocks, a sub-model containing 2 Transformer blocks, and a sub-model containing 3 Transformer blocks. In this way, each end-side device can effectively participate in the processing of the large model according to its own computing power, while ensuring the efficiency and accuracy of the overall task. In addition, this dynamic allocation mechanism can also effectively improve the flexibility and scalability of the system. In practical applications, as the performance of the device improves or the task requirements change, the system can flexibly adjust the number of Transformer blocks of each sub-model to adapt to new requirements. For example, when the computing power of an edge device is enhanced, more Transformer blocks can be reallocated to the device so that it can handle more complex tasks, thereby making full use of the computing resources of the device.

[0139] Step S103, obtaining a target Hamiltonian circuit.

[0140] It should be noted that a Hamiltonian circuit is a special graph theory path that passes through each vertex in the graph exactly once and eventually returns to the starting point. In this embodiment, the purpose of obtaining the target Hamiltonian circuit is to determine the data flow order between sub-models and ensure that the data dependencies between sub-models are correctly processed when the large model is split, thereby ensuring the operating efficiency and accuracy of the entire model.

[0141] In a feasible implementation, step S103 may include: obtaining a transmission link between each end-side device, and constructing a directed graph based on each end-side device and the transmission link, wherein the directed graph includes multiple original nodes; adding a virtual node to the directed graph, and constructing a directed edge based on the virtual node and each original node; obtaining the weight of each directed edge, and generating a directed weighted graph based on the weight of each directed edge; and determining a target Hamiltonian circuit based on the directed weighted graph.

[0142] It should be noted that the end-side devices and the direct transmission links between them are modeled as a directed graph G = (U, A), where U is a set of device nodes, A is a set of directed edges, and its element a = (u, u′) represents the directed transmission link from device u to device u′, that is, the original node in the directed graph is the end-side device, and the directed edge is the directed transmission link from device u to device u′.

[0143] It can be understood that it is checked whether there are devices that are not assigned Transformer blocks. If so, the nodes corresponding to these devices and their associated edges are removed from the directed graph G; a virtual node u is added to the directed graph G. end , and the virtual node u end Establish directed edges with all nodes in the G directed graph; calculate the weights of each directed edge to form a new graph, namely the directed weighted graph G′(U - ,A - ,W), the weight of each directed edge is as follows:

[0144]

[0145] Among them, Weight(u,u') is the weight of the directed edge, U － is a node set of a directed weighted graph, u and u' are device nodes on the end side, and u end A virtual node.

[0146] In a feasible implementation, the determining of the target Hamiltonian circuit based on the directed weighted graph includes: taking a virtual node in the directed weighted graph as a starting point and performing path planning to generate a path table; determining the path length from each visited node in the visited node set to an unvisited node based on the path table; determining whether there is a reference Hamiltonian circuit based on the path length; when the reference Hamiltonian circuit exists, removing the virtual node and reconstructing the path through a backtracking table to generate a target Hamiltonian circuit.

[0147] It should be noted that the dynamic programming method is used to find the shortest directed Hamiltonian circuit based on the directed weighted graph. The circuit passes through each vertex once along the directed edge and finally returns to the starting vertex, thereby constructing a The path table Dis(V,v), where V is the set of visited vertices, v is the current vertex, and each element in the table represents the path from the virtual vertex u end Starting from, passing through all vertices in set V, and finally reaching vertex v, the shortest path length (i.e., the sum of edge weights)

[0148] It is understandable that for each subset and vertex v∈V, calculate the distance from v to the unvisited vertex The path length is calculated, and the path table and backtracking table are updated to obtain Dis(V′,v′) and Back(V′,v′). After calculating the shortest paths of all subsets, find a path that can return to the virtual vertex u. end The shortest Hamiltonian circuit; if the optimal path is found, the complete path is reconstructed through the backtracking table Back(V′,v′) and the virtual vertex u is removed end To form the final path, that is, the target Hamiltonian circuit.

[0149] It is worth mentioning that the backtracking table is an important data structure used to record the predecessor node of each node in the path planning process. Through the backtracking table, it is possible to trace back from the end point to the starting point, thereby reconstructing a complete Hamiltonian circuit. In the process of dynamic planning, the construction of the backtracking table is completed step by step. Whenever the shortest path from a virtual vertex to a visited vertex is determined, the predecessor node of the vertex in the backtracking table will be updated.

[0150] Step S104: deploying the plurality of sub-models in sequence on corresponding end-side devices based on the target Hamiltonian circuit to obtain sub-models deployed on different end-side devices.

[0151] It should be noted that the target Hamiltonian circuit is a path that covers all the end-side device nodes and visits each node only once, and finally returns to the shortest path of the starting point. After determining the target Hamiltonian circuit, the sub-models can be deployed to the corresponding end-side devices in the order of the circuit.

[0152] In the specific implementation, the deployment order of the sub-models is first determined, that is, arranged according to the node order of the target Hamiltonian circuit. Then, for each end-side device, the corresponding sub-model is allocated according to its position in the target Hamiltonian circuit. During the allocation process, factors such as the computing power, storage space, and network bandwidth of the device need to be considered to ensure that the sub-model can run normally on the device and meet performance requirements.

[0153] In a feasible implementation, step S104 may include: determining the arrangement order of the end-side devices based on the target Hamiltonian circuit; obtaining model parameters of each sub-model; and sending the model parameters to the corresponding end-side devices in sequence according to the arrangement order of the end-side devices to obtain sub-models deployed on different end-side devices.

[0154] It should be noted that the arrangement order of the end-side devices determined according to the target Hamiltonian circuit can ensure the order and efficiency of the sub-model deployment. The model parameters of each sub-model are obtained, including but not limited to the model's weight, bias, activation function and other information, which are the basis for the sub-model to perform reasoning and learning.

[0155] It is understandable that the model parameters are sent to the corresponding end-side devices in sequence according to the arrangement order of the end-side devices. During the sending process, compression technology can be used to reduce the amount of data transmission while ensuring the integrity and accuracy of the data.

[0156] like Figure 7 As shown, Figure 7For the model segmentation deployment decision flowchart, the device connection topology diagram is constructed, the maximum number of Trms are allocated according to the descending abnormality of the device computing power, and it is determined whether there are unallocated Trms. If so, the device connection diagram is updated. If not, the edge weights of the connection diagram are determined, the shortest Hamiltonian circuit is determined, and then the Trm deployment strategy is determined.

[0157] In this embodiment, by accurately determining the number of codec layer allocations and then dividing the model according to the memory capacity of each end-side device, and then deploying the sub-models in sequence on the corresponding end-side devices according to the target Hamiltonian circuit, the idle waiting time of heterogeneous devices during fine-tuning training can be effectively shortened, thereby improving the convergence speed of training.

[0158] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the large model adjustment method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0159] This application also provides a large model adjustment device, please refer to Figure 8 , the large model adjustment device comprises:

[0160] The deployment module 10 is used to split and deploy the large model to be adjusted to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is embedded with a corresponding adjustable adapter.

[0161] The updating module 20 is used to update the adjustable adapter layer by layer according to the training status of the large model to be adjusted, so as to obtain a large model in which the adjustable adapter after the updated configuration is embedded.

[0162] The propagation module 30 is used to perform forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain the large model with adjusted parameters.

[0163] The present embodiment provides a large model adjustment device. The present embodiment divides and deploys the large model to be adjusted to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and a corresponding adjustable adapter is embedded behind each codec layer; the adjustable adapter is updated layer by layer according to the training status of the large model to be adjusted to obtain a large model embedded with the adjustable adapter after the updated configuration; forward propagation calculation and backward propagation calculation are performed on the large model embedded with the adjustable adapter after the updated configuration to obtain a large model with adjusted parameters, which can effectively improve the efficiency of large model parameter adjustment and save equipment resources.

[0164] In summary, this embodiment can accurately adapt to the corresponding end-side devices by splitting and deploying the large model to be adjusted, obtain sub-models deployed on different end-side devices, and update the adjustable adapters in each sub-model layer by layer, thereby effectively shortening the back propagation path, thereby performing forward propagation and backward propagation, and realizing fine-tuning of the large model through multiple devices, and quickly and efficiently obtaining the adjusted large model, overcoming the technical defects of being unable to use multiple devices for model fine-tuning and the failure of model fine-tuning due to limited device resources, and can effectively improve the efficiency of large model parameter adjustment and save device resources.

[0165] Optionally, the deployment module 10 is further used to determine the number of codec layers allocated to each end-side device according to the memory capacity of each end-side device; divide the large model to be adjusted into multiple sub-models according to the number of codec layers allocated; obtain a target Hamiltonian circuit; and deploy the multiple sub-models in sequence on the corresponding end-side devices based on the target Hamiltonian circuit to obtain sub-models deployed on different end-side devices.

[0166] Optionally, the deployment module 10 is further used to obtain multiple end-side devices and corresponding forward propagation calculation delays, and sort each end-side device based on the forward propagation calculation delay to obtain sorted end-side devices; determine the threshold value of the number of codec layers of each end-side device according to the memory capacity of each end-side device; determine the number of codec layers allocated to each end-side device according to the threshold value of the number of codec layers and the sorted end-side devices.

[0167] Optionally, the deployment module 10 is also used to obtain the transmission links between each end-side device, and construct a directed graph based on each end-side device and the transmission links, wherein the directed graph includes multiple original nodes; add virtual nodes to the directed graph, and construct directed edges based on the virtual nodes and the original nodes; obtain the weight of each directed edge, and generate a directed weighted graph based on the weight of each directed edge; determine the target Hamiltonian circuit based on the directed weighted graph.

[0168] Optionally, the deployment module 10 is further used to take the virtual node in the directed weighted graph as the starting point and perform path planning to generate a path table; determine the path length from each visited node in the visited node set to the unvisited node based on the path table; determine whether there is a reference Hamiltonian circuit based on the path length; when the reference Hamiltonian circuit exists, remove the virtual node and reconstruct the path through the backtracking table to generate a target Hamiltonian circuit.

[0169] Optionally, the deployment module 10 is further used to determine the arrangement order of the end-side devices based on the target Hamiltonian circuit; obtain model parameters of each sub-model; and send the model parameters to the corresponding end-side devices in sequence according to the arrangement order of the end-side devices to obtain sub-models deployed on different end-side devices.

[0170] Optionally, the update module 20 is also used to obtain the connection order of the codec layers in each sub-model in the large model to be adjusted; activate the corresponding adjustable adapter layer by layer according to the training state of the large model to be adjusted and the connection order to obtain the adjustable adapter in the activated state; adjust the width configuration of the adjustable adapter in the activated state to obtain the adjustable adapter with updated configuration; embed the adjustable adapter with updated configuration into the large model to be adjusted to obtain the large model with the adjustable adapter embedded with the updated configuration.

[0171] Optionally, the update module 20 is also used to obtain the gradient information of the current adjustable parameters of the activated adjustable adapter; determine the trace of the Fisher information matrix based on the gradient information of the current adjustable parameters; sort the adjustable adapters in the activated state based on the trace of the Fisher information matrix to obtain a target adapter; update the bottleneck dimension of the target adapter based on a preset bottleneck dimension growth step to obtain the adjustable adapter after the updated configuration.

[0172] Optionally, the propagation module 30 is also used to perform forward propagation calculations and backward propagation calculations on the sub-models of each end-side device based on the task data samples to achieve pipeline parallel training and obtain a trained large model; evaluate the training convergence according to the loss value of the trained large model to obtain an evaluation result; when the evaluation result is that the training has converged, use the trained large model as the large model after parameter adjustment; when the evaluation result is that the training has not converged, perform polling training on the trained large model until the training converges to obtain a large model after parameter adjustment.

[0173] Optionally, the propagation module 30 is also used to determine a forward propagation path and a backward propagation path based on the end-side device; perform forward propagation training on the sub-model of each end-side device along the forward propagation path based on the task data sample, and perform backward propagation training on the sub-model of each end-side device along the backward propagation path to realize pipeline parallel training and obtain a trained large model.

[0174] The large model adjustment device provided by the present application adopts the large model adjustment method in the above embodiment, which can solve the technical problem of how to effectively improve the efficiency of large model parameter adjustment and save equipment resources. Compared with the prior art, the beneficial effects of the large model adjustment device provided by the present application are the same as the beneficial effects of the large model adjustment method provided by the above embodiment, and the other technical features in the large model adjustment device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0175] The present application provides a large model adjustment device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the large model adjustment method in the above-mentioned embodiment one.

[0176] Reference below Fig. 9 , which shows a schematic diagram of the structure of a large model adjustment device suitable for implementing the embodiment of the present application. The large model adjustment device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The large model adjustment device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0177] like Fig. 9 As shown, the large model adjustment device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the large model adjustment device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the large model adjustment device to communicate with other devices wirelessly or by wire to exchange data. Although the large model adjustment device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0178] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0179] The large model adjustment device provided by the present application adopts the large model adjustment method in the above embodiment, which can solve the technical problem of how to effectively improve the efficiency of large model parameter adjustment and save equipment resources. Compared with the prior art, the beneficial effects of the large model adjustment device provided by the present application are the same as the beneficial effects of the large model adjustment method provided by the above embodiment, and the other technical features in the large model adjustment device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0180] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0181] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0182] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the large model adjustment method in the above-mentioned embodiment.

[0183] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0184] The computer-readable storage medium may be included in the large model adjustment device; or may exist independently without being assembled into the large model adjustment device.

[0185] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the large model adjustment device, the large model adjustment device: divides and deploys the large model to be adjusted, and obtains sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and a corresponding adjustable adapter is embedded behind each codec layer; updates the adjustable adapter layer by layer according to the training status of the large model to be adjusted, and obtains a large model with the adjustable adapter embedded after the updated configuration; performs forward propagation calculation and backward propagation calculation on the large model with the adjustable adapter embedded after the updated configuration, and obtains the large model with adjusted parameters.

[0186] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0187] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0188] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0189] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned large model adjustment method, and can solve the technical problem of how to effectively improve the efficiency of large model parameter adjustment and save equipment resources. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the large model adjustment method provided in the above-mentioned embodiment, and will not be repeated here.

[0190] The present application also provides a computer program product, including a computer program, which implements the steps of the large model adjustment method as described above when executed by a processor.

[0191] The computer program product provided by this application can solve the technical problem of how to effectively improve the efficiency of large model parameter adjustment and save equipment resources. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the large model adjustment method provided by the above embodiment, which will not be repeated here.

[0192] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A large model adjustment method, characterized in that: The method comprises: The large model to be adjusted is split and deployed to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is embedded with a corresponding adjustable adapter; updating the adjustable adapter layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model in which the adjustable adapter with the updated configuration is embedded; The large model of the adjustable adapter embedded with the updated configuration is subjected to forward propagation calculation and backward propagation calculation to obtain the large model with adjusted parameters.

2. The method according to claim 1, characterized in that The large model to be adjusted is divided and deployed to obtain multiple sub-models and corresponding terminal-side devices, including: Determine the number of codec layers to be allocated to each end-side device according to the memory capacity of each end-side device; Dividing the large model to be adjusted into a plurality of sub-models according to the number of allocated coding and decoding layers; Get the target Hamiltonian circuit; Based on the target Hamiltonian circuit, the plurality of sub-models are sequentially deployed on corresponding end-side devices to obtain sub-models deployed on different end-side devices.

3. The method according to claim 2, characterized in that The step of determining the number of codec layers to be allocated to each end-side device according to the memory capacity of each end-side device includes: Acquire multiple end-side devices and corresponding forward propagation calculation delays, and sort the end-side devices based on the forward propagation calculation delays to obtain sorted end-side devices; Determine the threshold of the number of codec layers of each end-side device according to the memory capacity of each end-side device; The number of codec layers to be allocated to each end-side device is determined according to the codec layer number threshold and the sorted end-side devices.

4. The method according to claim 2, characterized in that The step of obtaining a target Hamiltonian circuit comprises: Acquire transmission links between the end-side devices, and construct a directed graph according to the end-side devices and the transmission links, wherein the directed graph includes a plurality of original nodes; Adding a virtual node to the directed graph, and constructing directed edges based on the virtual node and each original node; Obtaining the weight of each directed edge, and generating a directed weighted graph based on the weight of each directed edge; A target Hamiltonian circuit is determined based on the directed weighted graph.

5. The method according to claim 4, characterized in that Determining a target Hamiltonian circuit based on the directed weighted graph comprises: Taking the virtual node in the directed weighted graph as a starting point and performing path planning to generate a path table; Determine the path length from each visited node in the visited node set to an unvisited node based on the path table; determining whether there is a reference Hamiltonian circuit according to the path length; When the reference Hamiltonian circuit exists, the virtual node is removed and the path is reconstructed through the backtracking table to generate a target Hamiltonian circuit.

6. The method according to claim 2, characterized in that The method of sequentially deploying the plurality of sub-models on corresponding end-side devices based on the target Hamiltonian circuit to obtain sub-models deployed on different end-side devices includes: Determining an arrangement order of the end-side devices based on the target Hamiltonian circuit; Obtain model parameters of each sub-model; The model parameters are sent to the corresponding end-side devices in sequence according to the arrangement order of the end-side devices to obtain sub-models deployed on different end-side devices.

7. The method according to claim 1, characterized in that The step of updating the adjustable adapter layer by layer according to the training state of the large model to be adjusted to obtain a large model in which the adjustable adapter with the updated configuration is embedded includes: Obtaining the connection order of the codec layers in each sub-model in the large model to be adjusted; Activating the corresponding adjustable adapters layer by layer according to the training state of the large model to be adjusted and the connection sequence to obtain an adjustable adapter in an activated state; Adjusting the width configuration of the adjustable adapter in the activated state to obtain an adjustable adapter with an updated configuration; The adjustable adapter after the updated configuration is embedded into the large model to be adjusted to obtain the large model in which the adjustable adapter after the updated configuration is embedded.

8. The method according to claim 7, characterized in that The step of adjusting the width configuration of the adjustable adapter in the activated state to obtain the adjustable adapter after the updated configuration includes: Acquiring gradient information of current adjustable parameters of the adjustable adapter in the activated state; Determining a trace of a Fisher information matrix based on the gradient information of the current adjustable parameter; sorting the adjustable adapters in the activation state based on the trace of the Fisher information matrix to obtain a target adapter; The bottleneck dimension of the target adapter is updated based on a preset bottleneck dimension growth step to obtain an adjustable adapter after the updated configuration.

9. The method according to claim 1, characterized in that The step of performing forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain the large model with adjusted parameters includes: Based on the task data samples, forward propagation calculation and backward propagation calculation are performed on the sub-models of each end-side device to achieve pipeline parallel training and obtain the trained large model; Evaluate the training convergence according to the loss value of the trained large model to obtain an evaluation result; When the evaluation result is that the training has converged, the trained large model is used as the large model after parameter adjustment; When the evaluation result is that the training has not converged, the trained large model is subjected to polling training until the training converges, thereby obtaining the large model with adjusted parameters.

10. The method according to claim 9, characterized in that The forward propagation calculation and the backward propagation calculation are performed on the sub-models of each end-side device based on the task data sample to realize pipeline parallel training and obtain the trained large model, including: Determining a forward propagation path and a backward propagation path based on the terminal side device; Based on the task data samples, forward propagation training is performed on the sub-model of each end-side device along the forward propagation path, and backward propagation training is performed on the sub-model of each end-side device along the backward propagation path to realize pipeline parallel training and obtain a trained large model.

11. A large model adjustment device, characterized in that: The large model adjustment device comprises: A deployment module is used to split and deploy the large model to be adjusted to obtain sub-models deployed on different end-side devices, wherein each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is embedded with a corresponding adjustable adapter; An updating module, used for updating the adjustable adapter layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model of the adjustable adapter embedded with the updated configuration; The propagation module is used to perform forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain the large model with adjusted parameters.

12. A large model adjustment device, characterized in that: The large model adjustment device comprises: a memory, a processor, and a large model adjustment program stored in the memory and executable on the processor, wherein the large model adjustment program is configured to implement the large model adjustment method according to any one of claims 1 to 10.

13. A storage medium, characterized in that: The storage medium stores a large model adjustment program, and when the large model adjustment program is executed by the processor, the large model adjustment method according to any one of claims 1 to 10 is implemented.

Images