Large model adjustment method and apparatus, and device and storage medium

Abstract

Disclosed in the present application are a large model adjustment method and apparatus, and a device and a storage medium. The method comprises: performing segmentation and deployment on a large model to be adjusted, so as to obtain sub-models deployed on different end-side devices, wherein each sub-model comprises a different number of encoding and decoding layers and a different number of adjustable adapters, and a corresponding adjustable adapter is embedded after each encoding and decoding layer; on the basis of the training state of the large model to be adjusted, updating the adjustable adapters layer by layer, so as to obtain a large model embedded with the configuration-updated adjustable adapters; and performing forward propagation computation and backward propagation computation on the large model embedded with the configuration-updated adjustable adapters, so as to obtain a parameter-adjusted large model.

Description

Large model adjustment methods, devices, equipment and storage media Technical Field

[0001] This application relates to the field of large model technology, and in particular to large model adjustment methods, apparatus, devices and storage media. Background Technology

[0002] In recent years, general-purpose large-scale model technology based on the Transformer architecture has made significant progress in the field of artificial intelligence, demonstrating outstanding performance in various tasks such as natural language processing and computer vision. These large models, pre-trained on massive datasets, acquire powerful generalization capabilities and adapt to specific tasks during the fine-tuning phase, achieving superior performance on multiple benchmarks. The fine-tuning phase of large models relies on domain-specific sample data, which is widely distributed across edge devices (such as smartphones and IoT devices) and continuously generated during user interaction. To effectively utilize distributed data while protecting data privacy, performing large-scale model fine-tuning tasks on edge devices has become a hot technology. Mainstream general-purpose large models typically have multi-layered Transformer modules, each layer containing multi-head self-attention and feedforward neural networks. These hierarchical structures enable Transformers to efficiently learn complex data patterns, but also introduce large-scale matrix operations. Performing such computationally intensive intelligent tasks on edge devices faces significant computational resource bottlenecks, easily leading to processing latency and excessive energy consumption. Furthermore, large models typically have hundreds of millions or even tens of billions of parameters. Their forward / backward propagation computations require vast amounts of memory to store intermediate results such as activation values, gradients, and optimizer states, posing a significant challenge to the memory capacity of edge devices. This makes them prone to memory overflow issues during model loading and data processing, leading to abnormal interruptions in fine-tuning tasks. By leveraging the communication interconnection and resource complementarity capabilities between connected edge devices, and utilizing model segmentation and multi-device pipeline parallel computing techniques, large models are divided into multiple sub-models, which are then distributed to different edge devices for collaborative execution. This significantly reduces the computational and memory resource consumption of individual devices. These edge devices coexist on a local area network, forming a pipeline topology based on the hierarchical order of their deployed sub-models within the original large model. The data samples required for fine-tuning training are stored in the "head" device of the pipeline, and their corresponding sample labels are passed to the "tail" device. During forward propagation, each device applies its assigned sub-model layer to the input passed from the previous stage and passes the output of the last layer to the next stage device. Backpropagation proceeds in reverse order, with gradient information being passed back to each device sequentially. To improve training efficiency, multiple devices can process different batches of data simultaneously. A copy of the sub-model weights used by each device during forward propagation is saved for subsequent backpropagation of the corresponding batches.

[0003] However, existing pipelined parallelism-based collaborative fine-tuning techniques suffer from two main problems: 1) Current pipelined parallelism-based collaborative training designs generally follow a chain-like sequential training mechanism. This mechanism requires the first device in the chain to hold all input data and share relevant labels with the last device, raising privacy concerns and increasing interaction overhead. Furthermore, only data samples from the pipeline "head" device are allowed for training, while other devices serve only as auxiliary computing nodes. Therefore, this pipelined parallel training framework cannot utilize data from multiple devices to fine-tune a unified model. In practical applications, due to storage limitations, the data samples on a single device may be quite limited, while multiple devices within a local area network can often collect different types of task data. Inefficient use of this data will impair the performance of the fine-tuned model. 2) Pipeline parallel training allows forward propagation computation using unupdated model parameters. To unify the model update direction, each device is required to store different versions of model parameters for averaging updates. This introduces significant storage overhead, making the already limited memory and storage capacity of edge devices face even greater resource challenges, easily leading to fine-tuning task failures.

[0004] Therefore, how to effectively improve the efficiency of large model parameter tuning and save equipment resources is an urgent problem to be solved.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide a method, apparatus, device and storage medium for adjusting large models, aiming to solve the technical problem of how to effectively improve the efficiency of large model parameter tuning and save equipment resources.

[0007] To achieve the above objectives, this application proposes a large model adjustment method, which includes:

[0008] The large model to be adjusted is divided and deployed to obtain sub-models deployed on different end devices. Each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is backed by a corresponding adjustable adapter.

[0009] The adjustable adapter is updated layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model with the adjusted adapter embedded in the updated configuration.

[0010] Forward and backward propagation calculations are performed on the large model of the embedded and updated adjustable adapter to obtain the large model with adjusted parameters.

[0011] In one embodiment, the step of segmenting and deploying the large model to be adjusted to obtain multiple sub-models and corresponding edge devices includes:

[0012] The number of codec layers allocated to each end-side device is determined based on the memory capacity of each end-side device.

[0013] The large model to be adjusted is divided into multiple sub-models according to the number of encoding / decoding layers allocated;

[0014] Obtain the target Hamiltonian cycle;

[0015] Based on the target Hamiltonian circuit, multiple sub-models are sequentially deployed on corresponding end-side devices to obtain sub-models deployed on different end-side devices.

[0016] In one embodiment, determining the number of codec layers allocated to each end-side device based on the memory capacity of each end-side device includes:

[0017] Multiple end-side devices and their corresponding forward propagation calculation delays are obtained, and the end-side devices are sorted based on the forward propagation calculation delays to obtain the sorted end-side devices.

[0018] The threshold for the number of encoding / decoding layers for each end-side device is determined based on the memory capacity of each end-side device.

[0019] The number of codec layers allocated to each end-side device is determined based on the threshold number of codec layers and the sorted end-side devices.

[0020] In one embodiment, obtaining the target Hamiltonian circuit includes:

[0021] Obtain the transmission links between each end-side device, and construct a directed graph based on each end-side device and the transmission links, wherein the directed graph includes multiple original nodes;

[0022] Add virtual nodes to the directed graph, and construct directed edges based on the virtual nodes and each original node;

[0023] Obtain the weight of each directed edge, and generate a directed weighted graph based on the weight of each directed edge;

[0024] The target Hamiltonian cycle is determined based on the directed weighted graph.

[0025] In one embodiment, determining the target Hamiltonian circuit based on the directed weighted graph includes:

[0026] Using the virtual nodes in the directed weighted graph as starting points, path planning is performed to generate a path table;

[0027] The path length from each visited node to an unvisited node in the visited node set is determined based on the path table.

[0028] Determine whether a reference Hamiltonian cycle exists based on the path length;

[0029] When the reference Hamiltonian cycle exists, the virtual node is removed and the path is reconstructed using a backtracking table to generate the target Hamiltonian cycle.

[0030] In one embodiment, the step of sequentially deploying multiple sub-models on corresponding end-side devices based on the target Hamiltonian circuit to obtain sub-models deployed on different end-side devices includes:

[0031] The arrangement order of the end-side devices is determined based on the target Hamiltonian circuit;

[0032] Obtain the model parameters for each sub-model;

[0033] The model parameters are sent sequentially to the corresponding end-side devices according to the arrangement order of the end-side devices, so as to obtain sub-models deployed on different end-side devices.

[0034] In one embodiment, updating the tunable adapter layer by layer according to the training state of the large model to be adjusted to obtain a large model with an embedded updated tunable adapter includes:

[0035] Obtain the connection order of the encoding and decoding layers in each sub-model within the larger model to be adjusted;

[0036] Based on the training state of the large model to be adjusted and the connection order, the corresponding adjustable adapters are activated layer by layer to obtain the activated adjustable adapters.

[0037] Adjust the width configuration of the activated adjustable adapter to obtain the adjusted adapter with the updated configuration;

[0038] The updated adjustable adapter is embedded into the large model to be adjusted to obtain a large model with the updated adjustable adapter.

[0039] In one embodiment, adjusting the width configuration of the activated adjustable adapter to obtain an updated adjustable adapter includes:

[0040] Obtain the gradient information of the current adjustable parameters of the activated adjustable adapter;

[0041] The trace of the Fisher information matrix is ​​determined based on the gradient information of the currently adjustable parameters;

[0042] The tunable adapters in the active states are sorted based on the trace of the Fisher information matrix to obtain the target adapter;

[0043] The target adapter is updated in the bottleneck dimension based on a preset bottleneck dimension growth step size to obtain an adjustable adapter with updated configuration.

[0044] In one embodiment, performing forward and backward propagation calculations on the large model of the embedded updated adjustable adapter to obtain the parameter-adjusted large model includes:

[0045] Based on the task data samples, forward and backward propagation calculations are performed on the sub-models of each end device to achieve pipelined parallel training and obtain the trained large model.

[0046] The training convergence is evaluated based on the loss value of the trained large model, and the evaluation result is obtained.

[0047] When the evaluation result indicates that the training has converged, the trained large model is used as the parameter-adjusted large model.

[0048] When the evaluation result indicates that the training has not converged, the trained large model is subjected to polling training until the training converges, resulting in a parameter-adjusted large model.

[0049] In one embodiment, the forward and backward propagation calculations are performed on the sub-models of each end-side device based on task data samples to achieve pipelined parallel training and obtain a trained large model, including:

[0050] The forward propagation path and the backward propagation path are determined based on the end-side device;

[0051] Based on the task data samples, the sub-models of each end-side device are trained by forward propagation along the forward propagation path, and the sub-models of each end-side device are trained by backward propagation along the backward propagation path, so as to realize pipelined parallel training and obtain the trained large model.

[0052] Furthermore, to achieve the above objectives, this application also proposes a large model adjustment device, which includes:

[0053] The deployment module is used to divide and deploy the large model to be adjusted into sub-models deployed on different end devices. Each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer has a corresponding adjustable adapter embedded in it.

[0054] The update module is used to update the adjustable adapter layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model with the adjusted adapter embedded with the updated configuration.

[0055] The propagation module is used to perform forward and backward propagation calculations on the large model of the embedded and updated adjustable adapter to obtain the large model with adjusted parameters.

[0056] In addition, to achieve the above objectives, this application also proposes a large model adjustment device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the large model adjustment method as described above.

[0057] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the large model adjustment method described above.

[0058] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the large model adjustment method described above.

[0059] This application provides a method for adjusting a large model. First, the large model to be adjusted is segmented and deployed to obtain sub-models deployed on different edge devices. Each sub-model includes a different number of encoding / decoding layers and adjustable adapters, with each encoding / decoding layer embedding a corresponding adjustable adapter. The adjustable adapters are then updated layer by layer according to the training state of the large model to be adjusted, resulting in a large model with the updated adjustable adapters embedded. Forward and backward propagation calculations are performed on the large model with the updated adjustable adapters to obtain the parameter-adjusted large model. This method effectively improves the efficiency of large model parameter tuning and saves device resources.

[0060] In summary, this application, by segmenting and deploying the large model to be adjusted, can accurately adapt to the corresponding end-side devices, obtaining sub-models deployed on different end-side devices. By updating the adjustable adapters in each sub-model layer by layer, the backpropagation path is effectively shortened, thereby enabling forward and backward propagation. This allows for fine-tuning of the large model through multiple devices, quickly and efficiently obtaining the adjusted large model. It overcomes the technical defects of not being able to utilize multiple devices for model fine-tuning and model fine-tuning failure due to limited device resources. It can effectively improve the efficiency of large model parameter tuning and save device resources. Attached Figure Description

[0061] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 is a flowchart of the large model adjustment method provided in Embodiment 1 of this application;

[0064] Figure 2 is a schematic diagram of pipelined parallel training provided in Embodiment 1 of the large model adjustment method of this application;

[0065] Figure 3 is a flowchart of the adapter online configuration decision-making process provided in Embodiment 1 of the large model adjustment method of this application;

[0066] Figure 4 is a schematic diagram of the Transformer large model structure of the fused serial adapter provided in Embodiment 1 of the large model adjustment method of this application;

[0067] Figure 5 is a schematic diagram of multi-device collaborative fine-tuning training provided in Embodiment 1 of the large model adjustment method of this application;

[0068] Figure 6 is a flowchart of the large model adjustment method provided in Embodiment 2 of this application;

[0069] Figure 7 is a flowchart of the model segmentation and deployment decision-making process provided in Embodiment 2 of the large model adjustment method of this application;

[0070] Figure 8 is a schematic diagram of the module structure of the large model adjustment device according to an embodiment of this application;

[0071] Figure 9 is a schematic diagram of the hardware operating environment involved in the large model adjustment method of this application.

[0072] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0073] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0074] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0075] The main solution of this application embodiment is as follows: the large model to be adjusted is segmented and deployed to obtain sub-models deployed on different end devices, wherein each sub-model includes a different number of encoding / decoding layers and adjustable adapters, and each encoding / decoding layer is embedded with a corresponding adjustable adapter; the adjustable adapters are updated layer by layer according to the training state of the large model to be adjusted to obtain a large model with the adjusted adapters embedded with the updated configuration; forward propagation calculation and backward propagation calculation are performed on the large model with the adjusted adapters embedded with the updated configuration to obtain the large model with adjusted parameters.

[0076] Existing pipelined parallelism-based collaborative fine-tuning techniques suffer from two main problems: 1) Current pipelined parallelism-based collaborative training designs generally follow a chain-like sequential training mechanism. This mechanism requires the first device in the chain to hold all input data and share relevant labels with the last device, raising privacy concerns and increasing interaction overhead. Furthermore, only data samples from the pipeline "head" device are allowed for training, while other devices act as auxiliary computing nodes. Therefore, this pipelined parallel training framework cannot utilize data from multiple devices to fine-tune a unified model. In practical applications, due to storage limitations, the data samples on a single device may be quite limited, while multiple devices within a local area network can often collect different types of task data. Inefficient use of this data will impair the performance of the fine-tuned model. 2) Pipeline parallel training allows forward propagation computation using outdated model parameters. To unify the model update direction, each device is required to store different versions of model parameters for averaging updates. This introduces significant storage overhead, posing a greater resource challenge to edge devices with limited memory and storage capacity, easily leading to fine-tuning task failures. Therefore, how to effectively improve the efficiency of large model parameter tuning and save equipment resources is an urgent problem to be solved.

[0077] This application divides and deploys the large model to be adjusted, which can accurately adapt to the corresponding end devices and obtain sub-models deployed on different end devices. The adjustable adapters in each sub-model are updated layer by layer, which effectively shortens the backpropagation path and enables forward and backward propagation. This allows for fine-tuning of the large model through multiple devices, and quickly and efficiently obtains the adjusted large model. It overcomes the technical defects of not being able to use multiple devices for model fine-tuning and model fine-tuning failure caused by limited device resources. It can effectively improve the efficiency of large model parameter tuning and save device resources.

[0078] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or large model adjustment device capable of performing the above functions. The following description uses a large model adjustment device as an example to illustrate this embodiment and the subsequent embodiments.

[0079] Based on this, this application provides a large model adjustment method. Referring to Figure 1, which is a flowchart of the first embodiment of the large model adjustment method of this application, the large model adjustment method includes steps S10 to S30:

[0080] Step S10: The large model to be adjusted is divided and deployed to obtain sub-models deployed on different end devices. Each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is backed by a corresponding adjustable adapter.

[0081] It should be noted that a large model is a machine learning model containing a large number of parameters, such as a deep learning network. It is typically used to handle complex tasks, such as natural language processing and image recognition. In this embodiment, the large model to be adjusted is a pre-trained model that already possesses the basic ability to handle a specific task. However, to adapt to a specific application scenario, further adjustments or fine-tuning are required. For example, in distributed computing networks such as smart homes, several network-connected devices with limited computing power and memory can collaboratively perform complex Transformer large model fine-tuning training using their respective local label data.

[0082] It is understood that the large model to be adjusted includes multiple codec layers. A codec layer, or transformer layer, is a codec structure used to handle the conversion between input and output data. In the implementation, the codec layers of the large model to be adjusted are segmented and deployed across different edge devices. Each device deploys a sub-model containing a different number of codec layers to accommodate the device's computing power and storage resources.

[0083] It is worth noting that an adjustable adapter is a mechanism for adjusting model parameters. It can fine-tune the model according to the specific needs and environmental conditions of the end device. In this embodiment, a corresponding adjustable adapter is embedded after each encoding and decoding layer, so that each sub-model can independently adjust its parameters to adapt to different application scenarios and data distributions.

[0084] It's worth noting that deploying the large model to be adjusted across a group of edge devices allows for adaptation to the varying computational speeds, memory capacities, and link rates of these devices. This addresses the limitations of a single device in terms of computational power and memory resources for large model fine-tuning tasks. By embedding an adjustable adapter layer after each Transformer layer on each edge device, only the parameters of the active adapter layer are trained during fine-tuning, while the original Transformer layers remain inactive and do not undergo parameter update calculations. This improves the computational efficiency of the edge devices and reduces the memory overhead introduced by storing intermediate parameters.

[0085] In its implementation, the collaborative fine-tuning system adopts a hierarchical model segmentation strategy. Using the Transformer module as the basic segmentation granularity, it allocates and deploys the modules on a set of end devices according to rules. At the same time, it maintains parameter copies of the input embedding layer (Emb) and output header layer (Hed) on each device, thereby constructing a ring topology with any data source device as the start and end node. The system also switches the data source node in a polling manner in different training rounds to achieve effective utilization of the local data of all participating devices.

[0086] Figure 2 illustrates a pipelined parallel training process, including edge devices u1, u2, u3, and u4. During startup, forward propagation of data is performed. Fi represents the forward computation of data batch i, and Bi represents the backward computation of data batch i. Before the backward propagation (B1) of the first data batch is completed, forward propagation of data batches 1, 2, 3, and 4 has already begun, all based on the same version of model weights as the first data batch. This may lead to different model update directions for the first four data batches, potentially prolonging the convergence time. Furthermore, to unify the model update directions across the four data batches, each device is required to store four versions of model parameters for averaging updates. This introduces significant storage overhead, further exacerbating the resource challenges faced by edge devices with limited memory and storage capacity, and potentially causing fine-tuning tasks to fail.

[0087] Step S20: Update the adjustable adapter layer by layer according to the training state of the large model to be adjusted, and obtain the large model with the adjusted adapter embedded in the updated configuration.

[0088] It should be noted that in this embodiment, the adjustable adapters during the task execution phase are dynamically configured, the model training status and device resource changes are perceived online, the lower-level adapters are activated layer by layer as needed, and the adjustable parameter amount of the high-importance adapters in the activated state is adaptively increased, so as to save device resource consumption in the early stage of training without sacrificing convergence speed and fine-tuning performance.

[0089] In one feasible implementation, step S20 may include: obtaining the connection order of the encoding and decoding layers in each sub-model in the large model to be adjusted; activating the corresponding adjustable adapters layer by layer according to the training state of the large model to be adjusted and the connection order to obtain the activated adjustable adapters; adjusting the width configuration of the activated adjustable adapters to obtain the adjusted adapters with updated configurations; and embedding the adjusted adapters with updated configurations into the large model to be adjusted to obtain a large model with the adjusted adapters with updated configurations.

[0090] It should be noted that the hierarchy of the adjustable adapters embedded after each encoding / decoding layer can be determined based on the connection order of the encoding / decoding layers in the large model to be adjusted. To address the progressive learning characteristic of "from easy to difficult" during fine-tuning training, a strategy of dynamically expanding the number of adjustable parameters in the activated state is adopted. Specifically, using a certain number of training epochs as a cycle, by determining when the training process encounters a temporary plateau (i.e., further training under the current adapter configuration will no longer bring significant accuracy improvement), the parameters of the lower-level adapters are gradually unfrozen from top to bottom, and the bottleneck dimensions of some important activated adapters are increased. In this embodiment, a bottom-up, layer-by-layer parameter solidification strategy can also be used to shorten the backpropagation path; this embodiment does not specifically limit this approach, and this embodiment uses a top-down, layer-by-layer parameter unfreezing strategy as an example for explanation.

[0091] It is understandable that activating the adjustable adapters of each layer layer by layer, i.e., the deep expansion of the adapters, means that at the beginning of fine-tuning training, the parameters of the top layer (i.e. the layer closest to the model output) are first unfrozen. After every k training rounds, the parameters of the next layer of adapters are unfrozen to participate in the update calculation. Once any parameter position of any adapter layer is unfrozen, it will remain in an active state until the training converges.

[0092] In one feasible implementation, adjusting the width configuration of the tunable adapter in the active state to obtain an updated tunable adapter includes: obtaining gradient information of the current adjustable parameters of the tunable adapter in the active state; determining the trace of the Fisher information matrix based on the gradient information of the current adjustable parameters; sorting the tunable adapters in the active state based on the trace of the Fisher information matrix to obtain a target adapter; and updating the bottleneck dimension of the target adapter based on a preset bottleneck dimension growth step size to obtain an updated tunable adapter.

[0093] It should be noted that during fine-tuning training, the Fisher Information Matrix is ​​a key tool for measuring the importance of parameters. The Fisher Information Matrix is ​​obtained based on the gradient information of the model parameters and can provide a quantitative indicator of the degree of influence of the parameters on the model output. By calculating the trace of the Fisher Information Matrix (i.e. the sum of the diagonal elements of the matrix), the importance of the adjustable adapter can be determined.

[0094] Understandably, calculating the gradient information of all parameters in the currently active tunable adapter layer reflects the contribution of each parameter to the model performance improvement during the current training phase. Based on this gradient information, a Fisher information matrix can be constructed, and its trace can be calculated. The size of the trace can serve as a measure of adapter importance; adapters with larger traces indicate a more significant impact on model performance. The tunable adapters in the active states are sorted according to the trace of the Fisher information matrix to determine the target adapter, which is the most important adapter in the current training phase. The bottleneck dimension of the target adapter is updated based on a preset bottleneck dimension growth step. The bottleneck dimension refers to the limitation or adjustment of the number of parameters in the tunable adapter layer. By gradually increasing the bottleneck dimension, fine-tuning of important parameters can be achieved, thereby further improving model performance without sacrificing convergence speed and fine-tuning performance. Using the trace of the Fisher information matrix to guide parameter optimization of the tunable adapter layer effectively balances model training efficiency and performance, ensuring efficient and accurate model fine-tuning on edge devices.

[0095] It is worth noting that in this embodiment, the trace of the Fisher information matrix (denoted as tr(F)) can be used to evaluate the training state and guide the adapter configuration decision. Alternatively, the approximation of the Fisher information or other metrics can be used to evaluate the training state and the real-time importance of different model parameters or modules to training performance. This embodiment does not impose specific limitations on this. This embodiment uses the trace of the Fisher information matrix as an example for illustration.

[0096] It is worth noting that adapter width expansion refers to calculating the trace (i.e., tr(F) value) of the Fisher information matrix of an activated adapter layer based on the gradient information of its current adjustable parameters, sorting all activated adapters in descending order according to their tr(F) values, and considering the top α% of adapters as important adapters for the current training phase. Under the memory constraints of their respective end devices, the bottleneck dimension of the top α% of important adapters is linearly increased, and the newly added adjustable parameters are initialized according to a zero-mean Gaussian distribution, as shown in the following equation:

[0097] Where γ is the growth step size of the bottleneck dimension. It is the maximum allowable bottleneck dimension of the j-th adapter under the memory constraints of the edge device u.

[0098] It is worth noting that the configuration update interval refers to the interval calculated by comparing the average tr(F) of all adjustable parameters with the average duration of a single training epoch. The ratio (including device computation latency and point-to-point communication latency) is used to evaluate the performance of model convergence and system latency under the current adapter configuration strategy, as shown in the following formula:

[0099] Where TE is the training efficiency value, and k is the configuration update interval. Let r be the average duration of a single training epoch, and r be the number of training epochs.

[0100] When the training efficiency value of the current k rounds decreases compared to the previous 2k to k+1 rounds, i.e. TE(r-2k:rk)>TE(rk:r), the configuration update interval k is linearly increased to k+Δ, so that the system continues to use the current adapter depth and width configuration, avoiding aggressively increasing the number of adjustable parameters.

[0101] As shown in Figure 3, the online configuration decision flowchart of the adapter is as follows: When the training round r = 1, it is determined whether r%k = 0 is satisfied. If not, the r-th round of training is executed and r = r + 1 is set. If satisfied, the training efficiency TE(rk:r) is evaluated, the adapter layers are sorted by importance, the width of the top α% important adapter layers is increased, the adapter activation depth is increased, the newly added model parameters are initialized, and it is determined whether TE(r-2k:rk) > TE(rk:r) is satisfied. If satisfied, the configuration update interval k is set to k + Δ. If not satisfied, the configuration update interval k is maintained.

[0102] Figure 4 shows a schematic diagram of the Transformer large model structure with integrated serial adapters. The large model consists of an input embedding layer (Emb), an output header layer (Hed), and multiple Transformer layers, with forward and backward propagation between Transformer layers. Each Transformer layer consists of three residual connections and layer normalization (Add & Norm), an adapter, a feed-forward layer, and a multi-head attention mechanism. Each adapter module mainly consists of two feed-forward sub-layers. The first feed-forward sub-layer (down-project) takes the output of the Transformer block as input, projects the original input dimension d (high-dimensional features) to m (low-dimensional features), and controls the size of m to limit the number of parameters in the adapter module. Then, a nonlinearity layer is passed through the middle. In the output stage, the second feed-forward sub-layer (up-project) restores the input dimension, remapping m (low-dimensional features) back to d (the original high-dimensional features) as the output of the adapter module.

[0103] In the specific implementation, a top-down adapter layer-by-layer unfreezing strategy is adopted during fine-tuning training. Only the parameters of the active adapters participate in the backpropagation update calculation. The central controller makes online decisions on the adapter configuration strategy based on real-time observation of the computing power, memory, and link status of each end-device during training, as well as the evaluation of the model convergence status. This includes configuring the top-down unfreezing depth of the adapters and the structural width of each active adapter. Based on this configuration, for any training iteration, the forward propagation calculation starts from the end-device holding the data sample being processed in the current iteration. The feature representation of the data is generated through the Emb layer of that device, and then sequentially passes through each collaborating end-device according to the Transformer layer deployment order. Finally, it is transmitted to the original input end-device to generate the model output result and calculate the loss value based on the corresponding data labels. This propagation process forms a ring topology, and neighboring end-devices can transmit the inter-layer parameters required for calculation through point-to-point communication methods such as direct inter-device communication. Similarly, the backpropagation computation of this training iteration is performed in reverse along the end-side devices in this ring topology, with inter-layer gradient values ​​being passed between adjacent devices in the topology. The backpropagation process stops at the deepest adapter layer in the current thaw depth configuration.

[0104] Step S30: Perform forward and backward propagation calculations on the large model of the embedded and updated adjustable adapter to obtain the large model with adjusted parameters.

[0105] It's important to note that forward propagation and backpropagation are two crucial steps that alternate during training. Forward propagation processes the input data through various layers of the model, ultimately producing a prediction. This process involves the collaborative work of the input embedding layer, the Transformer layer, and the output head layer. In each Transformer layer, the data first undergoes residual connections and layer normalization, then dimensionality reduction and expansion of features are performed by the adapter module, and finally further processed by the feedforward layer and multi-head attention mechanism until the final feature representation is output. Backpropagation, based on forward propagation, uses the error calculated from the loss function to backpropagate the error signal and update the model parameters. This process also involves various sub-modules of the Transformer layer, including the adapter module, residual connections, and layer normalization. Through gradient descent, the model parameters are adjusted to reduce the difference between the prediction and the true value.

[0106] In one feasible implementation, step S20 may include: performing forward and backward propagation calculations on the sub-models of each end-side device based on task data samples to achieve pipelined parallel training and obtain a trained large model; evaluating the training convergence based on the loss value of the trained large model to obtain an evaluation result; when the evaluation result indicates training convergence, using the trained large model as the parameter-adjusted large model; when the evaluation result indicates training non-convergence, performing polling training on the trained large model until training convergence to obtain the parameter-adjusted large model.

[0107] It should be noted that in pipelined parallel training, each edge device is responsible for a different part of the model, allowing the entire training process to proceed in parallel. After receiving a data sample, each edge device first performs a forward propagation computation, processing the data through the part of the model it is responsible for. After processing, the data is passed to the next edge device, and so on, until all edge devices have sequentially completed the computation of their respective parts of the model. After completing its task, each edge device passes the result to the next device, forming a pipeline operation.

[0108] Understandably, in pipelined parallel training, since each edge device executes sequentially, it's crucial to ensure the efficiency and accuracy of data transmission between devices. To achieve this, high-speed network connections are typically used between edge devices to reduce data transmission time. Furthermore, to guarantee training stability, each edge device needs to remain synchronized, ensuring that while one device is processing data, other devices have already completed their corresponding computational tasks.

[0109] It's worth noting that during the implementation of pipelined parallel training, the central controller is responsible for coordinating the work of each edge device. It allocates tasks rationally based on the computing power, memory, and link status of each device, ensuring that each device can work efficiently during training. Furthermore, the central controller dynamically adjusts the pipelined parallel training strategy based on the model's convergence status evaluation, such as adjusting task allocation and update intervals, to improve training efficiency and model performance. Pipeline parallel training can significantly improve the training speed of large-scale models and shorten the time from training to deployment. Simultaneously, because each edge device can process data in parallel, it can fully utilize distributed computing resources, improving computational efficiency.

[0110] It's worth noting that during training, the loss value reflects the degree of difference between the model's predictions and the true labels. When the loss value reaches a low level, it indicates that the model has fitted the training data well, and the training process can be considered converged. To accurately assess training convergence, the central controller periodically calculates the loss value of the entire model and compares it with a preset threshold. If the loss value is below the threshold, the model is considered converged, and training can be stopped, using the current model as the final parameter-adjusted large model. Conversely, if the loss value remains high, it indicates that the model has not yet fully learned the data features and training needs to continue. In this case, the central controller decides whether to perform polling training based on the current training status and resource availability. This involves cyclically training across various edge devices until the model converges. Polling training helps ensure that each edge device can fully participate in the training process while avoiding a decrease in overall training efficiency due to the computational bottleneck of a single device. In this way, the model can achieve optimal training results with limited computing resources.

[0111] It's worth noting that during fine-tuning training, the top-down adapter layer-by-layer unfreezing strategy ensures the model's training efficiency and performance. To further optimize this process, a gradient accumulation-based training method can be introduced. This method allows gradients to accumulate in each training batch instead of updating model parameters immediately. In this way, the model can observe gradient trends over a longer period, thus adjusting parameters more accurately. Specifically, when a training batch begins, the gradient is initialized to zero. As each training sample is forward and backward propagated, the gradient is accumulated to the initial value. Only after all samples in a batch have been processed are the model parameters updated based on the accumulated gradients. This method is particularly suitable for processing small batches of data, effectively reducing the variance of gradient estimation and improving model stability and convergence speed.

[0112] It is understood that in this embodiment, the adapter fine-tuning technology is integrated, and an adapter layer is embedded after each Transformer layer. In the early stage of fine-tuning, the underlying adapter is kept in an inactive state. Since the gradient descent update is only performed on the parameters of the adapter in the unfrozen state during the fine-tuning process, the characteristic of stopping the backpropagation calculation in advance is utilized to input multiple batches of data in advance to perform forward propagation calculation, thereby realizing pipeline parallel computing between multiple devices. Other efficient parameter fine-tuning technologies, such as LoRA and cue fine-tuning, can also be used. This embodiment does not make specific restrictions on this. This embodiment uses the adapter fine-tuning technology as an example for explanation.

[0113] Understandably, by adopting the on-demand, layer-by-layer parameter activation strategy, the backpropagation computation path is shortened. By taking advantage of the fixed parameters of the underlying model, the forward propagation computation of a new round of training iterations can be started in advance. Without introducing model obsolescence or increasing the storage overhead of parameter copies, parallel computing between devices is promoted, which can significantly accelerate the fine-tuning process.

[0114] In one feasible implementation, the step of performing forward and backward propagation calculations on the sub-models of each end-side device based on task data samples to achieve pipelined parallel training and obtain a trained large model includes: determining the forward propagation path and the backward propagation path based on the end-side device; performing forward propagation training on the sub-models of each end-side device along the forward propagation path based on task data samples, and performing backward propagation training on the sub-models of each end-side device along the backward propagation path to achieve pipelined parallel training and obtain a trained large model.

[0115] It's important to note that the forward and backward propagation paths define the order in which data flows between the various endpoint devices. The forward propagation path starts with the input data, sequentially passing through the sub-models of each endpoint device until the last device completes its forward propagation computation. The backward propagation path, on the other hand, starts from the last endpoint device, proceeding backward through each sub-model until the first device completes its backward propagation computation. This path configuration ensures efficient data transfer between the endpoint devices while maintaining the synchronization of the training process.

[0116] Understandably, for any training iteration, the sampling device for the sample data is called the iteration "initiating node," and the batch data of the current iteration is input into the input embedding layer model of this device; the end device with the lowest unfrozen adapter layer is called the iteration "terminating node," and the backpropagation computation of the current iteration stops at the activation adapter layer carried by this device.

[0117] In the specific implementation, during the initialization phase, each device uploads its status information to the central controller, including the communication rates of all connectable point-to-point links, the device's own computing speed, and memory limitations. The central controller determines the model segmentation and allocation strategy based on the collected system status information and distributes the pre-trained layer parameters to the corresponding edge devices. The central controller selects and notifies the first "initiator node" in each training round. Simultaneously, the central controller sends the necessary training settings (including learning rate, local iteration count, and batch size) to the edge devices to initialize training. In each training iteration, the selected "initiator node" samples a small batch of data from its local dataset, inputs it into its local input embedding layer, and sends the intermediate output parameters to the next corresponding edge device in the order of Transformer blocks. The intermediate results generated by each edge device's sub-model are passed along the circular topology, and the output of the last Transformer block is sent back to the "initiator node" device to calculate the training loss, thus completing the forward propagation of that data batch. Then, the "initiating node" starts backpropagation computation based on the locally computed gradient. The "initiating node" updates its output header layer parameters and propagates the gradient between end devices along the reverse loop topology. The end devices carrying the activation adapter layer in the backpropagation path update their adapter layer parameters accordingly. The backpropagation computation stops at the "termination node".

[0118] It is worth noting that, in this embodiment, since the underlying model does not contain activation parameters that need to be updated via backpropagation, the training process allows the "initiating node" device to continuously sample small batches of data and input them into the distributed large model before the forward and backward propagation calculations of the current iteration are completed, so as to start the forward propagation of a new training iteration. That is, the distributed training system can execute forward / backward propagation calculations for multiple iterations at the same time, and different devices may simultaneously undertake the calculation tasks of different training iterations, realizing a pipelined parallel training process with batch data granularity between edge devices.

[0119] It's worth noting that after completing a predetermined number of iterations, the "initiating node" device reports its local loss value to the central controller to evaluate training convergence. Simultaneously, it selects the next "initiating node" with the best channel quality and passes the latest parameters of its local output header layer to update its header layer. Once all end-side devices in the system have served as "initiating nodes," the current training round ends, and it determines whether it's an adapter configuration update round. If not, a new training round begins; if so, the central controller obtains training state information such as the Fisher matrix of each end-side device's sub-model, as well as real-time system state information such as communication, computing power, and memory, to determine the adapter update configuration. This configuration decision is then distributed to each end-side device, which initializes the newly added activation parameters based on the latest adapter configuration. This process is repeated until convergence, signifying that model fine-tuning is complete.

[0120] Figure 5 illustrates a multi-device collaborative fine-tuning training process. Four devices, u1, u2, u3, and u4, collaboratively support the fine-tuning of a large model with 12 Transformer layers. The Transformer layers are loaded onto devices u1, u2, u3, and u4 in a ratio of 4:5:2:3. It is assumed that the adapter unfreezing depth output by the online adapter configuration scheme is 3. Starting with local sample data from u1, the first training iteration propagates forward along the path u1→u2→u3→u4→u1 and backward along the path u1→u4. This iterative process is repeated. After h training iterations on u1's data, u1 transmits its latest Hed layer parameters to u2 to update u2's Hed layer. Subsequently, u2 samples a new batch of data from its local dataset and begins training on u2's data. One training cycle is considered complete when all devices have completed the above training process. This polling-based collaborative training process is repeated until the model converges.

[0121] In the above-mentioned multi-device collaborative fine-tuning training process, two parallel processing operations are further incorporated: 1) In different iterations of training on data from one device, the backpropagation update calculation process of the model and the forward propagation calculation process before the adapter unfreezing layer are independent of each other and can be processed in parallel; 2) In iterations of training on data from two different devices, the backpropagation update calculation process of the Transformer layer and the parameter transmission process of the Hed layer are independent of each other and can be executed in parallel. The above two types of computation-computation and computation-transmission parallel processing help to improve the utilization of device resources and accelerate the training process.

[0122] This embodiment provides a method for adjusting a large model. First, the large model to be adjusted is segmented and deployed to obtain sub-models deployed on different edge devices. Each sub-model includes a different number of encoding / decoding layers and adjustable adapters, with each encoding / decoding layer embedding a corresponding adjustable adapter. The adjustable adapters are then updated layer by layer according to the training state of the large model to be adjusted, resulting in a large model with the updated adjustable adapters embedded. Forward and backward propagation calculations are performed on the large model with the updated adjustable adapters to obtain the parameter-adjusted large model. This method effectively improves the efficiency of large model parameter tuning and saves device resources.

[0123] In summary, this embodiment, by segmenting and deploying the large model to be adjusted, can accurately adapt to the corresponding end-side devices, obtaining sub-models deployed on different end-side devices. By updating the adjustable adapters in each sub-model layer by layer, the backpropagation path is effectively shortened, thereby enabling forward and backward propagation. This allows for fine-tuning of the large model through multiple devices, quickly and efficiently obtaining the adjusted large model. It overcomes the technical defects of not being able to utilize multiple devices for model fine-tuning and model fine-tuning failures caused by limited device resources. It can effectively improve the efficiency of large model parameter tuning and save device resources.

[0124] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. Based on this, please refer to Figure 6, step S10 further includes steps S101-S104:

[0125] Step S101: Determine the number of codec layers allocated to each end-side device based on the memory capacity of each end-side device.

[0126] It should be noted that the memory capacity of the edge device is one of the important factors affecting its processing power. In this embodiment, by taking into account the memory capacity of the edge device, the encoding and decoding layers can be allocated more reasonably, ensuring that each device can efficiently execute its assigned tasks and avoiding performance bottlenecks or training failures caused by insufficient memory.

[0127] Understandably, in the model segmentation and allocation process, with the goal of minimizing training latency, the differences in computing speed, memory capacity and link rate between collaborative end devices are considered. Based on graph representation and optimization technology, the device ring topology construction decision is made, and the allocation and deployment scheme of the Transformer module on each device is determined, so as to shorten the idle waiting time of heterogeneous devices in the fine-tuning training process and enhance the convergence acceleration advantage brought by pipeline parallel training.

[0128] In the specific implementation, before starting the fine-tuning task, the central controller first determines the large model segmentation and deployment strategy based on information such as the available computing power, memory capacity, and point-to-point communication rate of the end devices within the system. This includes determining after which Transformer layers the model will be segmented to form multiple sets of sub-models containing consecutive Transformer layers, and which device each sub-model will be assigned to for processing. The central controller broadcasts the segmentation and deployment strategy and distributes the parameters of each sub-model to the corresponding end devices according to the strategy to perform the corresponding forward and backward propagation calculations. Each Transformer layer is followed by an adapter module with a specific initialization structure and parameters. The central controller sends this adapter module, along with the associated Transformer layer parameters, to the corresponding end devices. In addition to the assigned sub-models, each end device also locally deploys copies of Emb and Hed to perform feature representation of its local sample data, generate model output results, calculate loss values, and update the parameters of the output header layer.

[0129] In one feasible implementation, step S101 may include: acquiring multiple end-side devices and their corresponding forward propagation computation delays, and sorting each end-side device based on the forward propagation computation delays to obtain sorted end-side devices; determining a threshold for the number of codec layers for each end-side device based on the memory capacity of each end-side device; and determining the number of codec layers allocated to each end-side device based on the threshold for the number of codec layers and the sorted end-side devices.

[0130] It should be noted that for any collaborative fine-tuning task, a model splitting and deployment is performed before system construction. The end-devices and their direct transmission links are modeled as a directed graph G = (U, A), where U is the set of device vertices, A is the set of directed edges, and the element a = (u, u′) represents the directed transmission link from device u to device u′. The link transmission delay is defined as... The forward propagation computation delay for executing a Transformer module on the device is τ. tr (u,l), the forward propagation computation latency of the edge device reflects its computational capability. In this embodiment, by considering the forward propagation computation latency of the edge device, the encoding and decoding layers can be allocated more reasonably, ensuring that each device can efficiently execute its assigned tasks and avoiding performance bottlenecks or training failures caused by excessive computation latency.

[0131] Understandably, based on the available memory capacity of each device and the size of each Transformer block, the number of Transformer blocks allocated to each device is optimized with the goal of minimizing the total computation time of all devices.

[0132] In the specific implementation, the edge devices are sorted in descending order based on their computing power (i.e., forward propagation computation latency). For each sorted edge device, the maximum number of allocable Transformer blocks, i.e., the threshold for the number of encoding / decoding layers, is calculated based on its memory capacity and the memory requirement of a single Transformer block. The number of unallocated Transformer blocks is updated, and the number of Transformer blocks allocated to the next device is determined according to the sorting, until all devices are traversed. If there are still unallocated Transformer blocks after the traversal, it indicates that the edge devices participating in the collaborative fine-tuning cannot support the target large model, and the model segmentation and deployment algorithm terminates.

[0133] It is worth noting that the Transformer block allocation optimization process allows some terminals with limited computing power or memory to not be allocated Transformer blocks, but only to carry the input embedding layer (Emb) and output header layer (Hed), responsible for processing input data and calculating loss, and not participating in the forward and backward propagation calculations of Transformer blocks.

[0134] Step S102: Divide the large model to be adjusted into multiple sub-models according to the number of encoding / decoding layers allocated.

[0135] It should be noted that the size and complexity of each sub-model can be dynamically adjusted based on the number of Transformer blocks corresponding to each edge device to adapt to the computing power of different devices. For example, more Transformer blocks can be allocated to edge devices with strong computing power to handle more complex tasks, while fewer Transformer blocks are allocated to devices with weaker computing power, which are mainly responsible for basic input and output processing.

[0136] Understandably, if the large model to be adjusted in this embodiment has 12 Transformer layers, and the number of Transformer blocks corresponding to the edge devices are 4, 5, 2, and 3 respectively, then these 12 Transformer layers can be divided into four sub-models, namely, sub-models containing 4 Transformer blocks, 5 Transformer blocks, 2 Transformer blocks, and 3 Transformer blocks respectively. In this way, each edge device can effectively participate in the processing of the large model according to its own computing power, while ensuring the efficiency and accuracy of the overall task. Furthermore, this dynamic allocation mechanism can effectively improve the flexibility and scalability of the system. In practical applications, as device performance improves or task requirements change, the system can flexibly adjust the number of Transformer blocks in each sub-model to adapt to new needs. For example, when the computing power of an edge device is enhanced, more Transformer blocks can be reallocated to that device, enabling it to handle more complex tasks, thereby fully utilizing the device's computing resources.

[0137] Step S103: Obtain the target Hamiltonian circuit.

[0138] It should be noted that a Hamiltonian cycle is a special type of graph theory path that passes through each vertex of the graph exactly once and eventually returns to the starting point. In this implementation, the purpose of obtaining the target Hamiltonian cycle is to determine the data flow order between sub-models, ensuring that the data dependencies between sub-models are handled correctly when partitioning the large model, thereby guaranteeing the overall efficiency and accuracy of the model.

[0139] In one feasible implementation, step S103 may include: obtaining the transmission links between each end-side device, and constructing a directed graph based on each end-side device and the transmission links, wherein the directed graph includes multiple original nodes; adding virtual nodes to the directed graph, and constructing directed edges based on the virtual nodes and each original node; obtaining the weight of each directed edge, and generating a directed weighted graph based on the weight of each directed edge; and determining a target Hamiltonian cycle based on the directed weighted graph.

[0140] It should be noted that the end-side devices and their direct transmission links are modeled as a directed graph G = (U, A), where U is the set of device nodes and A is the set of directed edges. The element a = (u, u′) represents the directed transmission link from device u to device u′. That is, the original nodes in the directed graph are the end-side devices, and the directed edges are the directed transmission links from device u to device u′.

[0141] Understandably, the process involves checking for devices that have not been allocated Transformer blocks; if such devices exist, the nodes corresponding to these devices and their associated edges are removed from the directed graph G; and a virtual node u is added to the directed graph G. end and virtual node u end Establish directed edges between all nodes in the directed graph G; calculate the weight of each directed edge to form a new graph, namely the directed weighted graph G′(U). - A - The weights of each directed edge (W) are as follows:

[0142] Where Weight(u,u') is the weight of the directed edge, U- is the set of nodes in the directed weighted graph, and u and u' are the nodes of the edge device. end It is a virtual node.

[0143] In one feasible implementation, determining the target Hamiltonian cycle based on the directed weighted graph includes: using virtual nodes in the directed weighted graph as starting points and performing path planning to generate a path table; determining the path length from each visited node in the visited node set to an unvisited node based on the path table; determining whether a reference Hamiltonian cycle exists based on the path length; and if the reference Hamiltonian cycle exists, removing the virtual node and reconstructing the path using a backtracking table to generate the target Hamiltonian cycle.

[0144] It should be noted that the shortest directed Hamiltonian cycle is found using dynamic programming on a directed weighted graph. This cycle traverses each vertex along the directed edges once and finally returns to the starting vertex, thus constructing a... The path table Dis(V,v) is defined, where V is the set of visited vertices, v is the current vertex, and each element in the table represents the path from the virtual vertex u. end Starting from a point V, find the shortest path length (i.e., the sum of edge weights) that passes through all vertices of set V and finally reaches vertex v.

[0145] Understandably, for each subset Given a vertex v∈V, compute the distance from v to an unvisited vertex. The path length is calculated, and the path table and backtracking table are updated to obtain Dis(V′,v′) and Back(V′,v′). After calculating the shortest path for all subsets, a path is found that returns to the virtual vertex u. end Find the shortest Hamiltonian cycle; if the optimal path is found, reconstruct the complete path using the backtracking table Back(V′,v′) and remove the virtual vertex u. en d forms the final path, i.e. the target Hamiltonian circuit.

[0146] It is worth noting that the backtracking table is an important data structure used to record the predecessor node of each node in the path planning process. Through the backtracking table, the endpoint can be traced back to the starting point, thereby reconstructing a complete Hamiltonian cycle. In the process of dynamic programming, the construction of the backtracking table is completed step by step. Whenever the shortest path from a virtual vertex to a visited vertex is determined, the predecessor node of that vertex in the backtracking table is updated.

[0147] Step S104: Based on the target Hamiltonian circuit, deploy multiple sub-models sequentially on the corresponding end-side devices to obtain sub-models deployed on different end-side devices.

[0148] It should be noted that the target Hamiltonian cycle is a path that covers all end-device nodes, visits each node only once, and ultimately returns to the starting point—the shortest path. Once the target Hamiltonian cycle is determined, the sub-models can be deployed to the corresponding end-devices in the order of the cycle.

[0149] In the implementation, the deployment order of the sub-models is first determined, that is, arranged according to the node order of the target Hamiltonian loop. Then, for each end-side device, a corresponding sub-model is allocated according to its position in the target Hamiltonian loop. During the allocation process, factors such as the device's computing power, storage space, and network bandwidth need to be considered to ensure that the sub-model can run normally on the device and meet performance requirements.

[0150] In one feasible implementation, step S104 may include: determining the arrangement order of end-side devices based on the target Hamiltonian circuit; obtaining model parameters for each sub-model; and sequentially sending the model parameters to the corresponding end-side devices according to the arrangement order of the end-side devices to obtain sub-models deployed on different end-side devices.

[0151] It should be noted that the order of the end-side devices determined by the target Hamiltonian loop ensures the sequentiality and efficiency of the sub-model deployment. The model parameters of each sub-model are obtained, including but not limited to the model's weights, biases, and activation functions. These parameters are the foundation for the sub-models' inference and learning.

[0152] Understandably, model parameters are sent sequentially to the corresponding end-side devices according to their arrangement. During transmission, compression techniques can be used to reduce data transmission volume while ensuring data integrity and accuracy.

[0153] As shown in Figure 7, the model segmentation deployment decision flowchart is as follows: a device connection topology graph is constructed, the maximum number of TRMs is allocated according to the descending order of device computing power, it is determined whether there are any unallocated TRMs, if so, the device connection graph is updated, if not, the edge weights of the connection graph are determined, the shortest Hamiltonian cycle is determined, and then the TRM deployment strategy is determined.

[0154] In this embodiment, by accurately determining the number of encoding / decoding layers allocated based on the memory capacity of each end device and then segmenting the model, and then deploying the sub-models sequentially on the corresponding end devices according to the target Hamiltonian circuit, the idle waiting time of heterogeneous devices during fine-tuning training can be effectively shortened, and the convergence speed of training can be improved.

[0155] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the large model adjustment method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0156] This application also provides a large model adjustment device, as shown in Figure 8, the large model adjustment device includes:

[0157] The deployment module 10 is used to divide and deploy the large model to be adjusted, resulting in sub-models deployed on different end devices. Each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is backed by a corresponding adjustable adapter.

[0158] The update module 20 is used to update the adjustable adapter layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model with the adjusted adapter embedded in the updated configuration.

[0159] The propagation module 30 is used to perform forward propagation calculations and backward propagation calculations on the large model of the embedded and updated adjustable adapter to obtain the large model with adjusted parameters.

[0160] This embodiment provides a large model adjustment device. The large model to be adjusted is segmented and deployed to obtain sub-models deployed on different edge devices. Each sub-model includes a different number of encoding / decoding layers and adjustable adapters, with each encoding / decoding layer embedded with a corresponding adjustable adapter. The adjustable adapters are updated layer by layer according to the training state of the large model to be adjusted, resulting in a large model with the updated adjustable adapters embedded. Forward and backward propagation calculations are performed on the large model with the updated adjustable adapters embedded to obtain the parameter-adjusted large model. This effectively improves the efficiency of large model parameter tuning and saves device resources.

[0161] In summary, this embodiment, by segmenting and deploying the large model to be adjusted, can accurately adapt to the corresponding end-side devices, obtaining sub-models deployed on different end-side devices. By updating the adjustable adapters in each sub-model layer by layer, the backpropagation path is effectively shortened, thereby enabling forward and backward propagation. This allows for fine-tuning of the large model through multiple devices, quickly and efficiently obtaining the adjusted large model. It overcomes the technical defects of not being able to utilize multiple devices for model fine-tuning and model fine-tuning failures caused by limited device resources. It can effectively improve the efficiency of large model parameter tuning and save device resources.

[0162] In one embodiment, the deployment module 10 is further configured to determine the number of codec layers allocated to each end-side device based on the memory capacity of each end-side device; divide the large model to be adjusted into multiple sub-models according to the number of codec layers allocated; obtain a target Hamiltonian circuit; and deploy the multiple sub-models sequentially on the corresponding end-side devices based on the target Hamiltonian circuit to obtain sub-models deployed on different end-side devices.

[0163] In one embodiment, the deployment module 10 is further configured to acquire multiple end-side devices and their corresponding forward propagation computation delays, and sort each end-side device based on the forward propagation computation delays to obtain sorted end-side devices; determine the number threshold of the number of codec layers for each end-side device based on the memory capacity of each end-side device; and determine the number of codec layers allocated to each end-side device based on the number threshold of the number of codec layers and the sorted end-side devices.

[0164] In one embodiment, the deployment module 10 is further configured to acquire the transmission links between each end-side device, and construct a directed graph based on each end-side device and the transmission links, wherein the directed graph includes multiple original nodes; add virtual nodes to the directed graph, and construct directed edges based on the virtual nodes and each original node; acquire the weight of each directed edge, and generate a directed weighted graph based on the weight of each directed edge; and determine a target Hamiltonian cycle based on the directed weighted graph.

[0165] In one embodiment, the deployment module 10 is further configured to take the virtual nodes in the directed weighted graph as the starting point and perform path planning to generate a path table; determine the path length from each visited node in the visited node set to an unvisited node based on the path table; determine whether a reference Hamiltonian cycle exists based on the path length; and if the reference Hamiltonian cycle exists, remove the virtual node and reconstruct the path through a backtracking table to generate a target Hamiltonian cycle.

[0166] In one embodiment, the deployment module 10 is further configured to determine the arrangement order of end-side devices based on the target Hamiltonian circuit; obtain model parameters of each sub-model; and send the model parameters sequentially to the corresponding end-side devices according to the arrangement order of the end-side devices to obtain sub-models deployed on different end-side devices.

[0167] In one embodiment, the update module 20 is further configured to: obtain the connection order of the encoding and decoding layers in each sub-model in the large model to be adjusted; activate the corresponding adjustable adapter layer by layer according to the training state of the large model to be adjusted and the connection order to obtain the adjustable adapter in the activated state; adjust the width configuration of the adjustable adapter in the activated state to obtain the adjustable adapter after the update configuration; and embed the adjustable adapter after the update configuration into the large model to be adjusted to obtain the large model with the adjustable adapter after the update configuration.

[0168] In one embodiment, the update module 20 is further configured to: obtain gradient information of the current adjustable parameters of the tunable adapter in the active state; determine the trace of the Fisher information matrix based on the gradient information of the current adjustable parameters; sort the tunable adapters in the active state based on the trace of the Fisher information matrix to obtain a target adapter; and update the bottleneck dimension of the target adapter based on a preset bottleneck dimension growth step size to obtain an updated tunable adapter.

[0169] In one embodiment, the propagation module 30 is further configured to perform forward and backward propagation calculations on the sub-models of each end-side device based on task data samples to achieve pipelined parallel training and obtain a trained large model; evaluate the training convergence based on the loss value of the trained large model to obtain an evaluation result; when the evaluation result indicates training convergence, use the trained large model as the parameter-adjusted large model; when the evaluation result indicates training non-convergence, perform polling training on the trained large model until training convergence to obtain the parameter-adjusted large model.

[0170] In one embodiment, the propagation module 30 is further configured to determine the forward propagation path and the backward propagation path based on the end-side device; perform forward propagation training on the sub-models of each end-side device along the forward propagation path based on the task data samples, and perform backward propagation training on the sub-models of each end-side device along the backward propagation path, so as to realize pipelined parallel training and obtain the trained large model.

[0171] The large model adjustment device provided in this application, employing the large model adjustment method described in the above embodiments, can solve the technical problem of how to effectively improve the efficiency of large model parameter tuning and save equipment resources. Compared with the prior art, the large model adjustment device provided in this application has the same effect as the large model adjustment method provided in the above embodiments, and other technical features in the large model adjustment device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0172] This application provides a large model adjustment device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the large model adjustment method in Embodiment 1 above.

[0173] Referring now to Figure 9, a schematic diagram of a structure suitable for implementing the large-scale model adjustment device of this application is shown. The large-scale model adjustment device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The large-scale model adjustment device shown in Figure 9 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0174] As shown in Figure 9, the large model adjustment device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the large model adjustment device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the large model adjustment device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a large model adjustment device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0175] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0176] The large model adjustment device provided in this application, employing the large model adjustment method described in the above embodiments, can solve the technical problem of how to effectively improve the efficiency of large model parameter tuning and save equipment resources. Compared with the prior art, the large model adjustment device provided in this application has the same effect as the large model adjustment method described in the above embodiments, and other technical features of this large model adjustment device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0177] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0178] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0179] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the large model adjustment method described in the above embodiments.

[0180] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0181] The aforementioned computer-readable storage medium may be included in the large model adjustment device; or it may exist independently and not be assembled into the large model adjustment device.

[0182] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a large model adjustment device, the large model adjustment device performs the following actions: It divides and deploys the large model to be adjusted into sub-models deployed on different end devices. Each sub-model includes a different number of encoding / decoding layers and adjustable adapters, with each encoding / decoding layer embedded with a corresponding adjustable adapter; it updates the adjustable adapters layer by layer according to the training state of the large model to be adjusted, resulting in a large model with the updated adjustable adapters embedded; and it performs forward and backward propagation calculations on the large model with the updated adjustable adapters embedded, resulting in a large model with adjusted parameters.

[0183] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0184] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0185] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0186] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described large model tuning method. This solves the technical problem of how to effectively improve the efficiency of large model parameter tuning and save equipment resources. Compared with the prior art, the computer-readable storage medium provided in this application has the same effect as the large model tuning method provided in the above embodiments, and will not be elaborated further here.

[0187] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the large model adjustment method described above.

[0188] The computer program product provided in this application can solve the technical problem of how to effectively improve the efficiency of large model parameter tuning and save equipment resources. Compared with the prior art, the effect of the computer program product provided in this application is the same as that of the large model adjustment method provided in the above embodiments, and will not be described in detail here.

[0189] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for adjusting a large model, wherein, The method includes: The large model to be adjusted is divided and deployed to obtain sub-models deployed on different end devices. Each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer is backed by a corresponding adjustable adapter. The adjustable adapter is updated layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model with the adjusted adapter embedded in the updated configuration. Forward and backward propagation calculations are performed on the large model of the embedded and updated adjustable adapter to obtain the large model with adjusted parameters.

2. The method as described in claim 1, wherein, The process of segmenting and deploying the large model to be adjusted results in multiple sub-models and corresponding edge devices, including: The number of codec layers allocated to each end-side device is determined based on the memory capacity of each end-side device. The large model to be adjusted is divided into multiple sub-models according to the number of encoding / decoding layers allocated; Obtain the target Hamiltonian cycle; Based on the target Hamiltonian circuit, multiple sub-models are sequentially deployed on corresponding end-side devices to obtain sub-models deployed on different end-side devices.

3. The method as described in claim 2, wherein, The step of determining the number of codec layers allocated to each end-side device based on the memory capacity of each end-side device includes: Multiple end-side devices and their corresponding forward propagation calculation delays are obtained, and the end-side devices are sorted based on the forward propagation calculation delays to obtain the sorted end-side devices. The threshold for the number of encoding / decoding layers for each end-side device is determined based on the memory capacity of each end-side device. The number of codec layers allocated to each end-side device is determined based on the threshold number of codec layers and the sorted end-side devices.

4. The method of claim 2, wherein, The acquisition of the target Hamiltonian circuit includes: Obtain the transmission links between each end-side device, and construct a directed graph based on each end-side device and the transmission links, wherein the directed graph includes multiple original nodes; Add virtual nodes to the directed graph, and construct directed edges based on the virtual nodes and each original node; Obtain the weight of each directed edge, and generate a directed weighted graph based on the weight of each directed edge; The target Hamiltonian cycle is determined based on the directed weighted graph.

5. The method of claim 4, wherein, The determination of the target Hamiltonian cycle based on the directed weighted graph includes: Using the virtual nodes in the directed weighted graph as starting points, path planning is performed to generate a path table; The path length from each visited node to an unvisited node in the visited node set is determined based on the path table. Determine whether a reference Hamiltonian cycle exists based on the path length; When the reference Hamiltonian cycle exists, the virtual node is removed and the path is reconstructed using a backtracking table to generate the target Hamiltonian cycle.

6. The method of claim 2, wherein, The step of sequentially deploying multiple sub-models on corresponding end-side devices based on the target Hamiltonian circuit to obtain sub-models deployed on different end-side devices includes: The arrangement order of the end-side devices is determined based on the target Hamiltonian circuit; Obtain the model parameters for each sub-model; The model parameters are sent sequentially to the corresponding end-side devices according to the arrangement order of the end-side devices, so as to obtain sub-models deployed on different end-side devices.

7. The method of claim 1, wherein, The step of updating the adjustable adapter layer by layer according to the training state of the large model to be adjusted, to obtain a large model with the adjusted adapter embedded in the updated configuration, includes: Obtain the connection order of the encoding and decoding layers in each sub-model within the larger model to be adjusted; Based on the training state of the large model to be adjusted and the connection order, the corresponding adjustable adapters are activated layer by layer to obtain the activated adjustable adapters. Adjust the width configuration of the activated adjustable adapter to obtain the adjusted adapter with the updated configuration; The updated adjustable adapter is embedded into the large model to be adjusted to obtain a large model with the updated adjustable adapter.

8. The method of claim 7, wherein, The step of adjusting the width configuration of the activated adjustable adapter to obtain the updated adjustable adapter includes: Obtain the gradient information of the current adjustable parameters of the activated adjustable adapter; The trace of the Fisher information matrix is ​​determined based on the gradient information of the currently adjustable parameters; The tunable adapters in the active states are sorted based on the trace of the Fisher information matrix to obtain the target adapter; The target adapter is updated in the bottleneck dimension based on a preset bottleneck dimension growth step size to obtain an adjustable adapter with updated configuration.

9. The method of claim 1, wherein, The process of performing forward and backward propagation calculations on the large model of the embedded and updated adjustable adapter to obtain the parameter-adjusted large model includes: Based on the task data samples, forward and backward propagation calculations are performed on the sub-models of each end device to achieve pipelined parallel training and obtain the trained large model. The training convergence is evaluated based on the loss value of the trained large model, and the evaluation result is obtained. When the evaluation result indicates that the training has converged, the trained large model is used as the parameter-adjusted large model. When the evaluation result indicates that the training has not converged, the trained large model is subjected to polling training until the training converges, resulting in a parameter-adjusted large model.

10. The method of claim 9, wherein, The forward and backward propagation calculations are performed on the sub-models of each end device based on task data samples to achieve pipelined parallel training and obtain a large trained model, including: The forward propagation path and the backward propagation path are determined based on the end-side device; Based on the task data samples, the sub-models of each end-side device are trained by forward propagation along the forward propagation path, and the sub-models of each end-side device are trained by backward propagation along the backward propagation path, so as to realize pipelined parallel training and obtain the trained large model.

11. A large model adjustment device, wherein, The large model adjustment device includes: The deployment module is used to divide and deploy the large model to be adjusted into sub-models deployed on different end devices. Each sub-model includes a different number of codec layers and adjustable adapters, and each codec layer has a corresponding adjustable adapter embedded in it. The update module is used to update the adjustable adapter layer by layer according to the training state of the large model to be adjusted, so as to obtain a large model with the adjusted adapter embedded with the updated configuration. The propagation module is used to perform forward and backward propagation calculations on the large model of the embedded and updated adjustable adapter to obtain the large model with adjusted parameters.

12. A large model adjustment device, wherein, The large model adjustment device includes: a memory, a processor, and a large model adjustment program stored in the memory and executable on the processor, the large model adjustment program being configured to implement the large model adjustment method as described in any one of claims 1 to 10.

13. A storage medium, wherein, The storage medium stores a large model adjustment program, which, when executed by a processor, implements the large model adjustment method as described in any one of claims 1 to 10.

Images