Time and memory efficient edge collaborative fine-tuning method and system for large language models

Abstract

The application discloses a time and memory efficient large language model edge collaborative fine-tuning method and system, the method comprising: constructing a bypass subnetwork for a large language model to obtain target LLMs; testing and fine-tuning the target LLMs by using each edge device, and recording relevant information of the edge device and the model; based on the relevant information, dividing the target LLMs and the edge device to obtain a division arrangement scheme; based on the division arrangement scheme, performing edge device collaborative target LLM fine-tuning to obtain a fine-tuning result. The system comprises a target model construction module, a test fine-tuning module, an arrangement module and a collaborative fine-tuning module. By using the application, under the premise of protecting user data privacy and security, resource-efficient and expandable model fine-tuning is realized, so that the corresponding large language model can adapt to downstream intelligent applications. The application can be widely applied to the Internet of Things field.

Description

Technical Field

[0001] This invention relates to the field of the Internet of Things, and in particular to a time- and memory-efficient method, system, and apparatus for edge collaborative fine-tuning of large language models. Background Technology

[0002] Large Language Models (LLMs) have revolutionized the field of machine intelligence due to their superior performance across a wide range of machine learning tasks. While these models were initially developed in cloud data centers, they have rapidly expanded to mobile edge devices, driving the development of edge intelligence applications, including intelligent personal assistants and private psychological counseling. To continuously improve model performance, LLMs require ongoing tracking of user interactions, data collection, and fine-tuning to enhance service quality. This fine-tuning allows pre-trained large language models to adapt to various specific downstream edge intelligence applications, such as intelligent personal assistants and private psychological counseling.

[0003] Fine-tuning of personal large language models (LLMs) typically uses highly sensitive user privacy data, which may include users' daily conversations, movement trajectories, and medical records. Using cloud computing resources provided by cloud service providers for LLM fine-tuning requires uploading all target fine-tuning data to these commercial companies' cloud servers, posing a serious risk of user privacy data leakage and raising concerns about data privacy among individual users. Edge fine-tuning technology can fully utilize idle computing resources in edge environments while completely protecting user data privacy, making it a widely used computing paradigm in privacy-sensitive edge applications. However, the resource-intensive nature of LLM fine-tuning presents significant challenges for fine-tuning on resource-constrained edge devices. Summary of the Invention

[0004] In view of this, in order to solve the problem that existing large oracle model fine-tuning methods are limited by the inherent resource shortage of a single edge device, and thus cannot efficiently fine-tune individual LLMs at the edge, this invention proposes a time- and memory-efficient large language model edge collaborative fine-tuning method, which includes the following steps:

[0005] Constructing bypass subnetworks for large language models yields target LLMs;

[0006] The target LLMs were tested and fine-tuned using various edge devices, and relevant information about the edge devices and the model was recorded.

[0007] Based on the aforementioned information, the target LLMs and the edge devices are divided to obtain a partitioning and orchestration scheme;

[0008] Based on the aforementioned partitioning and orchestration scheme, the target LLMs for edge device collaboration are fine-tuned to obtain the fine-tuning results.

[0009] In some embodiments, it also includes:

[0010] The bypass subnetwork is fine-tuned using cached data.

[0011] This preferred step caches the intermediate activation results of all input samples and saves them to the device's storage. When fine-tuning the same sample again, these results can be read directly from the hard drive. This avoids repeatedly executing the forward propagation process of LLMs when fine-tuning the same sample.

[0012] In some embodiments, the step of testing and fine-tuning the target LLMs using each edge device and recording relevant information about the edge devices and the model specifically includes:

[0013] Deploy the target LLMs on edge devices;

[0014] Fine-tuning of the target LLMs was performed offline based on test data;

[0015] Record the training speed of the edge device under different loads, the network communication speed between devices and the memory budget, as well as the intermediate activation results, model weights and gradient tensor dimensions of the target LLMs.

[0016] In some embodiments, the step of partitioning the target LLMs and the edge devices based on the relevant information to obtain a partitioning and orchestration scheme specifically includes:

[0017] Define the constraint objectives, which are to minimize the difference in execution time between each pipeline stage and to minimize the communication time.

[0018] Based on the constraint objectives and dynamic programming algorithm, a partitioning and arrangement scheme is obtained;

[0019] The partitioning and arrangement scheme includes multiple sub-models and device groups, and the sub-models and device groups have corresponding relationships.

[0020] In some embodiments, the state transition equation of the dynamic programming algorithm is:

[0021]

[0022] in, Indicates that it is a group of equipment The execution time of the slowest pipeline stage in the optimal subpipeline with s pipeline stages obtained from the submodel orchestration of LLM layers 0 to y; n and m are index subscripts, where:

[0023] Indicates that it is a group of equipment The execution time of the slowest pipeline stage in the optimal subpipeline with s-1 pipeline stages obtained by orchestrating the submodels from 0 to q in the LLM layer;

[0024] T(q+1→y,{d n-m …,d n-1}) indicates that the equipment group {d n-m ,…,d n-1 The time required for a single pipeline stage, consisting of sub-models from q+1 to y in the LLM layer, to perform forward and backward propagation;

[0025] Where T(x→y,{d0…,d) n-1 The calculation method for}) is as follows:

[0026]

[0027] Where M is the number of micro-batches into which a batch is divided, B is the number of samples in a micro-batch, and m d u represents the peak memory overhead of device d during fine-tuning. d This represents the memory budget of device d.

[0028] In some embodiments, the formula for the shortest communication time is expressed as follows:

[0029]

[0030] in, This indicates the start phase of training within a batch of samples. This represents the training execution phase within a batch of samples. This represents the end of the training phase in a batch of samples, where 's' represents the number of phases. W s The execution time of the forward propagation of pipeline stage i as measured under the configuration. W s The execution time of backpropagation in pipeline stage i as measured under the given configuration. This represents the communication time for the forward propagation of pipeline stages i and i+1. Communication time for backpropagation between pipeline stages i and i+1, AR s (i) indicates the configuration of W s The All-Reduce collection communication time required for pipeline stage i in the pipeline.

[0031] In some embodiments, the step of fine-tuning the bypass subnetwork using cached data specifically includes:

[0032] Save the intermediate activation results during the fine-tuning process to obtain cached data;

[0033] The model parameters of the bypass sub-network and the cached data are synchronized to all edge devices;

[0034] The bypass subnetwork is fine-tuned using a data-parallel approach.

[0035] This invention also proposes a time- and memory-efficient edge-cooperative fine-tuning system for large language models, the system comprising:

[0036] The target model building module is used to build bypass subnetworks for large language models to obtain target LLMs.

[0037] The test fine-tuning module uses various edge devices to test and fine-tune the target LLMs, and records relevant information about the edge devices and the model.

[0038] The orchestration module, based on the relevant information, divides the target LLMs and the edge devices to obtain a partitioning and orchestration scheme;

[0039] The collaborative fine-tuning module, based on the partitioning and orchestration scheme, performs target LLMs fine-tuning for edge device collaboration to obtain the fine-tuning results.

[0040] Based on the above scheme, this invention provides a time- and memory-efficient edge collaborative fine-tuning method and system for large language models. First, this scheme allows user data to be processed directly on the device that generates the data, thereby effectively protecting user privacy and data security. Simultaneously, this mode eliminates the need for extensive data exchange with cloud data centers, reducing problems caused by backbone network instability and poor service quality. By assembling a parallel bypass network for the target LLMs, the computational and memory overhead during model fine-tuning is effectively reduced. Leveraging the characteristic that the backbone model parameters remain constant, an intermediate activation result caching technique is designed, further reducing the computational resources required for fine-tuning LLMs in edge environments. It fully utilizes available idle computing power in edge environments, employing multiple edge devices for collaborative computation, thereby breaking down the limited computing resource barriers of individual edge devices and achieving more resource-efficient and scalable edge-side fine-tuning of personal LLMs. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the steps of the time- and memory-efficient edge collaborative fine-tuning method for large language models of the present invention.

[0042] Figure 2 This is a schematic diagram of the target LLMs after adding a bypass subnetwork in a specific embodiment of the present invention;

[0043] Figure 3This is a schematic diagram of the LLMs fine-tuning process in the parallel pipeline stage of a specific embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the training process of a batch of samples according to a specific embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram illustrating an example of an edge-collaborative large language model using an intermediate activation result caching and data-parallel training architecture in a specific embodiment of the present invention.

[0046] Figure 6 This is a schematic diagram of the comparative experimental data of the present invention;

[0047] Figure 7 This is a structural block diagram of the time- and memory-efficient edge collaborative fine-tuning system for large language models according to the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0050] It should be understood that the terms "system," "apparatus," "unit," and / or "module" used in this application are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0051] Unless the context explicitly indicates an exception, words such as "a," "an," "a kind," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list; a method or apparatus may also include other steps or elements. An element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0052] In the description of the embodiments of this application, "a plurality of" refers to two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0053] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them.

[0054] Reference Figure 1 This is a flowchart illustrating an optional example of the time- and memory-efficient edge-cooperative fine-tuning method for large language models proposed in this invention. This method can be applied to computer devices, and the model fine-tuning method proposed in this embodiment may include, but is not limited to, the following steps:

[0055] Step S1: Construct a bypass subnetwork for the large language model to obtain the target LLMs;

[0056] Step S2: Use each edge device to test and fine-tune the target LLMs, and record the relevant information of the edge devices and the model;

[0057] Step S3: Based on the relevant information, the target LLMs and the edge devices are divided to obtain a partitioning and orchestration scheme;

[0058] Step S4: Based on the partitioning and orchestration scheme, fine-tune the target LLMs for edge device collaboration to obtain the fine-tuning results.

[0059] This invention can be deployed in multi-device edge environments such as smart homes and smart factories. These environments include multiple IoT devices with certain computing capabilities, interconnected via a local area network. By coordinating the computing resources of these IoT devices (such as mobile phones, personal computers, and smart home devices), this invention can efficiently fine-tune large-scale personal language models while protecting user privacy. This fine-tuning enables the pre-trained large language model to adapt to various specific downstream edge intelligence applications, such as intelligent personal assistants and private psychological counseling.

[0060] In some feasible embodiments, step S1 specifically includes:

[0061] This step sets up a Parallel Adapters algorithm to fine-tune the target LLMs model. This algorithm adds a parallel, lightweight bypass network alongside the target LLMs. The structure of the target LLMs model after adding Parallel Adapters is as follows. Figure 2 As shown, the blue modules represent LLMs layers with frozen parameters, while the red modules represent parallel networks whose parameters can be trained. Compared to existing LLMs fine-tuning techniques such as Adapters and LoRA, the ParallelAdapters technology proposed in this patent opens an independent parallel channel for trainable lightweight parallel networks. This channel takes the intermediate activation results generated by each LLMs layer as input and outputs the final prediction result. This design eliminates the need for backpropagation through the entire LLMs backbone network during LLMs fine-tuning, significantly reducing the computational overhead of backpropagation and the memory burden caused by storing a large number of intermediate activation results generated by the LLMs backbone network. This makes it more suitable for resource-constrained edge environments.

[0062] In some feasible embodiments, step S2 specifically includes:

[0063] S2.1 Deploy the target LLMs on edge devices;

[0064] S2.2, Fine-tuning the target LLMs offline based on test data;

[0065] S2.3 Record the training speed of the edge device under different loads, the network communication speed between devices and the memory budget, as well as the intermediate activation results, model weights and gradient tensor dimensions of the target LLMs.

[0066] Specifically, offline fine-tuning of standard LLMs is performed using test sample data, and information required for subsequent dynamic programming model partitioning and orchestration algorithms is recorded. Definition and Let u be the execution time required for the forward and backward propagation of the l-th layer on device d with a batch size of β, respectively. d This represents the memory budget for device d. In addition, it is necessary to collect the intermediate activation results of the LLMs model, the tensor dimensions of the model weights and gradients, to calculate the required memory and communication time.

[0067] In some feasible embodiments, step S4 specifically includes:

[0068] like Figure 3As shown, to perform hybrid data parallelism and pipelined parallelism fine-tuning of LLMs across multiple edge devices, the LLM first needs to be divided into multiple sub-models, each containing one or more consecutive LLM layers. Edge devices are divided into multiple device groups, with the number of device groups equal to the number of sub-models, and each device group containing one or more edge devices. Each device group is associated with a sub-model, and all devices within that group work together in data parallelism to fine-tune the sub-model. Multiple device groups then fine-tune the entire target LLM in pipelined parallelism; each device group and its associated sub-model is called a pipeline stage. After training begins, a batch of data is divided into multiple micro-batches and simultaneously input into the hybrid parallel architecture to improve the parallel efficiency of parallel training. After all micro-batches of a batch have been trained, the devices in each pipeline stage perform an All-Reduce aggregate communication to synchronize gradients in the Parallel Adapters network, and then update the model parameters uniformly after synchronization.

[0069] The LLMs partitioning and device partitioning designed in this embodiment are divided into two steps:

[0070] S3.1, We will try to make the execution time required for each pipeline stage as close as possible;

[0071] Assuming the target LLM contains L layers, define It is an ordered set of all edge devices participating in collaborative computing. These are the first n devices in this ordered set. (Definition) For equipment group The goal is to determine the execution time of the slowest pipeline stage in the optimal subpipeline with s pipeline stages obtained from the submodel orchestration of LLM layers x to y. To solve this orchestration problem, we design a dynamic programming algorithm to recursively address it. The state transition equation of the dynamic programming algorithm is as follows:

[0072]

[0073] The first item in max is the device group. The second term is the execution time of the slowest pipeline stage in the optimal sub-pipeline with s-1 pipeline stages obtained from the sub-model orchestration of LLM layers 0 to q. n-m ,…,d n-1 The time required for forward and backward propagation to be performed in a single pipeline stage consisting of the sub-models from q+1 to y in the LLM layer. Where T(x→y,D) n The calculation method for ) is as follows:

[0074]

[0075] Where M is the number of micro-batches in a batch, and B is the number of samples in a micro-batch. During fine-tuning, a micro-batch is evenly distributed across n devices for parallel data processing. The peak memory overhead of device d during fine-tuning is defined as m. d u d This represents the memory budget for device d. Before a memory overflow error occurs, The value is determined by the sum of the forward and backward propagation execution times required for the slowest edge device. In this embodiment, the value is set to positive infinity when a memory overflow problem occurs. During the dynamic programming orchestration process, the pipeline orchestration configuration is recorded, including the split points of the LLMs model and the way devices are grouped.

[0076] T(x→y,{d0…,d n-1 In the}), x and y represent the transition from layer x to layer y in the LLM layer, respectively. For example, for the T(q+1→y,{d) mentioned in the previous paragraph... n-m …,d n-1}), x is assigned as q+1.

[0077] S3.2, Search LLMs partitions and device orchestration configurations based on communication time. s .

[0078] After completing the dynamic programming orchestration, we will obtain a set of configuration files, each corresponding to a configuration that orchestrates the target LLM model and all edge devices into different pipeline stages, s. This set of configuration files is defined as follows: Our next task is to determine the optimal number of pipeline stages. Using these configurations, we can determine the number of stages for each configuration (W). s Measure the execution time of the forward and backward propagation of pipeline stage i, and define them as follows: and Similarly, we define the forward and backward communication times for pipeline stages i and i+1 as follows: and Define AR s (i) Configure W s The All-Reduce collection communication time required for pipeline stage i. For example... Figure 4 As shown, we can divide the training of a batch of samples into three stages: the start stage, the execution stage, and the end stage. These are defined as follows: and Their calculation methods are as follows:

[0079]

[0080] The goal is to find the number of stages *s* that minimizes these three factors, along with its corresponding LLM partitioning and device orchestration configuration *W*. s :

[0081]

[0082] In some feasible embodiments, step S4 specifically includes:

[0083] Based on the optimal LLMs partitioning and device orchestration configuration obtained from S3, the LLMs model is segmented, and all edge devices are divided into groups with corresponding configurations. The corresponding LLM sub-models are deployed to the devices in the corresponding device groups and loaded into the device memory.

[0084] A batch of samples is divided into multiple micro-batches and injected into a hybrid pipeline for execution. All devices use the PyTorch deep learning framework for automatic differencing and forward and backward propagation computation of the target LLM model's computation graph. Each terminal device uses the GLOO backend for device-to-device communication, exchanging intermediate computation data required during the computation process.

[0085] In some feasible embodiments, it also includes:

[0086] Step S5: Fine-tune the bypass subnetwork using cached data.

[0087] S5.1 Save the intermediate activation results during the fine-tuning process to obtain cached data;

[0088] In the scenario of fine-tuning LLMs for individuals, the fine-tuning dataset is typically a small dataset collected from peripheral environments. Therefore, the same small dataset is often repeatedly used for fine-tuning until the target LLM achieves the desired model performance. Simultaneously, when using Parallel Adapters for LLM fine-tuning, the LLM backbone network parameters remain constant, ensuring that for the same input sample, the intermediate activation results generated by the LLM backbone network are consistent. For example... Figure 2 As shown in the figure, the intermediate activation results marked by the red circles remain unchanged for the same sample during the fine-tuning process. Therefore, in this embodiment, during the fine-tuning process, these intermediate activation results of all input samples are cached and saved to the device's storage. When the same sample is used for fine-tuning again, they are read directly from the hard drive. This avoids repeatedly executing the forward propagation process of LLMs when using the same sample for fine-tuning.

[0089] S5.2. Synchronize the model parameters of the bypass sub-network and the cached data to all edge devices;

[0090] S5.3 Fine-tuning of the bypass subnetwork using a data parallel approach.

[0091] Specifically, the model parameters of the lightweight parallel adapters are synchronized to all devices, as are the intermediate activation results collected from each device in step S5. Finally, multiple edge devices fine-tune the lightweight parallel adapters individually using a data-parallel approach. When intermediate activation results are needed as input to the parallel network, this embodiment reads the intermediate activation results from the hard drive. Figure 5 An example of edge-coordinated large language models using an intermediate activation result cache and a data-parallel training architecture is demonstrated:

[0092] Furthermore, after all the samples used for fine-tuning have completed one training iteration, the intermediate activation results generated by the forward propagation computation of all target fine-tuning samples in the LLMs backbone model have been cached, that is... Figure 1 The red section in the image indicates that forward and backward propagation of the LLM backbone network is no longer necessary, and all devices can release the LLM backbone network from memory to reduce memory overhead.

[0093] Based on the above scheme, the present invention also provides comparative experimental data:

[0094] This invention discloses a time- and memory-efficient collaborative edge computing framework for fine-tuning personal large language models. The experimental environment for this embodiment is as follows: This example constructs an edge device cluster consisting of four NVIDIA Jetson Nanos. All terminal devices use the PyTorch deep learning framework for automatic differencing and forward and backward propagation of the deep neural network model computation graph, using GLOO as the distributed communication backend. We implemented three common large language models: T5-Base (containing 0.25 billion parameters), BART-Large (containing 0.41 billion parameters), and T5-Large (containing 0.74 billion parameters), and conducted experiments on four different tasks in the GLUE dataset: MRPC, STS-B, SST-2, and QNLI. We compared three benchmark edge collaborative computing methods:

[0095] Collaborative Method 1: Fine-tuning is performed only on a single Jetson Nano.

[0096] Collaborative Method 2: Fine-tuning of pipeline parallelism on four Jetson Nanos.

[0097] Collaborative Method 3: Fine-tuning of data parallelism on four Jetson Nanos.

[0098] Collaborative Method 4: The edge collaborative parallel technology proposed in this invention.

[0099] These three edge collaborative computing methods are combined with three mainstream LLM fine-tuning techniques, which are:

[0100] Fine-tuning technique 1: Perform full parameter fine-tuning without freezing the parameters of the LLMs backbone model.

[0101] Fine-tuning technique 2: Efficient fine-tuning technique using Adapter parameters.

[0102] Fine-tuning technique 3: Employs LoRA's parameter fine-tuning technique for high efficiency.

[0103] Fine-tuning technique 4: The fine-tuning technique for Parallel Adapters proposed in this invention.

[0104] These three edge collaboration methods and these three common fine-tuning techniques can be combined to create nine different combinations. (Next) Figure 6 This is a comparison of the fine-tuning time required for the example of this invention and these nine different combinations. For the MRPC and STS-B tasks, the datasets are relatively small, so we trained three times using the complete dataset. For the SST-2 and QNLI tasks, the datasets are larger, so we trained once using the complete dataset.

[0105] Depend on Figure 6 As can be seen, the combination of the proposed collaborative method 4 and fine-tuning technique 4 is significantly better than the nine combinations consisting of three other edge collaborative methods and three common fine-tuning techniques in terms of fine-tuning time, achieving a speedup of up to 8.6 times or more. The figure shows OOM (Out of Memory), indicating that some combinations of fine-tuning techniques and collaborative methods encountered memory overflow issues, while the method proposed in this invention did not encounter this problem. This sufficiently demonstrates the superiority of this invention in terms of time and memory during LLMs fine-tuning.

[0106] like Figure 7 As shown, a time- and memory-efficient edge-cooperative fine-tuning system for large language models includes:

[0107] The target model building module is used to build bypass subnetworks for large language models to obtain target LLMs.

[0108] The test fine-tuning module uses various edge devices to test and fine-tune the target LLMs, and records relevant information about the edge devices and the model.

[0109] The orchestration module, based on the relevant information, divides the target LLMs and the edge devices to obtain a partitioning and orchestration scheme;

[0110] The collaborative fine-tuning module, based on the partitioning and orchestration scheme, performs target LLMs fine-tuning for edge device collaboration to obtain the fine-tuning results.

[0111] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0112] Time- and memory-efficient edge-coordinated fine-tuning device for large language models:

[0113] At least one processor;

[0114] At least one memory for storing at least one program;

[0115] When the at least one program is executed by the at least one processor, the at least one processor implements the time- and memory-efficient large language model edge co-tuning method as described above.

[0116] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0117] A storage medium storing processor-executable instructions, which, when executed by a processor, are used to implement the time- and memory-efficient edge-cooperative fine-tuning method for large language models as described above.

[0118] The content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0119] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A time- and memory-efficient edge-cooperative fine-tuning method for large language models, characterized in that, Includes the following steps: Constructing bypass subnetworks for large language models yields target LLMs; The target LLMs were tested and fine-tuned using various edge devices, and relevant information about the edge devices and the model was recorded. Based on the aforementioned information, the target LLMs and the edge devices are divided to obtain a partitioning and orchestration scheme; Based on the aforementioned partitioning and orchestration scheme, the target LLMs for edge device collaboration are fine-tuned to obtain the fine-tuning results; The step of dividing the target LLMs and the edge devices based on the relevant information to obtain a partitioning and orchestration scheme specifically includes: Define the constraint objectives, which are to minimize the difference in execution time between each pipeline stage and to minimize the communication time. Based on the constraint objectives and dynamic programming algorithm, a partitioning and arrangement scheme is obtained; The partitioning and arrangement scheme includes multiple sub-models and device groups, and the sub-models and device groups have a corresponding relationship. The state transition equation of the dynamic programming algorithm is: in, Indicates that it is a group of equipment and LLM layer arrive The sub-model arrangement yields the following characteristics: The execution time of the slowest pipeline stage in the optimal sub-pipeline of each pipeline stage, where m and n represent different index subscripts; Indicates that it is a group of equipment and LLM layer arrive The sub-model arrangement yields the following characteristics: The execution time of the slowest pipeline stage in the optimal sub-pipeline of each pipeline stage; Indicates that it is a group of equipment and LLM layer arrive The time required for a single pipeline stage, composed of sub-models, to perform forward and backward propagation; in The calculation method is as follows: in, The number of micro-batches into which a batch is divided. The number of samples in a micro-batch. Indicates the equipment during fine-tuning. Peak memory overhead, Indicates device Memory budget, This represents the time required to perform the forward propagation of LLM layer l on device d. This represents the time required to perform the backpropagation of LLM layer l on device d.

2. The time- and memory-efficient edge-cooperative fine-tuning method for large language models according to claim 1, characterized in that, Also includes: The bypass subnetwork is fine-tuned using cached data.

3. The time- and memory-efficient edge-cooperative fine-tuning method for large language models according to claim 2, characterized in that, The step of testing and fine-tuning the target LLMs using various edge devices and recording relevant information about the edge devices and the model specifically includes: Deploy the target LLMs on edge devices; Fine-tuning of the target LLMs was performed offline based on test data; Record the training speed of the edge device under different loads, the network communication speed between devices and the memory budget, as well as the intermediate activation results, model weights and gradient tensor dimensions of the target LLMs.

4. The time- and memory-efficient edge-cooperative fine-tuning method for large language models according to claim 1, characterized in that, The formula for the shortest communication time is expressed as follows: in, This indicates the start phase of training within a batch of samples. This represents the training execution phase within a batch of samples. This indicates the end of the training phase in a batch of samples. Indicates the number of stages. express The measured pipeline stage under the configuration The execution time of the forward propagation, express The measured pipeline stage under the configuration The execution time of the backpropagation. Indicates the assembly line stage and Forward propagation communication time, Assembly line stage and Backpropagation communication time, Indicates configuration The assembly line stage Required All-Reduce collection communication time.

5. The time- and memory-efficient edge-cooperative fine-tuning method for large language models according to claim 4, characterized in that, The step of fine-tuning the bypass subnetwork using cached data specifically includes: Save the intermediate activation results during the fine-tuning process to obtain cached data; The model parameters of the bypass sub-network and the cached data are synchronized to all edge devices; The bypass subnetwork is fine-tuned using a data-parallel approach.

6. A time- and memory-efficient edge-cooperative fine-tuning system for large language models, characterized in that: include: The target model building module is used to build bypass subnetworks for large language models to obtain target LLMs. The test fine-tuning module uses various edge devices to test and fine-tune the target LLMs, and records relevant information about the edge devices and the model. The orchestration module, based on the relevant information, divides the target LLMs and the edge devices to obtain a partitioning and orchestration scheme; The collaborative fine-tuning module, based on the partitioning and orchestration scheme, performs target LLMs fine-tuning for edge device collaboration to obtain the fine-tuning results; Based on the relevant information, the target LLMs and the edge devices are partitioned to obtain a partitioning and orchestration scheme. Specifically, this includes: defining constraint objectives, which are minimizing the difference in execution time among each pipeline stage and minimizing communication time; obtaining a partitioning and orchestration scheme based on the constraint objectives and a dynamic programming algorithm; the partitioning and orchestration scheme includes multiple sub-models and device groups, and the sub-models and device groups have corresponding relationships. The state transition equation of the dynamic programming algorithm is: in, Indicates that it is a group of equipment and LLM layer arrive The sub-model arrangement yields the following characteristics: The execution time of the slowest pipeline stage in the optimal sub-pipeline of each pipeline stage, where m and n represent different index subscripts; Indicates that it is a group of equipment and LLM layer arrive The sub-model arrangement yields the following characteristics: The execution time of the slowest pipeline stage in the optimal sub-pipeline of each pipeline stage; Indicates that it is a group of equipment and LLM layer arrive The time required for a single pipeline stage, composed of sub-models, to perform forward and backward propagation; in The calculation method is as follows: in, The number of micro-batches into which a batch is divided. The number of samples in a micro-batch. Indicates the equipment during fine-tuning. Peak memory overhead, Indicates device Memory budget, This represents the time required to perform the forward propagation of LLM layer l on device d. This represents the time required to perform the backpropagation of LLM layer l on device d.

Images