Large language model training method and related equipment

Abstract

One or more embodiments of the invention provide a large language model training method and related equipment. The method comprises the following steps: decomposing an encoder of the large language model into a plurality of sub-models; the sub-models are deployed on devices of a device cluster, the large language model is deployed on a server, and the sub-models are in one-to-one correspondence with the devices; the large language model is iteratively trained, and the equipment cluster and the server process data in a pipeline parallel mode in each iteration training. The invention provides a method for training a large language model on distributed equipment based on an encoder block segmentation mechanism of pipeline parallelism. According to the method and the device, the limitation of equipment resources can be solved while the user data privacy is ensured.

Description

Technical Field

[0001] One or more embodiments of this application relate to the field of artificial intelligence technology, and in particular to a method and related equipment for training large-scale language models. Background Technology

[0002] A language model is a fundamental concept in the field of natural language processing; it is a mathematical model used to calculate the probability of language. With the development of artificial intelligence technology, large language models (LLMs), such as the GPT series models proposed by OpenAI and the LLaMA model designed by Facebook Menta AI, have achieved remarkable results in language generation and understanding.

[0003] The following problems still exist in LLM training: a large amount of high-quality training data is required, but with the advancement of technology, users are paying more attention to data privacy, and it is difficult to call a large amount of high-quality data to the central server; the storage and computing resources required for training and inference are huge, and edge devices cannot bear the computing pressure.

[0004] How to train large-scale language models reasonably and effectively remains a problem worthy of further research. Summary of the Invention

[0005] In view of this, the purpose of one or more embodiments of this application is to provide a large-scale language model training method and related equipment to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, one or more embodiments of this application provide a method for training large-scale language models, including:

[0007] The encoder of the large language model is decomposed into multiple sub-models;

[0008] The sub-models are deployed on devices in the device cluster, and the large language model is deployed on a server, with each sub-model corresponding to a specific device.

[0009] The large language model is trained iteratively, wherein in each iteration, the device cluster and the server process the data in a pipelined parallel manner.

[0010] Optionally, for iterative training of the large language model, the device cluster and the server perform the following operations:

[0011] The device cluster obtains the forward propagation result of the encoder based on local data and sends the forward propagation result to the server;

[0012] The server obtains the gradient result based on the forward propagation result and sends the gradient result to the device cluster;

[0013] The device cluster obtains the backpropagation result of the encoder based on the gradient result to update the local encoder parameters, and sends the local encoder parameters to the server;

[0014] The server obtains global encoder parameters based on the encoder parameters and sends the global encoder parameters to all devices in the cluster.

[0015] Optionally, for the device cluster to obtain the encoder's forward propagation result based on local data, the devices in the device cluster sequentially perform the following operations:

[0016] The first device in the device cluster obtains the forward propagation result of the sub-model on that device based on the local data, and sends the forward propagation result to the next device;

[0017] The intermediate device in the device cluster obtains the forward propagation result of the sub-model on the device based on the local data and the received forward propagation result, and sends the forward propagation result of the sub-model on the device to the next device;

[0018] The last device in the device cluster obtains the forward propagation result of the sub-model on the device based on the local data and the received forward propagation result, and sends the forward propagation result of the sub-model on the device to the server.

[0019] Optionally, the server obtains the gradient result based on the forward propagation result, including:

[0020] The server obtains the gradient result based on the forward propagation result and the local data, through forward and backward propagation.

[0021] Optionally, the local data is obtained from local source data through an embedding layer.

[0022] Optionally, before the server obtains the global encoder parameters based on the encoder parameters, it further includes:

[0023] The encoder parameters of all the device clusters are aggregated to obtain aggregate encoder parameters;

[0024] The aggregate encoder parameters are sent to the server.

[0025] Optionally, the encoder of the large language model is decomposed into multiple sub-models in units of encoder blocks; the device cluster obtains the forward propagation result of the encoder based on the local data decomposed into multiple micro-batches.

[0026] The number of encoder blocks and the number of micro-batches are determined by the following objective function and constraint function;

[0027]

[0028] p represents the number of sub-models, l represents the number of encoder blocks, m represents the number of micro-batches, and num represents the number of micro-batches. EB M represents the total number of encoder blocks, S represents the capacity of the device, and S represents the total number of encoder blocks. ME P represents the number of individual encoder blocks. i P represents the transmit power of the i-th device. max This indicates the device's maximum transmission power.

[0029] Based on the same inventive concept, one or more embodiments of this application also provide a large-scale language model training device, including:

[0030] The decomposition module is configured to decompose the encoder of the large language model into multiple sub-models;

[0031] The deployment module is configured to deploy the sub-model on devices in the device cluster and deploy the large language model on a server, wherein the sub-model corresponds one-to-one with the device;

[0032] The training module is configured to iteratively train the large language model, wherein in each iteration, the device cluster and the server process the data in a pipelined parallel manner.

[0033] Based on the same inventive concept, one or more embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the large language model training method as described in any of the above.

[0034] Based on the same inventive concept, one or more embodiments of this application also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute any of the large language model training methods described above.

[0035] As can be seen from the above, the large language model training method provided in one or more embodiments of this application decomposes the encoder of the large language model into multiple sub-models; deploys the sub-models on devices in a device cluster, and deploys the large language model on a server, with each sub-model corresponding to one of the devices; iteratively trains the large language model, wherein in each iteration, the device cluster and the server process data in a pipelined parallel manner.

[0036] The method provided in one or more embodiments of this application avoids the problem of data privacy leakage by decomposing the model into multiple sub-models and deploying them on a device cluster for training; at the same time, it avoids the problem of weak computing power of edge devices by using a pipelined parallel approach.

[0037] The large-scale language model training device, electronic device, and computer-readable storage medium provided in this application can all implement the steps of the above-mentioned large-scale language model training method, and therefore also have the beneficial effects of the above-mentioned large-scale language model training method. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in one or more embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only one or more embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a large-scale language model training method according to one or more embodiments of this application;

[0040] Figure 2 This is a schematic diagram of the structure of a large language model training device according to one or more embodiments of this application;

[0041] Figure 3 This is a schematic diagram of the data decomposition structure of one or more embodiments of this application;

[0042] Figures 4(a) and 4(b) are schematic diagrams illustrating the training effects of one or more embodiments of this application;

[0043] Figures 5(a) and 5(b) are schematic diagrams of training time for one or more embodiments of this application;

[0044] Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to one or more embodiments of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0046] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in one or more embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0047] As described in the background section, the following problems still exist in LLM training: a large amount of high-quality training data is required, but with the advancement of technology, users are paying more attention to data privacy, and it is difficult to call a large amount of high-quality data to the central server; the storage and computing resources required for training and inference are huge, and edge devices cannot bear the computing pressure.

[0048] Related technologies propose solutions such as segmenting large language models before training to alleviate computational pressure. However, these technologies do not consider the model architecture or data privacy issues when segmenting large language models.

[0049] Specifically, large-scale language models based on the Transformer encoder-decoder architecture, such as the GPT series models proposed by OpenAI and the LLaMA model designed by Facebook Menta AI, currently dominate the market. However, these technologies do not offer model segmentation solutions for this architecture, nor do they consider data privacy issues.

[0050] Therefore, this application proposes a method for training large-scale language models. First, based on the Transformer encoder-decoder architecture, the encoder in the large-scale language model is split into multiple sub-models. In implementing this application, the applicant discovered that Federated Learning (FL) has the advantage of sharing knowledge (parameters, gradients, etc.) without compromising user privacy. Therefore, edge device clusters are used to train the encoder part of the model. Simultaneously, to avoid the limited computing power of edge device clusters from being unable to support the model, the applicant introduces a pipelined parallel strategy, dividing the model into multiple sub-models and deploying them on multiple devices.

[0051] refer to Figure 1The large-scale language model training method according to one or more embodiments of this application includes the following steps:

[0052] Step S101: Decompose the encoder of the large language model into multiple sub-models.

[0053] It is understood that a large language model based on the Transformer encoder-decoder architecture includes both an encoder and a decoder. Since the decoder in this architecture cannot be directly split, this application decomposes the encoder into multiple sub-models. Preferably, the encoder can be decomposed into multiple sub-models in units of Transformer Encoder Blocks (TEBs).

[0054] Step S102: Deploy the above sub-models on the devices in the device cluster, and deploy the above large language model on the server. The above sub-models correspond one-to-one with the devices.

[0055] The sub-models of the aforementioned large language model are deployed across multiple device clusters. Each device cluster comprises the same number of devices. It is important to note that the number of sub-models is also equal to the number of devices in each device cluster.

[0056] Deploying a sub-model on a device in a device cluster means that any sub-model corresponds to a device in each device cluster for deployment.

[0057] For example, one embodiment of this application is an edge-to-edge collaborative federated learning system consisting of one server and N device clusters, where each device cluster consists of p devices. In this embodiment, p also represents the number of sub-models split by the encoder.

[0058] Step S103: Iteratively train the large language model, wherein in each iteration, the device cluster and the server process the data in a pipelined parallel manner.

[0059] The iterative training operation of the large-scale language model in this application includes: each device cluster obtains the forward propagation result of the corresponding encoder based on local data and sends the forward propagation result to the server; the server performs forward and backward propagation based on all forward propagation results to obtain gradient results and sends the gradient results to each device cluster; each device cluster obtains the backward propagation result of the corresponding encoder based on the gradient results and updates the local encoder parameters, and then sends the local encoder parameters to the server; the server obtains the global encoder parameters based on all encoder parameters and sends the global encoder parameters to all device clusters.

[0060] It is important to note that the aforementioned device cluster operates the encoder and updates local encoder parameters based on a pipelined parallel mechanism. Specifically, this includes: the first device in the cluster obtains the forward propagation result of its sub-model based on local data and sends the forward propagation result to the next device; the middle device in the cluster obtains the forward propagation result of its sub-model based on local data and the forward propagation result received from the previous device, and sends the forward propagation result of its sub-model to the next device; the last device in the cluster obtains the forward propagation result of its sub-model based on local data and the forward propagation result received from the previous device, and sends the forward propagation result of its sub-model to the server.

[0061] The server deploys the complete large-scale language model (including encoder and decoder) and is responsible for performing decoder training and aggregating encoder parameters from all device clusters to update the global model. Decoder training may involve: obtaining gradient results through forward and back propagation based on forward propagation results sent from the device cluster and local data.

[0062] The subsequent steps include: the server sending the gradient results to each device cluster; the device clusters obtaining the encoder backpropagation results based on the gradient data and updating the encoder parameters, then sending the updated encoder parameters back to the server; and the server performing global integration based on the updated encoder parameters to obtain global encoder parameters, which are then sent to each device cluster. The process of the device clusters obtaining the encoder backpropagation results based on the gradient data is the reverse operation of the device clusters obtaining the forward propagation results. The server can integrate the updated encoder parameters using methods such as federated averaging.

[0063] Preferably, in the process of obtaining the forward propagation results based on local data, the local data can be divided into multiple micro-batches to improve training efficiency.

[0064] like Figure 3 As shown, in one embodiment of this application, local data is divided into 6 micro-batches. Figure 3 In this diagram, FE represents the forward propagation of the encoder block, and BE represents the backward propagation of the encoder block. The numbers following FE and BE indicate the encoder block number, and the numbers below FE and BE indicate the micro-batch number. It can be seen that in this embodiment, a sub-model is divided into 4 encoder blocks, each encoder block corresponds to 6 micro-batches, and each square represents the training process of one micro-batch. The data in the micro-batch sequentially passes through the encoder blocks on all devices, then through all decoder blocks to perform the forward propagation operation, followed by backward propagation.

[0065] Since the optimal segmentation scheme for a large language model varies across different scenarios, it is necessary to compute the optimal segmentation scheme for each environment. Most related techniques employ a method of traversing all segmentation schemes to find the optimal one. However, LLMs have more layers and parameters, significantly increasing the number of segmentation schemes and training costs, rendering the traversal methods of related techniques unsuitable.

[0066] Therefore, this application constructs an optimization problem with the objective of minimizing training time, and obtains the optimal large model splitting scheme through calculation.

[0067] In implementing this application, the applicant first determines that, for transmission speed, the uplink rate from device i to another device should be:

[0068]

[0069] Where B is the transmission bandwidth between the two devices, and P... i This is the device's transmission power; assuming these two values ​​are equal for all devices. h dd I is the channel gain between devices, and N0 is the noise power spectral density. dd Interference is caused by a device that uses the same RB to communicate with another device.

[0070] Since data transmission between devices is an equal communication process, with downlink and uplink rates being equal, the uplink rate can be directly used as the transmission rate.

[0071] Similarly, the uplink rate sent from device i to the server can be obtained:

[0072]

[0073] Where h ds It is the channel gain between the device and the server, I ds The interference is caused by a device that uses the same RB to connect to another server.

[0074] Since the downlink rate depends on the server's transmit power, and the above assumes the server has strong communication and computing capabilities, the data transmission rate from the server can be considered infinite. Therefore, the downlink rate of this process can be considered infinite, and the time of the corresponding phase can be ignored.

[0075] Based on this, the iterative training process of this application can be regarded as consisting of 11 stages, and the time delay statistics of each stage are as follows.

[0076] Considering the encoder of any device performing forward propagation based on local data as the first stage, in this stage all local data of the device passes through all sub-models of the encoder, then a total of (p+m-1) micro-batches pass through the encoder for forward propagation. The formula for calculating the latency in this stage is:

[0077]

[0078] Where p represents the number of devices in the device cluster (i.e., the number of sub-models), m represents the number of micro-batches into which a batch of data is divided when using the pipelined parallel mechanism, b represents the batch size, l represents the number of encoder blocks held by each device, and W EF C represents the workload of forward propagating one unit of data through one encoder block. d This indicates the device's computing power.

[0079] The second stage involves the forward propagation results flowing between devices in the cluster. In this stage, the forward propagation results are transmitted from one device to another, a total of (p+m-2) times. The formula for calculating the communication latency in this stage is:

[0080]

[0081] Among them, D EF v represents the size of intermediate data transmitted between encoder blocks per unit of data. d This indicates the device-to-device transmission rate.

[0082] The third stage involves the device cluster transmitting the forward propagation results to the server. The formula for calculating the communication latency in this stage is:

[0083]

[0084] Among them, D EF This indicates the size of the intermediate data transmitted per unit of data from the encoder to the decoder; v ds This indicates the data transfer rate from the device to the server.

[0085] The fourth stage is the forward propagation of the decoders on the server. In this stage, local data is propagated forward by the server through all decoders. The latency in this stage is negligible.

[0086] The fifth stage is the backpropagation of the decoders on the server. In this stage, local data is propagated through all decoders on the server, with each decoder performing the backpropagation. The latency in this stage is negligible.

[0087] The sixth stage involves the server transmitting gradient data to the device cluster. The latency in this stage is negligible.

[0088] The seventh segment involves the transmission of gradient data from one device to another within the device cluster, with a total of (p+m-2) transmissions. The communication latency for this stage is calculated using the following formula:

[0089]

[0090] Among them, D EB This indicates the size of the gradient data transmitted between encoder blocks.

[0091] The eighth stage involves the backward propagation of data on the encoder deployed on the device, requiring the calculation of a total of (p+m-1) data units. The formula for calculating the latency in this stage is:

[0092]

[0093] Among them, W EB This represents the amount of work required to backpropagate a unit of data through an encoder block.

[0094] The ninth stage involves the device cluster transmitting encoder parameters to the server. The formula for calculating the communication latency in this stage is:

[0095]

[0096] Among them, S ME This represents the model size of an encoder block.

[0097] The tenth stage involves the server performing global aggregation on large language models using a federated averaging algorithm. Communication latency in this stage is negligible.

[0098] The eleventh stage involves the server transmitting the updated encoder parameters to all devices in the cluster. The communication latency in this stage is negligible.

[0099] Based on the above, the time delay of the entire training process can be obtained using the following formula:

[0100] T = t EF +t EFT +t EDT +t EBT +t EB +t LMT

[0101] This application optimizes the objective function for training large-scale language models as follows:

[0102]

[0103] The objective function described above aims to find the optimal l and m to minimize the training time of a large language model.

[0104] The constraint function is set as follows:

[0105]

[0106] The above constraint functions respectively mean that all devices in the cluster must hold the complete encoder part; the number of encoder blocks deployed on the device meets the device memory requirements; the transmit power is less than or equal to the maximum transmit power; and the number of model blocks, the number of devices, and the number of micro-batches are all positive integers.

[0107] To verify the validity of this application, the applicant conducted the following experiment.

[0108] The experiment used the base-Transformer model commonly used in machine translation, which consists of 6 encoder blocks and 6 decoder blocks. Table 1 shows the number of parameters in each layer of the model. By comparing the number of parameters, we can roughly obtain the size of the sub-models after splitting.

[0109] Table 1 shows the number of parameters for each layer in the model.

[0110] Layer name Number of parameters Embedding Layer <![CDATA[7.04×10 6 ]]> Encoder Blocks 1-6 <![CDATA[3.15×10 6 ]]> Decoder Blocks <![CDATA[4.20×10 6 ×6=2.52×10 7 ]]> Linear and Softmax Layer <![CDATA[4.03×10 6 <!-- 7 -->]]> Total <![CDATA[5.52×10 7 ]]>

[0111] The dataset used is the multi30K dataset. multi30K is a dataset for machine translation tasks between German and English, in which each content has both human-translated English and German versions of the text, which can be used as source and target data for training and inference of machine translation tasks.

[0112] The control group of the experiment used a large language model based on Transformer, trained using a federated learning approach. It combines Transformer with a simple federated averaging algorithm and selects two device clusters as clients for the federated learning.

[0113] The experimental group used a large Transformer-based language model, trained using federated learning and pipelined parallelism, and combined the GPipe method with a federated averaging algorithm. The experimental group had two variables, l and m, where l represents the number of encoder blocks per device and m represents the number of micro-batches. Setting l to 1, 2, and 3, and m to 2 and 8, generated six experimental groups. For example, with l=2 and m=8, each device would hold 2 encoder blocks, there would be 3 devices in a cluster, and the number of micro-batches would be set to 8 when using the GPipe method.

[0114] By using the method proposed in this application, the shortest training time of 146.4 s can be pre-calculated when l=2 and m=6. As a comparison, when l=1 and m=2 are randomly selected, the training time is 148.1 s. This is longer than the "shortest training time" obtained by this invention. This demonstrates that by using the algorithm proposed in this invention, optimal parameters can be pre-calculated to obtain the shortest training time.

[0115] Analysis of the actual experimental results shows that when using the federated averaging algorithm for training, each device must store data with a resolution of 5.52 × 10⁻⁶. 7 The entire model with 10 parameters. By using the method of this invention, each device only needs to store the embedding layer and a few encoder blocks, which is only 7.04 × 10 6 +l×3.15×10 6 Parameters (in 1.02×10) 7 and 1.65×10 7 (between), that is, 1 / 5 to 1 / 3 of the previous one. This means that, using the method of the present invention, the model size held by each device can be reduced to 1 / 5 to 1 / 3 of the original model size.

[0116] Furthermore, this application analyzes the training performance of the model. Table 2 shows the training loss and test accuracy of the model under different values ​​of l and m. Figures 4(a) and 4(b) show the BLEU (Bilingual Evaluation Alternate) scores of the model. The BLEU score is a method for measuring the quality of machine translation. It scores the quality of machine translation by comparing the output of the machine translation with one or more human-translated references. The higher the score, the closer the machine translation is to human translation. Figure 4(a) shows the different BLEU scores corresponding to different values ​​of l when m=8. We show the different BLEU scores corresponding to different values ​​of m when l=2 in Figure 4(b).

[0117] According to Table 2, Figure 4(a) and Figure 4(b), the training loss, test accuracy and BLEU score are almost the same regardless of the values ​​of m and l. That is, the method of the present invention will not affect the training effect of the model.

[0118] Table 2. Training loss and test accuracy of the model under different values ​​of l and m.

[0119]

[0120]

[0121] Figures 5(a) and 5(b) show the training time for different values ​​of m when using the technical solution of this application. Figure 5(a) shows the training results for l=3 with different values ​​of m; Figure 5(b) shows the training results for l=2 with different values ​​of m. It can be seen that when a batch is divided into several micro-batches, the total training time will decrease slightly, but the impact is not significant.

[0122] In Figure 5(a), "org" indicates that we use only traditional federated learning, while in Figure 5(b), "org" indicates that we use both traditional federated learning and traditional pipelined parallelism (unlike the control group). As we can see from Figure 5(b), the method of this invention reduces training time compared to using traditional pipelined parallelism. This demonstrates the advantage of this invention in shortening training time.

[0123] In summary, the technical solution of this application has the following technical effects:

[0124] A federated learning framework for large-scale Transformer encoder-decoder architectures is proposed. Each device trains a portion of the Transformer encoder locally, then iteratively updates it locally, uploading the updated parameters to the server for aggregation into a global FL model containing a Transformer decoder. Within the cluster, intermediate data needs to be transmitted between multiple devices, with each device's output serving as input to the next Transformer encoder block. To protect data privacy, all transmitted data is embedded or processed, rather than the raw data.

[0125] We design a large model segmentation mechanism based on encoder blocks and employ a pipelined parallel strategy to execute the training logic of the large model. The encoder part of the large model is divided into several sub-models, each based on a Transformer encoder block, and each sub-model is deployed on a single device. The large model computation tasks on each device are executed in a pipelined parallel manner, including single forward propagation, backpropagation, and aggregation tasks.

[0126] An optimization model for wireless networks is constructed to obtain the optimal number of block partitions and the number of pipelined parallel subtasks, aiming to achieve the shortest training latency. The optimal parameters are the number of sub-models the encoder is divided into and the number of micro-batches each training batch is divided into. Simultaneously, this patent models the overall training process, constructs a convex optimization problem, and solves it.

[0127] We propose a fast packet forwarding mechanism with an embedded DPU, which offloads the aggregation computation process of federated learning to the DPU, enabling the model parameters on the local terminal device to be pre-calculated during network transmission, thereby reducing the communication pressure between the terminal and the server and the computation pressure on the edge server.

[0128] It should be noted that the local data received by the server is the same as the local data used by the device cluster to train the encoder, in order to calculate the expected output corresponding to the local data and to calculate the loss function. Preferably, the local data is obtained by processing the device source data through the embedding layer, thereby protecting data privacy.

[0129] Preferably, considering the frequent communication between the device cluster and edge server, and between devices, resulting in high communication latency, large network data traffic, and insufficient network bandwidth resources, a Data Processing Unit (DPU) is introduced in some embodiments to accelerate model parameter forwarding. Some embodiments of this application adopt an on-the-net computing approach and architecture, inserting the DPU onto the switches along the network transmission path from the cluster to the edge server. This allows for the pre-aggregation of model parameters within the cluster during network transmission, reducing data traffic transmitted over the network. Specifically, during the device cluster's update of its local encoder sub-model and the transmission of parameters between the device cluster and the server, the DPU on the switch can perform parameter aggregation in advance. This ensures that only one copy of the aggregated encoder parameter data is transmitted subsequently, reducing data traffic during transmission.

[0130] It should be noted that the methods of one or more embodiments of this application can be executed by a single device, such as a computer or server. The methods of this embodiment can also be applied in a distributed scenario, where multiple devices cooperate to complete the process. In such a distributed scenario, one of these devices may execute only one or more steps of the methods of one or more embodiments of this application, and the multiple devices will interact with each other to complete the method described.

[0131] It should be noted that the above description describes specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0132] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a large-scale language model training device.

[0133] like Figure 2 As shown, the above-mentioned device includes:

[0134] Decomposition module 11 is configured to decompose the encoder of the large language model into multiple sub-models.

[0135] Deployment module 12 is configured to deploy the above sub-models on devices in the device cluster and deploy the above large language model on the server, wherein the above sub-models correspond one-to-one with the devices;

[0136] Training module 13 is configured to iteratively train the large language model described above, wherein in each iteration, the device cluster and the server process the data in a pipelined parallel manner.

[0137] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, when implementing one or more embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0138] The apparatus described above is used to implement the corresponding methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0139] Figure 6 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0140] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0141] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this application are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0142] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0143] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0144] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0145] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this application, and not necessarily all the components shown in the figures.

[0146] The electronic devices described above are used to implement the corresponding methods in the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0147] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0148] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0149] Additionally, to simplify the description and discussion, and to avoid obscuring one or more embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be illustrated in block diagram form to avoid obscuring one or more embodiments of this application, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which one or more embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that one or more embodiments of this application may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0150] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0151] One or more embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this application should be included within the protection scope of this disclosure.

Claims

1. A large language model training method, characterized in that: include: Decomposing an encoder of the large language model into a plurality of sub-models; Deploy the sub-model on a device of the device cluster, and deploy the large language model on a server, wherein the sub-model corresponds to the device one by one; The large language model is trained iteratively, wherein in each iterative training, the device cluster and the server process data in a pipeline parallel manner.

2. The method according to claim 1, characterized in that For iterative training of the large language model, the device cluster and the server perform the following operations: The device cluster obtains a forward propagation result of the encoder based on local data, and sends the forward propagation result to the server; The server obtains a gradient result based on the forward propagation result and sends the gradient result to the device cluster; The device cluster obtains a back-propagation result of the encoder based on the gradient result to update a local encoder parameter, and sends the local encoder parameter to the server; The server obtains global encoder parameters based on the encoder parameters and sends the global encoder parameters to all the device clusters.

3. The method according to claim 2, characterized in that For a device cluster to obtain a forward propagation result of the encoder based on local data, the devices in the device cluster sequentially perform the following operations: The first device of the device cluster obtains a forward propagation result of the sub-model on the device based on the local data, and sends the forward propagation result to the next device; The intermediate device of the device cluster obtains the forward propagation result of the sub-model on the device based on the local data and the received forward propagation result, and sends the forward propagation result of the sub-model on the device to the next device; The last device in the device cluster obtains the forward propagation result of the sub-model on the device based on the local data and the received forward propagation result, and sends the forward propagation result of the sub-model on the device to the server.

4. The method according to claim 3, characterized in that The server obtains a gradient result based on the forward propagation result, including: The server obtains the gradient result through forward propagation and backward propagation based on the forward propagation result and the local data.

5. The method according to claim 3 or 4, characterized in that: The local data is obtained through an embedding layer based on the local source data.

6. The method according to claim 4, characterized in that Before the server obtains the global encoder parameters based on the encoder parameters, the server further includes: Aggregating encoder parameters of all the device clusters to obtain aggregate encoder parameters; The aggregated encoder parameters are sent to the server.

7. The method according to claim 2, characterized in that: The encoder of the large language model is decomposed into a plurality of sub-models in units of encoder blocks; the device cluster obtains a forward propagation result of the encoder based on the local data decomposed into a plurality of micro-batches; The number of encoder blocks and the number of micro-batches are determined by the following objective function and constraint function: p represents the number of sub-models, l represents the number of encoder blocks, m represents the number of micro-batches, num EB represents the total number of encoder blocks, M represents the capacity of the device, S ME represents the number of individual encoder blocks, P i represents the transmission power of the ith device, P max Indicates the maximum transmit power of the device.

8. A large-scale language model training device, characterized in that: include: A decomposition module configured to decompose the encoder of the large language model into multiple sub-models; A deployment module, configured to deploy the sub-model on a device of a device cluster and deploy the large language model on a server, wherein the sub-model corresponds to the device in a one-to-one manner; The training module is configured to iteratively train the large language model, wherein in each iterative training, the device cluster and the server process data in a pipeline parallel manner.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute any one of claims 1 to 7.

Images