A large language model training method, system, device and storage medium

Abstract

The present application discloses a large language model training method, system, device and storage medium. The method divides the student model and the large language teacher model and deploys them in a central controller with sufficient resources and on a client with limited resources respectively. This ensures that the local data of the client is only used to train the local network structure, avoiding the risk of local data leakage. In addition, the method trains the network structure with heavy computational load through the central controller, while the client only trains the network structure with light computational load. This adapts to the resource limitations of the client device, allowing the resource-limited client to participate in the model training. Moreover, the knowledge of the teacher model is transferred to the student model through knowledge distillation, ensuring that the data is not leaked and significantly reducing the dependence on the central controller.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large language model training method, system, device and storage medium. Background Art

[0002] At present, artificial intelligence technology is developing rapidly, and the field of natural language processing has ushered in major technological breakthroughs. Compared with traditional models, large language models can achieve better performance in the natural language processing GLUE benchmark test and are widely used in various scenarios in people's lives.

[0003] However, existing large language models directly centralize a large amount of local user data from clients in the central processor for unified training and processing, which poses a risk of exposing users' sensitive information. It also increases dependence on the central processor's computing infrastructure and is not conducive to local deployment of clients. Summary of the Invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The main purpose of the embodiments of the present disclosure is to propose a large language model training method, system, device and storage medium, which can improve the privacy security, deployment efficiency and collaborative generalization of large language model training.

[0006] A first aspect of an embodiment of the present application provides a large language model training method for a central controller, the method comprising:

[0007] Receiving first features and second features respectively sent from multiple clients; the first features are features extracted by the clients from local data based on a first network structure of a student model, and the second features are features extracted by the clients from the local data based on a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model;

[0008] Inputting the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and inputting the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes the model substructures of the at least two model substructures except the first network structure, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model;

[0009] updating the second network structure of the student model according to the first to fourth features, obtaining updated parameters of the second network structure of the student model, and obtaining updated parameters of the first network structure of the student model corresponding to each of the clients, where the updated parameters of the first network structure of the student model are obtained by each of the clients updating the first network structure of the student model according to the first to fourth features;

[0010] If the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

[0011] An embodiment of the present application provides a large language model training method. By splitting the student model and the large language teacher model and deploying them in a resource-rich central controller and a resource-constrained client, respectively, it can ensure that the client's local data is only used to train the local network structure, avoiding the risk of local data leakage. In addition, this method uses the central controller to train the network structure with a large computational load, and the client only trains the network structure with a small computational load. It can adapt to the resource limitations of the client device, so that resource-constrained clients can also participate in model training. Moreover, through knowledge distillation, the knowledge of the teacher model is transferred to the student model, ensuring that the data is not leaked and significantly reducing dependence on the central controller.

[0012] In some embodiments of the present application, before receiving the first feature and the second feature respectively sent from the multiple clients, the method further includes:

[0013] Receiving initial features sent by multiple clients, wherein the initial features are features extracted by the clients from local data according to a first network structure of a teacher model, wherein the teacher model has at least two model substructures, and the first network structure of the teacher model is at least one model substructure of the at least two model substructures;

[0014] Inputting the initial features into the second network structure of the teacher model to obtain intermediate features output by the second network structure of the teacher model; the second network structure of the teacher model is a model substructure of the at least two model substructures other than the first network structure of the teacher model;

[0015] Update the teacher model according to the intermediate features, and obtain update parameters of the first network structure of the teacher model and the second network structure of the teacher model corresponding to each client;

[0016] If the update parameters of the first network structure of the teacher model corresponding to each of the clients and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each of the clients is obtained.

[0017] In some embodiments of the present application, the first network structure of the teacher model includes an embedding layer and a head layer, and the second network structure of the teacher model is a compilation layer; the initial features are features extracted by the client from local data based on the embedding layer of the teacher model; and updating the teacher model based on the intermediate features to obtain updated parameters of the first network structure of the teacher model and updated parameters of the second network structure of the teacher model corresponding to each client include:

[0018] Sending the intermediate features to the multiple clients so that each of the clients inputs the intermediate features into the corresponding head layer of the teacher model to obtain result features output by the head layer of the teacher model, generating first gradient parameters and second gradient parameters of the teacher model according to the result features, updating the embedding layer and the head layer of the teacher model according to the first gradient parameters of the teacher model, and obtaining updated parameters of the embedding layer and the head layer of the teacher model;

[0019] Updating the compilation layer of the teacher model according to the second gradient parameter of the teacher model to obtain an updated parameter of the compilation layer of the teacher model;

[0020] If the update parameters of the first network structure of the teacher model of each client and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained, including:

[0021] If the update parameters of the embedding layer and the head layer of the teacher model of each client and the update parameters of the compilation layer of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained.

[0022] In some embodiments of the present application, the first network structure of the student model includes at least an embedding layer, and the second network structure of the student model includes at least a compilation layer; before updating the second network structure of the student model according to the first to fourth features, the method further includes:

[0023] Calculating a first loss function based on the first to fourth features;

[0024] Calculate the gradient parameters of the embedding layer and the gradient parameters of the compilation layer of the student model according to the first loss function;

[0025] Updating the second network structure of the student model according to the first to fourth features includes:

[0026] Updating the compilation layer of the student model according to the gradient parameters of the compilation layer;

[0027] After obtaining the updated parameters of the second network structure of the student model, the method further includes:

[0028] Sending the gradient parameters of the embedding layer of the student model to the multiple clients, so that each of the clients updates the embedding layer of the student model according to the gradient parameters of the embedding layer, thereby obtaining updated parameters of the embedding layer of the student model;

[0029] If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including:

[0030] If the update parameters of the embedding layer of the student model of each client and the update parameters of the compilation layer of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

[0031] In some embodiments of the present application, the first network structure of the student model further includes a header layer; before obtaining the large language student model generated by the student model corresponding to each client if the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition, the method further includes:

[0032] Obtaining a fifth feature and a sixth feature sent by each of the multiple clients, wherein the fifth feature is a feature extracted by the client from the local data according to the embedded layer of the updated student model, and the sixth feature is a feature extracted by the client from the local data according to the embedded layer of the large language teacher model;

[0033] Inputting the fifth feature into the compilation layer corresponding to the updated student model to obtain a seventh feature output by the compilation layer of the updated student model, and inputting the sixth feature into the compilation layer of the large language teacher model to obtain an eighth feature output by the compilation layer of the large language teacher model;

[0034] Sending the seventh feature and the eighth feature to the multiple clients, so that each of the clients inputs the seventh feature into the head layer of the student model to obtain a ninth feature output by the head layer of the student model, and inputs the eighth feature into the head layer of the large language teacher model to obtain a tenth feature output by the head layer of the large language teacher model, and calculating a second loss function based on the ninth feature and the tenth feature, calculating a first gradient parameter and a second gradient parameter of the student model based on the second loss function, and updating the head layer and the embedding layer of the student model based on the first gradient parameter of the student model to obtain updated parameters of the head layer and the embedding layer of the student model;

[0035] Updating the compilation layer of the student model according to the second gradient parameter of the student model to obtain updated parameters of the compilation layer of the student model;

[0036] If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including:

[0037] If the update parameters of the embedding layer, the update parameters of the head layer, and the update parameters of the compilation layer of the student model of each client meet the end conditions of the first iteration, a large language student model generated by the student model corresponding to each client is obtained.

[0038] In some embodiments of the present application, calculating the first loss function according to the first to fourth features includes:

[0039] Calculating a first mean square error based on the first feature and the second feature;

[0040] Calculating a second mean square error based on the third feature and the fourth feature;

[0041] A first loss function is calculated based on the first mean square error and the second mean square error.

[0042] In some embodiments of the present application, the second loss function is calculated according to the ninth feature and the tenth feature;

[0043] A cross entropy function is calculated according to the ninth feature and the tenth feature, and the cross entropy function is used as the second loss function.

[0044] A second aspect of an embodiment of the present invention provides a large language model training system, the system comprising:

[0045] a receiving module, configured to receive first features and second features respectively transmitted from a plurality of clients; the first features being features extracted by the clients from local data based on a first network structure of a student model, and the second features being features extracted by the clients from the local data based on a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is identical to the first network structure of the large language teacher model;

[0046] a processing module, configured to input the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and input the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes the model substructures of the at least two model substructures except the first network structure, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model;

[0047] an updating module, configured to update the second network structure of the student model according to the first to fourth features, obtain updated parameters of the second network structure of the student model, and obtain updated parameters of the first network structure of the student model corresponding to each of the clients, wherein the updated parameters of the first network structure of the student model are obtained by each of the clients updating the first network structure of the student model according to the first to fourth features;

[0048] The iteration module is used to obtain a large language student model generated by the student model corresponding to each client if the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition.

[0049] A third aspect of an embodiment of the present invention provides an electronic device, comprising: at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the above-mentioned large language model training method.

[0050] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned large language model training method.

[0051] It can be understood that the beneficial effects of the second to fourth aspects compared with the relevant technologies are the same as the beneficial effects of the first aspect compared with the relevant technologies. Please refer to the relevant description in the first aspect and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0053] Figure 1 This is a flow chart of a large language model training method provided in an embodiment of the present application;

[0054] Figure 2 This is a schematic diagram of an implementation method of a large language model training method provided in an embodiment of the present application;

[0055] Figure 3 This is another schematic diagram of an implementation method of a large language model training method provided in an embodiment of the present application;

[0056] Figure 4 This is another schematic diagram of an implementation method of a large language model training method provided in an embodiment of the present application;

[0057] Figure 5 This is another schematic diagram of an implementation method of a large language model training method provided in an embodiment of the present application;

[0058] Figure 6 This is a structural diagram of a large language model training system provided in an embodiment of the present application;

[0059] Figure 7 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application.

[0061] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0062] In the description of this application, it should be understood that descriptions involving orientation, such as the orientation or positional relationship indicated by up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application.

[0063] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technical personnel in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.

[0064] With the rapid development of artificial intelligence technology, the field of natural language processing has seen significant technological breakthroughs, and large-scale language models have become a key driver of this progress. Large-scale language models typically utilize deep neural networks based on the Transformer architecture. They are pre-trained on massive datasets to learn rich linguistic features. However, existing large-scale language models centralize training and processing of large amounts of local user data from clients on the central processing unit (CPU). This poses the risk of exposing sensitive user information, increases reliance on the CPU's computing infrastructure, and hinders local client deployment.

[0065] For this purpose, refer to Figure 1 The embodiment of the present application provides a large language model training method. The method is applied to a central controller. The controller can be a server, an electronic device, or a mobile terminal, etc., which is not specifically limited here. The method includes the following steps S110 to S140:

[0066] Step S110: receiving first features and second features respectively sent from multiple clients; the first feature is a feature extracted by the client from local data based on the first network structure of the student model, and the second feature is a feature extracted by the client from local data based on the first network structure of the large language teacher model, the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model.

[0067] In this step, the large language teacher model is a pre-trained model. One purpose of this embodiment is to transfer the knowledge of the large language teacher model to the student model through knowledge distillation until a large language student model corresponding to each client is obtained.

[0068] Knowledge distillation is a method that promotes the transfer of knowledge from a large-scale teacher model to a small-scale student model. By transferring the knowledge of a large teacher network to a small student network, the student network can approach the performance of the teacher network while maintaining a small size.

[0069] The following explains the specific training steps of the teacher model:

[0070] Step S210, receiving initial features sent by multiple clients respectively; the initial features are features extracted by the client from local data based on the first network structure of the teacher model, the teacher model has at least two model substructures, and the first network structure of the teacher model is at least one model substructure of the at least two model substructures.

[0071] Step S220: input the initial features into the second network structure of the teacher model to obtain the intermediate features output by the second network structure of the teacher model; the second network structure of the teacher model is a model substructure in at least two model substructures other than the first network structure of the teacher model.

[0072] Step S230: Update the teacher model according to the intermediate features to obtain update parameters of the first network structure of the teacher model and update parameters of the second network structure of the teacher model corresponding to each client.

[0073] Step S240: If the update parameters of the first network structure of the teacher model corresponding to each client and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained.

[0074] First, the teacher model is divided, and the several model substructures included in the teacher model are set in the client and the central controller respectively. The division here is mainly based on the complexity of the network. If the model substructure is more complex, the data processing is more complex and the required computing performance is higher. Therefore, the model substructure with high complexity (that is, the second network structure of the teacher model) can be set in the central controller, and the model substructure with low complexity (that is, the first network structure of the teacher model) can be set on the client.

[0075] For example, if the teacher model is a BERT (Bidirectional Encoder Representations from Transformers) model, the BERT model can be divided into an Embedding layer, a Transformer layer, and a Head layer. The Embedding layer and the Head layer are set on the client, and the Transformer layer is placed on the central controller.

[0076] The client can be any computing device that needs to perform tasks using a large language model, and is not limited here.

[0077] This embodiment provides a training process in multiple iterations of the teacher model. The client extracts initial features from local data based on the first network structure of the teacher model. The central controller then receives the initial features sent by multiple clients. The central controller then inputs the initial features into the second network structure of the teacher model, obtaining intermediate features output by the second network structure of the teacher model. The intermediate features are then sent to multiple clients, so that each client inputs the intermediate features into the head layer of the corresponding teacher model, obtaining the result features output by the head layer of the teacher model. The first gradient parameters and second gradient parameters of the teacher model are generated based on the result features. The first network structure of the teacher model is updated based on the first gradient parameters of the teacher model to obtain the updated parameters of the first network structure of the teacher model. The second network structure of the teacher model is updated based on the second gradient parameters of the teacher model to obtain the updated parameters of the second network structure of the teacher model. The central controller then obtains the updated parameters of the first network structure of the teacher model corresponding to each client. Finally, the central controller aggregates the parameters. If the updated parameters of the first network structure of the teacher model corresponding to each client and the updated parameters of the second network structure of the teacher model update each client meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained.

[0078] It should be noted that if the iteration end condition is met, the first network structure trained in the client and the second network structure trained by the central controller together form a large language teacher model. For example, when in use, the first network structure of the client extracts features from the data, and then the second network structure of the central controller processes the features to finally obtain the output result.

[0079] This embodiment is a training process of a teacher model. In this process, the teacher model is divided and deployed in a central controller with sufficient resources and on a client with limited resources, respectively. This ensures that the local data of the client is only used to train the local network structure, avoiding the risk of local data leakage. In addition, the central controller trains the network structure with a large computational load, and the client only trains the network structure with a small computational load. This can adapt to the resource limitations of the client device and distribute the complex training tasks of large-scale models to the server side, effectively reducing the computing and storage pressure on the client, allowing resource-limited clients to participate in the training of the model, and significantly reducing dependence on the central controller.

[0080] In some embodiments, before obtaining the update parameters of the first network structure of the teacher model corresponding to each client in step S230, the method further includes the following steps S310 to S340:

[0081] Step S310: Send the intermediate features to the multiple clients so that each of the clients inputs the intermediate features into the corresponding head layer of the teacher model to obtain the result features output by the head layer of the teacher model, and generate the first gradient parameters and the second gradient parameters of the teacher model according to the result features, update the embedding layer and the head layer of the teacher model according to the first gradient parameters of the teacher model, and obtain the updated parameters of the embedding layer and the head layer of the teacher model.

[0082] Step S320: Update the compilation layer of the teacher model according to the second gradient parameter of the teacher model to obtain the updated parameter of the compilation layer of the teacher model.

[0083] Step S330: If the update parameters of the embedding layer and the head layer of the teacher model of each client and the update parameters of the compilation layer of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained.

[0084] In this step, the teacher model includes a first network structure including an embedding layer and a head layer and a second network structure including a compilation layer. The teacher model is trained by receiving initial features extracted from local data inputted into the embedding layer of the corresponding teacher model from each client.

[0085] In this embodiment, the central controller first receives initial features sent by multiple clients respectively, wherein the initial features are features extracted by the clients from local data according to the embedding layer of the teacher model.

[0086] The initial features are then input into the compilation layer of the teacher model to obtain the intermediate features output by the compilation layer of the teacher model, and then the intermediate features are sent to multiple corresponding clients.

[0087] The client inputs the intermediate features into the head layer of the corresponding teacher model, obtains the result features output by the head layer of the teacher model, and generates a first gradient parameter and a second gradient parameter based on the result features output by the head layer of the teacher model, and then updates the embedding layer and the head layer of the teacher model according to the first gradient parameter to obtain the updated parameters of the embedding layer and the head layer of the teacher model, wherein the first gradient parameter includes the gradient parameters of the embedding layer and the head layer of the corresponding teacher model, and the second gradient parameter is the gradient parameter of the compilation layer.

[0088] The central controller updates the compilation layer of the teacher model using the second gradient parameter to obtain an updated parameter of the compilation layer of the teacher model.

[0089] Finally, if the update parameters of the embedding layer and head layer of each client's teacher model and the update parameters of the compilation layer of the central controller's teacher model meet the second iteration end conditions, a large language teacher model generated by the teacher model corresponding to each client is obtained.

[0090] For ease of understanding, the following implementation is provided by taking the central controller as a server as an example:

[0091] Combine Figure 2 、 Figure 3 and Figure 4 As shown, each task contains a certain number of users, for example, task A contains N clients Each client has its own local dataset

[0092] The specific large language teacher model is the BERT model, and its corresponding training process includes:

[0093] First, the teacher model is divided into three key parts: embedding layer, head layer and compilation layer, respectively. A 、H A and T A At the same time, the initialized embedding layer and head layer are deployed on the client side to handle feature extraction and embedding generation of local data; the compilation layer is deployed on the server side to handle more complex feature conversion and high-level semantic understanding.

[0094] Assume that the data input is X. During the forward propagation process, the kth client C kThrough the client's embedding layer Processing to achieve feature extraction and generate embedding representation These representations are then transmitted to the server, which receives the client C k After the embedding representation, further calculation and context modeling are performed through the compilation layer to generate the final prediction result

[0095] During the backpropagation process, the server first calculates the gradient parameters and updates the teacher model parameters The gradient parameters are then sent back to each client, and each client receives the gradient parameters and the output of the first layer. Update the parameters of the head layer and embedding layer of the teacher model. To ensure model consistency, the server can periodically aggregate the model parameters of each device and synchronously update the large language teacher model. The expression for the model parameter aggregation is as follows:

[0096]

[0097] Where D is the total number of samples in all N clients. The outputs of three key layers are obtained through the aggregation process: E A represents the parameters of the embedding layer, T A Represents the parameters of the compilation layer, H A The parameters of the head layer are repeatedly aggregated and updated until the model converges, thereby optimizing the allocation of computing resources and improving the efficiency and effectiveness of model training. Finally, the network architecture of the large language teacher model is rebuilt through the aggregated output parameters, and the final prediction output F(X) of the large language teacher model for the input data X is calculated and obtained:

[0098] F(X)=H A (T A (E A (X)))

[0099] In this embodiment, the teacher model BERT is divided into three key parts: the embedding layer, the header layer, and the compilation layer, which are respectively deployed in a central controller with sufficient resources and on a client with limited resources. This ensures that the client's local data is only used to train the local network structure, avoiding the risk of local data leakage. In addition, the central controller trains the network structure with a large computational load, and the client only trains the network structure with a small computational load. This can adapt to the resource limitations of the client device, allowing resource-limited clients to participate in the training of the model.

[0100] In step S110, the structure of the student model is consistent with that of the large language teacher model. That is, the first network structure of the student model is consistent with the first network structure of the large language teacher model, and the second network structure of the student model is consistent with the second network structure of the large language teacher model. If the first network structure of the large language teacher model is set on the client, then the first network structure of the student model is also set on the client. If the second network structure of the large language teacher model is set on the central controller, then the second network structure of the student model is also set on the central controller.

[0101] Similar to the teacher model training process, the central controller receives first and second features from multiple clients. The first feature is extracted from local data by the client based on the first network structure of the student model, while the second feature is extracted from local data by the client based on the first network structure of the large language teacher model. Because this embodiment addresses knowledge transfer to the student model, both the features extracted from local data by the first network structure of the student model and the features extracted from local data by the first network structure of the large language teacher model are required.

[0102] Step S120: input the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and input the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes a model substructure other than the first network structure in at least two model substructures, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model.

[0103] Similar to the training process of the teacher model, in this step, the central controller inputs the first feature and the second feature into the second network structure of the student model and the second network structure of the large language teacher model set in the central controller respectively, obtains the third feature output by the second network structure of the student model, and obtains the fourth feature output by the second network structure of the large language teacher model.

[0104] Since the local data is directly input into the first network structure corresponding to the local client, the extraction of data features is achieved and the leakage of local data is avoided.

[0105] Step S130: Update the second network structure of the student model according to the first to fourth features, obtain the updated parameters of the second network structure of the student model, and obtain the updated parameters of the first network structure of the student model corresponding to each client. The updated parameters of the first network structure of the student model are obtained by each client updating the first network structure of the student model according to the first to fourth features.

[0106] Similar to the teacher model training process, this embodiment updates the second network structure of the student model based on the first to fourth features, obtains updated parameters for the second network structure of the student model, and then obtains updated parameters for the first network structure of the student model corresponding to each client. The obtained parameters are used to determine the end of subsequent iterations.

[0107] Step S140: If the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

[0108] In this step, whether the first iteration end condition is met is determined based on the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model. When the first iteration end condition is met, a large language student model generated by the student model corresponding to each client is obtained.

[0109] It should be noted that in steps S110 to S140 , the large language teacher model is a trained model and does not need to be updated.

[0110] This method uses a central controller to train network structures with heavy computational loads, while the client only trains network structures with light computational loads. This method can adapt to the resource limitations of client devices, allowing resource-constrained clients to participate in model training. Furthermore, through knowledge distillation, the knowledge of the teacher model is transferred to the student model, ensuring that data is not leaked and significantly reducing dependence on the central controller.

[0111] In some embodiments of the present application, the first network structure of the student model includes at least an embedding layer, and the second network structure of the student model includes at least a compilation layer; before updating the second network structure of the student model according to the first to fourth features in step S130, the method further includes:

[0112] The updating of the teacher model according to the intermediate features to obtain update parameters of the first network structure of the teacher model and the second network structure of the teacher model corresponding to each client includes the following steps S410 to S450:

[0113] Step S410: Calculate a first loss function based on the first to fourth features.

[0114] Step S420: Calculate the gradient parameters of the embedding layer and the gradient parameters of the compilation layer of the student model according to the first loss function.

[0115] Updating the second network structure of the student model according to the first to fourth features in step S130 includes:

[0116] Step S430: Update the compilation layer of the student model according to the gradient parameters of the compilation layer.

[0117] After obtaining the updated parameters of the second network structure of the student model in step S130, the method further includes:

[0118] Step S440: Send the gradient parameters of the embedding layer of the student model to multiple clients, so that each client updates the embedding layer of the student model according to the gradient parameters of the embedding layer, and obtains the updated parameters of the embedding layer of the student model.

[0119] If the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including:

[0120] Step S450: If the update parameters of the embedding layer of the student model of each client and the update parameters of the compilation layer of the student model meet the end condition of the first iteration, a large language student model generated by the student model corresponding to each client is obtained.

[0121] In this embodiment, the student model has at least an embedding layer, which ensures that each client's local data is not leaked. After obtaining the first to fourth features, a first loss function is calculated based on the first to fourth features. Then, the gradient parameters of the embedding layer and the gradient parameters of the compilation layer of the student model are calculated based on the first loss function. These gradient parameters are then used to update the first and second network structures, thereby determining whether the iteration termination condition is met.

[0122] In some embodiments of the present application, the first network structure of the student model further includes a head layer; and before obtaining the large language student model generated by the student model corresponding to each client if the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition in step S140, the method further includes the following steps S510 to S550:

[0123] Step S510: Obtain the fifth feature and the sixth feature sent by each of the multiple clients, where the fifth feature is a feature extracted by the client from the local data based on the embedding layer of the updated student model, and the sixth feature is a feature extracted by the client from the local data based on the embedding layer of the large language teacher model.

[0124] Step S520: Input the fifth feature into the compilation layer corresponding to the updated student model to obtain the seventh feature output by the compilation layer of the updated student model, and input the sixth feature into the compilation layer of the large language teacher model to obtain the eighth feature output by the compilation layer of the large language teacher model.

[0125] Step S530: Send the seventh feature and the eighth feature to multiple clients, so that each client inputs the seventh feature into the head layer of the student model to obtain the ninth feature output by the head layer of the student model, and inputs the eighth feature into the head layer of the large language teacher model to obtain the tenth feature output by the head layer of the large language teacher model, and calculates the second loss function based on the ninth feature and the tenth feature, calculates the first gradient parameter and the second gradient parameter of the student model based on the second loss function, updates the head layer and the embedding layer of the student model based on the first gradient parameter of the student model, and obtains the updated parameters of the head layer and the embedding layer of the student model.

[0126] Step S540: Update the compilation layer of the student model according to the second gradient parameter of the student model to obtain the updated parameter of the compilation layer of the student model.

[0127] If the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including:

[0128] Step S550: If the update parameters of the embedding layer, the update parameters of the head layer, and the update parameters of the compilation layer of the student model of each client meet the end conditions of the first iteration, a large language student model generated by the student model corresponding to each client is obtained.

[0129] For ease of understanding, the following implementation is provided by taking the central controller as a server as an example:

[0130] Combine Figure 2 、 Figure 3 and Figure 4As shown, the student model is initialized on the client k, and the same model structure as the large language teacher model in the above embodiment is adopted, that is, the large language teacher model and the student model (both BERT models) are divided into an embedding layer, a transformer layer, and a head layer. According to the different complexity of the model substructure, the compilation layer with higher complexity than the embedding layer and the head layer is deployed on the server side. Each client uses its own local data set to independently train the embedding layer and the head layer of the corresponding student model, i.e., the sub-model. The server side receives the forward propagation and back propagation gradient parameters from the client and updates the compilation layer accordingly. Therefore, the student model of each client has the same structure as the teacher model, and is composed of an embedding layer and a head layer deployed on the client and a compilation layer on the server. Then, the student model is trained according to the trained large language teacher model to obtain a large language student model, which specifically includes the embedding layer and the compilation layer of the trained student model and the head layer of the trained student model:

[0131] The following first introduces the training of the student model embedding layer and compilation layer;

[0132] First, on client k, the local data D k Input into the embedding layer of the large language teacher model and the student model to obtain the embedding layer output vector T of the large language teacher model e , and the embedding layer output vector T of the student model e , the two vectors T e and T e Sent to the server S respectively.

[0133] The server S then receives the embedding layer output vector T of the large language teacher model from the client e , and the embedding layer output vector S of the student model e Finally, these output vectors are input into the embedding layers of the large language teacher model and the student model respectively to obtain the output vector T of the embedding layer of the large language teacher model t and the output vector S of the embedding layer of the student model t .

[0134] Finally, calculate the two embedding layer output vectors T e 、S e The mean squared error (MSE) between embedding , whose expression is:

[0135]

[0136] where d e The dimension of the embedding layer output vectors for the large language teacher model and the student model, T e(j) and S e (j) is the value of the jth element of the output vector. Calculate the two coding layer output vectors T t 、S t The mean square error between the two is used to determine L transformer .

[0137]

[0138] where d t The dimension of the output vector of the compilation layer of the large language teacher model and the student model, T t (j) and S t (j) is the corresponding value of the j-th element of the output vector.

[0139] The first loss function of the student model is calculated based on the mean square error of the embedding layer of the student model and the mean square error of the compilation layer of the student model, and its expression is:

[0140] L=L embedding +L transformer

[0141] Finally, the gradient parameters G of the embedding layers of the student model are calculated according to the first loss function of the student model. e and the gradient parameter G of the compilation layer t , and G e Sent to the client, each client according to G e Update the parameters of the embedding layer of the student model, and the server side will t Update the parameters of the compilation layer of the student model and then train the head layer of the student model.

[0142] The following introduces the training of the first layer of the student model;

[0143] On client k, local data D k Input into the embedding layer of the large language teacher model and the updated student model to obtain the embedding layer output vector T of the large language teacher model e , and the embedded layer output vector S of the updated student model e .

[0144] The embedding layer output vector T of the large language teacher model e And the embedding layer output vector S of the updated student model e Sent to the server S, which receives the embedding layer output vector T of the large language teacher model from the client e , and the embedded layer output vector S of the updated student model e Finally, these vectors are input into the compilation layer of the large language teacher model and the updated student model respectively to obtain the output vector T of the compilation layer of the large language teacher modelt And the output vector S of the compilation layer of the updated student model t .

[0145] Then the output vector T of the compilation layer of the large language teacher model is t And the output vector S of the compilation layer of the updated student model t Send to client k, client k will receive the output vector T of the compilation layer of the large language teacher model t And the output vector S of the compilation layer of the updated student model t Input them into the head layer of the large language teacher model and the updated student model respectively to obtain the output vector T of the head layer of the large language teacher model h And the output vector S of the head layer of the updated student model h , calculate T h and S h The cross entropy loss function between logits , which is expressed as follows:

[0146]

[0147] where d h is the output vector dimension of the head layer of the large language teacher model and the updated student model, T h (j) and S h (j) is the corresponding value of the j-th element of the output vector.

[0148] Thus, the second loss function of the updated student model is calculated. The expression of the second loss function L is as follows:

[0149] L=L logits

[0150] Finally, client k calculates the gradient parameter G of the head layer of the updated student model according to the second loss function h , and update the parameters of the head layer of the updated student model according to the update.

[0151] A set of experimental examples is provided below:

[0152] Instantiate a smaller BERT base The student model (layers l = 4, hidden layer dimension d = 312, feedforward size di = 1200, head layer number h = 12) has a total of 14.7M parameters. In comparison, the teacher model used is a standard BERT baseThe model has l = 12 layers, hidden layer dimension d = 768, feedforward network size di = 3072, and h = 12 head layers, for a total of 110M parameters. It can be seen that the number of parameters in the student model is significantly reduced compared to the teacher model, accounting for only 13.4% of the teacher model's parameters.

[0153] (1) Implement the training of the teacher model.

[0154] During this phase, a teacher model was used for fine-tuning. In a federated learning simulation environment, five clients were trained, with 10 global training rounds and one local training round per client. The learning rate was set to lr = 5e-5, and the number of training samples per batch was set to batchsize = 32. The FedAvg method was used for model aggregation. During training, the five clients independently completed their training tasks before the model was aggregated. The training dataset was randomly and evenly divided into five parts, with each client using one of the parts for training.

[0155] (2) Realize the distillation of the student model.

[0156] The training of the student model is divided into two steps. First, when training the Embedding layer and the Transformer layer, the Transformer layer of the student model is trained from the BERT of the teacher model. base Knowledge distillation is performed every three layers of the Transformer layer in [1]. This step is trained for a fixed number of 10 epochs, with a learning rate of lr = 5e-5. Secondly, when training the Head layer, the learning rate is adjusted to lr = 3e-5, and the number of training epochs is adjusted based on the specific task requirements. The training batch size for both steps is set to 32.

[0157] In addition, the following three benchmark schemes are considered for performance comparison:

[0158] 1) BERT base : Using pre-trained BERT base The model is tested.

[0159] 2) FedBERT: Tested using the BERT teacher model after fine-tuning in the first stage.

[0160] 3) Private Model: Deploy an initialized student model on each edge device and train it on its own private data to obtain a device-side private student model for testing. This part uses the student model structure.

[0161] The same data set partitioning mode as the teacher model training is adopted to ensure that the private data of each client is the same as that of the teacher training.

[0162] To comprehensively evaluate the performance of the model, we selected the General Language Understanding Evaluation (GLUE) benchmark as the test platform. The GLUE benchmark is a comprehensive collection of various natural language understanding tasks that provides a standardized testing environment for the model's various language processing capabilities.

[0163] In this study, three specific downstream tasks from the GLUE benchmark were selected for experimental verification. These tasks include:

[0164] SST-2 (Single Sentence Sentiment Analysis): In this task, the goal is to classify an input sentence as positive or negative based on its sentiment. If a sentence expresses positive sentiment, it is classified and labeled as 1; if it expresses negative sentiment, it is classified and labeled as 0. This task has 67,350 training examples, 873 validation examples, and 1,821 test examples. Model performance is evaluated using accuracy.

[0165] CoLA (Sentence Acceptability Analysis): This task aims to determine the grammatical validity of a sentence structure based on the input sentence. As a binary classification task, 0 represents a grammatically unacceptable sentence, while 1 represents a grammatically acceptable sentence. This task consists of 8,551 training examples, 1,043 validation examples, and 1,063 test examples. Model performance is evaluated using the Matthews correlation coefficient.

[0166] MRPC (Semantic Similarity Task): In this task, the goal is to evaluate the semantic equivalence of sentence pairs, aiming to deeply understand and identify whether two sentences express the same or similar meanings at the semantic level. In this task, 0 and 1 are used as labels, 0 represents that the sentence pair is semantically dissimilar, and 1 represents that the sentence pair is semantically similar. The dataset used for this task contains 5,800 pairs of sentences, which are extracted from online news sources and accompanied by human annotations to clearly indicate whether each pair of sentences has a paraphrase or semantic equivalence relationship. Due to the long sentence length in the sample and the uneven category distribution of the dataset, positive samples account for 68% and negative samples account for 32%. In order to more accurately measure the performance of the model, accuracy is used as the main evaluation indicator, which directly reflects the proportion of correct classifications by the model.

[0167] Because some GLUE tasks have limited training data, we expanded the dataset using data augmentation methods from Huawei's Tinybert. Through detailed experiments on these tasks, we gained a deeper understanding of the model's performance in different language understanding and reasoning scenarios, ensuring that the model's evaluation results are highly representative and accurate.

[0168] The model performance and baseline model performance were tested on the SST-2, CoLA, and MRPC tasks in the GLUE task, as shown in Table 1.

[0169] Table 1

[0170]

[0171] BERT base The large number of parameters makes it difficult to use in resource-constrained environments.

[0172] The FedBERT model is large in size and difficult to deploy on the local client after training.

[0173] The private model (PrivateModel) has a small number of parameters and is a small-scale model trained on local data. Its performance on all three tasks is significantly lower than that of the other three models, and its performance is poor.

[0174] Therefore, while maintaining the same small number of parameters, the student model in this application significantly improves performance, particularly on the SST-2 and CoLA tasks, far outperforming private models. This result demonstrates that the student model can not only be effectively trained and deployed in resource-constrained environments, but also achieves excellent task performance, demonstrating its advantages and applicability in real-world applications.

[0175] Combine Figure 5As shown, some embodiments of the present application can provide different types of task training sets and local data through different clients to simultaneously train multiple large language teacher models for different tasks. For example, a large language model corresponding to task A is trained for the teacher model, which is used to train the student model that performs task A. The student model that performs task A learns the large language teacher model for task A through knowledge distillation. This embodiment divides the student model and the large language teacher model and deploys them on a server with sufficient resources and a client with limited resources, respectively. This ensures that the client's local data is only used to train the local network structure, avoiding the risk of local data leakage. In addition, this method trains the network structure with a large computational load through the server, and the client only trains the network structure with a small computational load. This can adapt to the resource limitations of the client device, allowing resource-limited clients to participate in model training. Moreover, through knowledge distillation, the knowledge of the teacher model is transferred to the student model, ensuring that the data is not leaked and significantly reducing dependence on the central controller.

[0176] like Figure 6 As shown, some embodiments of the present application provide a large language model training system, which includes a receiving module 810, a processing module 820, an updating module 830, and an iteration module 840. Specifically:

[0177] A receiving module 810 is configured to receive first features and second features respectively transmitted from a plurality of clients; the first features are features extracted by the client from local data based on a first network structure of a student model, and the second features are features extracted by the client from local data based on a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is identical to the first network structure of the large language teacher model;

[0178] a processing module 820 configured to input the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and input the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes a model substructure other than the first network structure in at least two model substructures, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model;

[0179] An updating module 830 is configured to update the second network structure of the student model according to the first to fourth features, obtain updated parameters of the second network structure of the student model, and obtain updated parameters of the first network structure of the student model corresponding to each client, where the updated parameters of the first network structure of the student model are obtained by each client updating the first network structure of the student model according to the first to fourth features;

[0180] The iteration module 840 is used to obtain a large language student model generated by the student model corresponding to each client if the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition.

[0181] It should be noted that the large language model training system provided in this embodiment and the above-mentioned large language model training system method are based on the same inventive concept. Therefore, the relevant content of the above-mentioned large language model training method is also applicable to the content of the large language model training system. Therefore, it will not be repeated here.

[0182] like Figure 7 , an embodiment of the present application further provides an electronic device, the electronic device comprising:

[0183] At least one hydrogen fuel cell;

[0184] at least one memory;

[0185] at least one processor;

[0186] at least one program;

[0187] The program is stored in the memory, and the processor executes at least one program to implement the large language model training method described above in the present disclosure.

[0188] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0189] The electronic device according to the embodiment of the present application is described in detail below.

[0190] The processor 1600 may be implemented as a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present disclosure.

[0191] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called by the processor 1600 to execute a large language model training method of the embodiments of the present disclosure.

[0192] Input / output interface 1800, used for information input and output;

[0193] Communication interface 1900, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0194] bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0195] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0196] An embodiment of the present disclosure also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned large language model training method.

[0197] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0198] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0199] Those skilled in the art will understand that the technical solutions shown in the drawings do not constitute a limitation on the embodiments of the present disclosure, and may include more or fewer steps than shown in the drawings, or a combination of certain steps, or different steps.

[0200] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0201] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0202] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0203] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0204] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0205] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0206] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0207] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0208] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

[0209] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A large language model training method, characterized in that: For a central controller, the method includes: Receiving first features and second features respectively sent from multiple clients; the first features are features extracted by the clients from local data based on a first network structure of a student model, and the second features are features extracted by the clients from the local data based on a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model; Inputting the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and inputting the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes the model substructures of the at least two model substructures except the first network structure, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model; updating the second network structure of the student model according to the first to fourth features, obtaining updated parameters of the second network structure of the student model, and obtaining updated parameters of the first network structure of the student model corresponding to each of the clients, where the updated parameters of the first network structure of the student model are obtained by each of the clients updating the first network structure of the student model according to the first to fourth features; If the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained; The second network structure of the student model is deployed on the central controller, and the clients independently train the first network structure of the corresponding student model using their respective local data sets. The central controller receives the gradient parameters of forward propagation and backward propagation from the clients and uses them to update the second network structure.

2. The large language model training method according to claim 1, characterized in that Before receiving the first feature and the second feature respectively sent from the plurality of clients, the method further includes: Receiving initial features sent by multiple clients, wherein the initial features are features extracted by the clients from local data according to a first network structure of a teacher model, wherein the teacher model has at least two model substructures, and the first network structure of the teacher model is at least one model substructure of the at least two model substructures; Inputting the initial features into the second network structure of the teacher model to obtain intermediate features output by the second network structure of the teacher model; the second network structure of the teacher model is a model substructure of the at least two model substructures other than the first network structure of the teacher model; Update the teacher model according to the intermediate features, and obtain update parameters of the first network structure of the teacher model and the second network structure of the teacher model corresponding to each client; If the update parameters of the first network structure of the teacher model corresponding to each of the clients and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each of the clients is obtained.

3. The large language model training method according to claim 2, characterized in that The first network structure of the teacher model includes an embedding layer and a head layer, and the second network structure of the teacher model is a compilation layer; the initial features are features extracted by the client from local data based on the embedding layer of the teacher model; and updating the teacher model based on the intermediate features to obtain updated parameters of the first network structure of the teacher model and updated parameters of the second network structure of the teacher model corresponding to each client, including: Sending the intermediate features to the multiple clients so that each of the clients inputs the intermediate features into the corresponding head layer of the teacher model to obtain result features output by the head layer of the teacher model, generating first gradient parameters and second gradient parameters of the teacher model according to the result features, updating the embedding layer and the head layer of the teacher model according to the first gradient parameters of the teacher model, and obtaining updated parameters of the embedding layer and the head layer of the teacher model; Updating the compilation layer of the teacher model according to the second gradient parameter of the teacher model to obtain an updated parameter of the compilation layer of the teacher model; If the update parameters of the first network structure of the teacher model of each client and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained, including: If the update parameters of the embedding layer and the head layer of the teacher model of each client and the update parameters of the compilation layer of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained.

4. The large language model training method according to claim 1, characterized in that The first network structure of the student model includes at least an embedding layer, and the second network structure of the student model includes at least a compilation layer; before updating the second network structure of the student model according to the first to fourth features, the method further includes: Calculating a first loss function based on the first to fourth features; Calculate the gradient parameters of the embedding layer and the gradient parameters of the compilation layer of the student model according to the first loss function; Updating the second network structure of the student model according to the first to fourth features includes: Updating the compilation layer of the student model according to the gradient parameters of the compilation layer; After obtaining the updated parameters of the second network structure of the student model, the method further includes: Sending the gradient parameters of the embedding layer of the student model to the multiple clients, so that each of the clients updates the embedding layer of the student model according to the gradient parameters of the embedding layer, thereby obtaining updated parameters of the embedding layer of the student model; If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including: If the update parameters of the embedding layer of the student model of each client and the update parameters of the compilation layer of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

5. The large language model training method according to claim 4, characterized in that The first network structure of the student model further includes a head layer; before obtaining a large language student model generated by the student model corresponding to each client if the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet the first iteration end condition, the method further includes: Obtaining a fifth feature and a sixth feature sent by each of the multiple clients, wherein the fifth feature is a feature extracted by the client from the local data according to the embedded layer of the updated student model, and the sixth feature is a feature extracted by the client from the local data according to the embedded layer of the large language teacher model; Inputting the fifth feature into the compilation layer corresponding to the updated student model to obtain a seventh feature output by the compilation layer of the updated student model, and inputting the sixth feature into the compilation layer of the large language teacher model to obtain an eighth feature output by the compilation layer of the large language teacher model; Sending the seventh feature and the eighth feature to the multiple clients, so that each of the clients inputs the seventh feature into the head layer of the student model to obtain a ninth feature output by the head layer of the student model, and inputs the eighth feature into the head layer of the large language teacher model to obtain a tenth feature output by the head layer of the large language teacher model, and calculating a second loss function based on the ninth feature and the tenth feature, calculating a first gradient parameter and a second gradient parameter of the student model based on the second loss function, and updating the head layer and the embedding layer of the student model based on the first gradient parameter of the student model to obtain updated parameters of the head layer and the embedding layer of the student model; Updating the compilation layer of the student model according to the second gradient parameter of the student model to obtain updated parameters of the compilation layer of the student model; If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including: If the update parameters of the embedding layer, the update parameters of the head layer, and the update parameters of the compilation layer of the student model of each client meet the end conditions of the first iteration, a large language student model generated by the student model corresponding to each client is obtained.

6. The large language model training method according to claim 4, characterized in that The calculating a first loss function according to the first to fourth features includes: Calculating a first mean square error based on the first feature and the second feature; Calculating a second mean square error based on the third feature and the fourth feature; A first loss function is calculated based on the first mean square error and the second mean square error.

7. The large language model training method according to claim 5, characterized in that: calculating a second loss function based on the ninth feature and the tenth feature; A cross entropy function is calculated according to the ninth feature and the tenth feature, and the cross entropy function is used as the second loss function.

8. A large language model training system, characterized in that: The system comprises: a receiving module, configured to receive first features and second features respectively transmitted from a plurality of clients; the first features being features extracted by the clients from local data based on a first network structure of a student model, and the second features being features extracted by the clients from the local data based on a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is identical to the first network structure of the large language teacher model; a processing module, configured to input the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and input the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes the model substructures of the at least two model substructures except the first network structure, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model; an updating module, configured to update the second network structure of the student model according to the first to fourth features, obtain updated parameters of the second network structure of the student model, and obtain updated parameters of the first network structure of the student model corresponding to each of the clients, wherein the updated parameters of the first network structure of the student model are obtained by each of the clients updating the first network structure of the student model according to the first to fourth features; an iteration module, configured to obtain a large language student model generated by the student model corresponding to each client if the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model meet a first iteration end condition; The second network structure of the student model is deployed on the central controller, and the clients independently train the first network structure of the corresponding student model using their respective local data sets. The central controller receives the gradient parameters of forward propagation and backward propagation from the clients and uses them to update the second network structure.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute a large language model training method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a large language model training method according to any one of claims 1 to 7.

Images