Large language model training method, system and device and storage medium

Abstract

The invention discloses a big language model training method, system and device and a storage medium, and the method comprises the steps: segmenting a student model and a big language teacher model, and respectively deploying the student model and the big language teacher model in a central controller with sufficient resources and a client with limited resources, thereby guaranteeing that the local data of the client only carries out the training of a local network structure, and improving the training efficiency. In addition, according to the method, the network structure with the large calculation load is trained through the central controller, the client side only trains the network structure with the small calculation load, the method can adapt to the resource limitation of client side equipment, the client side with the limited resources can participate in model training, and the user experience is improved. And knowledge of the teacher model is transmitted to the student model through a knowledge distillation mode, so that data is ensured not to be leaked, and dependence on a central controller can be remarkably reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large language model training method, system, device and storage medium. Background Art

[0002] At present, artificial intelligence technology is developing rapidly, and the field of natural language processing has ushered in major technological breakthroughs. Compared with traditional models, large language models can achieve better performance in the natural language processing GLUE benchmark test, and are widely used in various scenarios in people's lives.

[0003] However, the existing large language models directly centralize the local user data of a large number of clients in the central processor for unified training and processing, which poses a risk of exposing users' sensitive information, increases dependence on the central processor's computing infrastructure, and is not conducive to local deployment of clients. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The main purpose of the embodiments of the present disclosure is to propose a large language model training method, system, device and storage medium, which can improve the privacy security, deployment efficiency and collaborative generalization of large language model training.

[0006] A first aspect of an embodiment of the present application provides a large language model training method for a central controller, the method comprising:

[0007] Receiving first features and second features respectively sent from a plurality of clients; the first features are features extracted by the client from local data according to a first network structure of a student model, and the second features are features extracted by the client from the local data according to a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model;

[0008] Inputting the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and inputting the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes the model substructures of the at least two model substructures except the first network structure, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model;

[0009] The second network structure of the student model is updated according to the first to fourth features, and update parameters of the second network structure of the student model are obtained, and the update parameters of the first network structure of the student model corresponding to each of the clients are obtained by each of the clients updating the first network structure of the student model according to the first to fourth features;

[0010] If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

[0011] The embodiment of the present application provides a large language model training method. By dividing the student model and the large language teacher model and deploying them in a central controller with sufficient resources and on a client with limited resources respectively, it can ensure that the local data of the client is only used to train the local network structure, avoiding the risk of local data leakage. In addition, the method trains the network structure with a large computing load through the central controller, and the client only trains the network structure with a small computing load. It can adapt to the resource limitations of the client device, so that the client with limited resources can also participate in the training of the model. Moreover, through the knowledge distillation method, the knowledge of the teacher model is transferred to the student model, which ensures that the data is not leaked and can significantly reduce the dependence on the central controller.

[0012] In some embodiments of the present application, before receiving the first feature and the second feature respectively sent from the multiple clients, the method further includes:

[0013] Receiving initial features sent by multiple clients respectively; the initial features are features extracted by the clients from local data according to a first network structure of a teacher model, the teacher model has at least two model substructures, and the first network structure of the teacher model is at least one model substructure of the at least two model substructures;

[0014] Inputting the initial features into the second network structure of the teacher model to obtain intermediate features output by the second network structure of the teacher model; the second network structure of the teacher model is a model substructure of the at least two model substructures except the first network structure of the teacher model;

[0015] Update the teacher model according to the intermediate features to obtain update parameters of the first network structure of the teacher model and the second network structure of the teacher model corresponding to each of the clients;

[0016] If the update parameters of the first network structure of the teacher model corresponding to each of the clients and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each of the clients is obtained.

[0017] In some embodiments of the present application, the first network structure of the teacher model includes an embedding layer and a head layer, and the second network structure of the teacher model is a compilation layer; the initial feature is a feature extracted by the client from the local data according to the embedding layer of the teacher model; the updating of the teacher model according to the intermediate feature to obtain the update parameters of the first network structure of the teacher model corresponding to each of the clients and the update parameters of the second network structure of the teacher model include:

[0018] Sending the intermediate features to the multiple clients, so that each of the clients inputs the intermediate features into the corresponding head layer of the teacher model, obtains the result features output by the head layer of the teacher model, generates the first gradient parameter and the second gradient parameter of the teacher model according to the result features, updates the embedding layer and the head layer of the teacher model according to the first gradient parameter of the teacher model, and obtains the updated parameters of the embedding layer and the head layer of the teacher model;

[0019] Updating the compilation layer of the teacher model according to the second gradient parameter of the teacher model to obtain an updated parameter of the compilation layer of the teacher model;

[0020] If the update parameters of the first network structure of the teacher model of each client and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained, including:

[0021] If the update parameters of the embedding layer and the head layer of the teacher model of each of the clients and the update parameters of the compilation layer of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each of the clients is obtained.

[0022] In some embodiments of the present application, the first network structure of the student model includes at least an embedding layer, and the second network structure of the student model includes at least a compilation layer; before updating the second network structure of the student model according to the first feature to the fourth feature, the method further includes:

[0023] Calculate a first loss function according to the first to fourth features;

[0024] Calculate the gradient parameters of the embedding layer and the gradient parameters of the compilation layer of the student model according to the first loss function;

[0025] Updating the second network structure of the student model according to the first to fourth features includes:

[0026] Updating the compilation layer of the student model according to the gradient parameters of the compilation layer;

[0027] After obtaining the update parameters of the second network structure of the student model, the method further includes:

[0028] Sending the gradient parameters of the embedding layer of the student model to the multiple clients, so that each of the clients updates the embedding layer of the student model according to the gradient parameters of the embedding layer, and obtains the updated parameters of the embedding layer of the student model;

[0029] If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including:

[0030] If the update parameters of the embedding layer of the student model of each client and the update parameters of the compilation layer of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

[0031] In some embodiments of the present application, the first network structure of the student model further includes a head layer; before obtaining a large language student model generated by the student model corresponding to each client if the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, the method further includes:

[0032] Acquire a fifth feature and a sixth feature respectively sent by the multiple clients, wherein the fifth feature is a feature extracted by the client from the local data according to the embedded layer of the updated student model, and the sixth feature is a feature extracted by the client from the local data according to the embedded layer of the large language teacher model;

[0033] Inputting the fifth feature into the compilation layer corresponding to the updated student model to obtain a seventh feature output by the compilation layer of the updated student model, and inputting the sixth feature into the compilation layer of the large language teacher model to obtain an eighth feature output by the compilation layer of the large language teacher model;

[0034] Sending the seventh feature and the eighth feature to the multiple clients, so that each of the clients inputs the seventh feature into the head layer of the student model to obtain a ninth feature output by the head layer of the student model, and inputs the eighth feature into the head layer of the large language teacher model to obtain a tenth feature output by the head layer of the large language teacher model, and calculating a second loss function according to the ninth feature and the tenth feature, calculating a first gradient parameter and a second gradient parameter of the student model according to the second loss function, updating the head layer and the embedding layer of the student model according to the first gradient parameter of the student model, and obtaining updated parameters of the head layer and the embedding layer of the student model;

[0035] Updating the compilation layer of the student model according to the second gradient parameter of the student model to obtain an updated parameter of the compilation layer of the student model;

[0036] If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including:

[0037] If the update parameters of the embedding layer, the update parameters of the head layer, and the update parameters of the compilation layer of the student model of each client meet the end conditions of the first iteration, a large language student model generated by the student model corresponding to each client is obtained.

[0038] In some embodiments of the present application, the calculating a first loss function according to the first to fourth features includes:

[0039] Calculate a first mean square error according to the first feature and the second feature;

[0040] Calculate a second mean square error according to the third feature and the fourth feature;

[0041] A first loss function is calculated based on the first mean square error and the second mean square error.

[0042] In some embodiments of the present application, the second loss function is calculated according to the ninth feature and the tenth feature;

[0043] A cross entropy function is calculated according to the ninth feature and the tenth feature, and the cross entropy function is used as the second loss function.

[0044] A second aspect of an embodiment of the present invention provides a large language model training system, the system comprising:

[0045] A receiving module, configured to receive first features and second features respectively sent from a plurality of clients; the first features are features extracted by the client from local data according to a first network structure of a student model, and the second features are features extracted by the client from the local data according to a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model;

[0046] a processing module, configured to input the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and input the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes the model substructures other than the first network structure in the at least two model substructures, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model;

[0047] An updating module, used for updating the second network structure of the student model according to the first to fourth features, obtaining update parameters of the second network structure of the student model, and obtaining update parameters of the first network structure of the student model corresponding to each of the clients, wherein the update parameters of the first network structure of the student model are obtained by each of the clients updating the first network structure of the student model according to the first to fourth features;

[0048] The iteration module is used to obtain a large language student model generated by the student model corresponding to each client if the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition.

[0049] A third aspect of an embodiment of the present invention provides an electronic device, comprising: at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the above-mentioned large language model training method.

[0050] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned large language model training method.

[0051] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0053] Figure 1 It is a flowchart of a large language model training method provided in an embodiment of the present application;

[0054] Figure 2 It is a schematic diagram of an implementation method of a large language model training method provided in an embodiment of the present application;

[0055] Figure 3 It is another schematic diagram of an implementation method of a large language model training method provided in an embodiment of the present application;

[0056] Figure 4 It is another schematic diagram of an implementation method of a large language model training method provided in an embodiment of the present application;

[0057] Figure 5 It is another schematic diagram of an implementation method of a large language model training method provided in an embodiment of the present application;

[0058] Figure 6 It is a structural diagram of a large language model training system provided in an embodiment of the present application;

[0059] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0061] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0062] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0063] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.

[0064] With the rapid development of artificial intelligence technology, the field of natural language processing has ushered in major technological breakthroughs, and large language models have become a key force in promoting this progress. Large language models usually use deep neural networks based on the Transformer architecture. They process massive data sets through pre-training to learn rich language features. However, existing large language models directly concentrate a large number of local user data of clients on the central processor for unified training and processing, which poses a risk of exposing users' sensitive information, increases dependence on the central processor computing infrastructure, and is not conducive to local deployment of clients.

[0065] For this purpose, refer to Figure 1 The embodiment of the present application provides a large language model training method, which is applied to a central controller. The controller may be a server, an electronic device, or a mobile terminal, etc., which is not specifically limited here. The method includes the following steps S110 to S140:

[0066] Step S110, receiving first features and second features respectively sent from multiple clients; the first feature is a feature extracted by the client from local data according to a first network structure of a student model, and the second feature is a feature extracted by the client from local data according to a first network structure of a large language teacher model, the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model.

[0067] In this step, the large language teacher model is a pre-trained model. One purpose of this embodiment is to transfer the knowledge of the large language teacher model to the student model by means of knowledge distillation until a large language student model corresponding to each client is obtained.

[0068] Knowledge distillation is a method to facilitate the transfer of knowledge from a large-scale teacher model to a small-scale student model. By transferring the knowledge of a large teacher network to a small student network, the student network can approach the performance of the teacher network while maintaining a small size.

[0069] The following explains the specific training steps of the teacher model:

[0070] Step S210, receiving initial features sent by multiple clients respectively; the initial features are features extracted by the client from local data according to the first network structure of the teacher model, the teacher model has at least two model substructures, and the first network structure of the teacher model is at least one model substructure of the at least two model substructures.

[0071] Step S220, input the initial features into the second network structure of the teacher model to obtain the intermediate features output by the second network structure of the teacher model; the second network structure of the teacher model is a model substructure of at least two model substructures except the first network structure of the teacher model.

[0072] Step S230, updating the teacher model according to the intermediate features, and obtaining update parameters of the first network structure of the teacher model and the second network structure of the teacher model corresponding to each of the clients.

[0073] Step S240: If the update parameters of the first network structure of the teacher model corresponding to each client and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained.

[0074] First, the teacher model is divided, and several model substructures included in the teacher model are set in the client and the central controller respectively. The division here is mainly based on the complexity of the network. If the model substructure is more complex, the data processing is more complex and the required computing performance is higher. Therefore, the model substructure with high complexity (that is, the second network structure of the teacher model) can be set in the central controller, and the model substructure with low complexity (that is, the first network structure of the teacher model) can be set in the client.

[0075] For example, if the teacher model is a BERT (Bidirectional Encoder Representations from Transformers) model, the BERT model can be divided into an Embedding layer, a Transformer layer, and a Head layer. The Embedding layer and the Head layer are set on the client, and the Transformer layer is placed on the central controller.

[0076] The client can be any computing device that needs to perform tasks through a large language model, and is not limited here.

[0077] This embodiment provides a training process in multiple iterations of the teacher model. The client extracts the initial features from the local data according to the first network structure of the teacher model, and then the central controller receives the initial features sent by multiple clients. Then the central controller inputs the initial features into the second network structure of the teacher model to obtain the intermediate features output by the second network structure of the teacher model. Then the intermediate features are sent to multiple clients so that each client inputs the intermediate features into the head layer of the corresponding teacher model to obtain the result features output by the head layer of the teacher model, and the first gradient parameters and the second gradient parameters of the teacher model are generated according to the result features, and the first network structure of the teacher model is updated according to the first gradient parameters of the teacher model to obtain the update parameters of the first network structure of the teacher model, and the second network structure of the teacher model is updated according to the second gradient parameters of the teacher model to obtain the update parameters of the second network structure of the teacher model. Then the central controller obtains the update parameters of the first network structure of the teacher model corresponding to each client. Finally, the central controller aggregates the parameters. If the update parameters of the first network structure of the teacher model corresponding to each client and the update parameters of the second network structure of the teacher model meet the second iteration end condition, the large language teacher model generated by the teacher model corresponding to each client is obtained.

[0078] It should be noted that if the iteration end condition is met, the first network structure trained in the client and the second network structure trained in the central controller together form a large language teacher model. For example, when in use, the first network structure of the client extracts features from the data, and then the second network structure of the central controller processes the features to finally obtain the output result.

[0079] This embodiment is a training process of a teacher model. In this process, the teacher model is divided and deployed in a central controller with sufficient resources and on a client with limited resources, respectively. This can ensure that the local data of the client is only used to train the local network structure, avoiding the risk of local data leakage. In addition, the central controller trains the network structure with a large computing load, and the client only trains the network structure with a small computing load. This can adapt to the resource limitations of the client device and distribute the complex training tasks of large-scale models to the server side, effectively reducing the computing and storage pressure on the client, allowing the client with limited resources to participate in the training of the model, and can also significantly reduce the dependence on the central controller.

[0080] In some embodiments, before obtaining the update parameters of the first network structure of the teacher model corresponding to each client in step S230, the method further includes the following steps S310 to S340:

[0081] Step S310: sending the intermediate features to the multiple clients, so that each of the clients inputs the intermediate features into the corresponding head layer of the teacher model, obtains the result features output by the head layer of the teacher model, and generates the first gradient parameters and the second gradient parameters of the teacher model according to the result features, updates the embedding layer and the head layer of the teacher model according to the first gradient parameters of the teacher model, and obtains the updated parameters of the embedding layer and the head layer of the teacher model.

[0082] Step S320: Update the compilation layer of the teacher model according to the second gradient parameter of the teacher model to obtain the update parameter of the compilation layer of the teacher model.

[0083] Step S330: If the update parameters of the embedding layer and the head layer of the teacher model of each of the clients and the update parameters of the compilation layer of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each of the clients is obtained.

[0084] In this step, the teacher model includes a first network structure including an embedding layer and a head layer and a second network structure including a compilation layer. The teacher model is trained by receiving initial features extracted from local data inputted into the embedding layer of the corresponding teacher model by each client.

[0085] In this embodiment, the central controller first receives initial features sent by multiple clients respectively, wherein the initial features are features extracted by the clients from local data according to the embedding layer of the teacher model.

[0086] The initial features are then input into the compilation layer of the teacher model to obtain the intermediate features output by the compilation layer of the teacher model, and then the intermediate features are sent to multiple corresponding clients.

[0087] The client inputs the intermediate features into the head layer of the corresponding teacher model, obtains the result features of the head layer output of the teacher model, and generates a first gradient parameter and a second gradient parameter based on the result features of the head layer output of the teacher model, and then updates the embedding layer and the head layer of the teacher model according to the first gradient parameters to obtain the updated parameters of the embedding layer and the head layer of the teacher model, wherein the first gradient parameters include the gradient parameters of the embedding layer and the head layer of the corresponding teacher model, and the second gradient parameters are the gradient parameters of the compilation layer.

[0088] The central controller updates the compilation layer of the teacher model using the second gradient parameter to obtain the update parameter of the compilation layer of the teacher model.

[0089] Finally, if the update parameters of the embedding layer and the head layer of the teacher model of each client and the update parameters of the compilation layer of the teacher model of the central controller meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained.

[0090] For ease of understanding, the following implementation is provided by taking the central controller as a server as an example:

[0091] Combination Figure 2 , Figure 3 and Figure 4 As shown, each task contains a certain number of users, for example, task A contains N clients Each client has its own local dataset

[0092] The specific large language teacher model is the BERT model, and its corresponding training process includes:

[0093] First, the teacher model is divided into three key parts: embedding layer, head layer and compilation layer, respectively. A , H A and T A At the same time, the initialized embedding layer and head layer are deployed on the client side to handle feature extraction and embedding generation of local data; the compilation layer is deployed on the server side to handle more complex feature conversion and high-level semantic understanding.

[0094] Assume that the data input is X. During the forward propagation process, the kth client C kThrough the client's embedding layer Processing to extract features and generate embedded representations These representations are then transmitted to the server, which receives the client C k After the embedding representation, further calculation and context modeling are performed through the compilation layer to generate the final prediction result

[0095] During the back-propagation process, the server first calculates the gradient parameters and updates the teacher model parameters. The gradient parameters are then sent back to each client, and each client receives the gradient parameters and the output of the first layer. Update the parameters of the head layer and embedding layer of the teacher model. In addition, to ensure model consistency, the server can periodically aggregate the model parameters of each device and synchronously update the large language teacher model. The expression of the model parameter aggregation is as follows:

[0096]

[0097] Where D is the total number of samples in all N clients. The outputs of three key layers are obtained through the aggregation process: E A represents the parameters of the embedding layer, T A Represents the parameters of the compilation layer, H A Represents the parameters of the head layer, and repeats the aggregation update steps until the model converges, thereby optimizing the allocation of computing resources and improving the efficiency and effect of model training. Finally, the network architecture of the large language teacher model is rebuilt through the aggregated output parameters, and then the final prediction output F(X) of the large language teacher model for the input data X is calculated and obtained:

[0098] F(X)=H A (T A (E A (X)))

[0099] In this embodiment, the teacher model BERT is divided into three key parts: an embedding layer, a header layer, and a compilation layer, which are deployed in a central controller with sufficient resources and on a client with limited resources, respectively. This ensures that the local data of the client is only used to train the local network structure, avoiding the risk of local data leakage. In addition, the central controller trains the network structure with a large computational load, and the client only trains the network structure with a small computational load. This can adapt to the resource limitations of the client device, so that the client with limited resources can participate in the training of the model.

[0100] In step S110, the structure of the student model is consistent with that of the large language teacher model, that is, the first network structure of the student model is consistent with the first network structure of the large language teacher model, and the second network structure of the student model is consistent with the second network structure of the large language teacher model. If the first network structure of the large language teacher model is set on the client, then the first network structure of the student model is also set on the client, and the second network structure of the large language teacher model is set on the central controller, then the second network structure of the student model is also set on the central controller.

[0101] Similar to the training process of the teacher model, the central controller receives the first feature and the second feature sent by each of the multiple clients; the first feature is a feature extracted by the client from the local data according to the first network structure of the student model, and the second feature is a feature extracted by the client from the local data according to the first network structure of the large language teacher model. Because this embodiment needs to solve the problem of transferring knowledge to the student model, it requires the features extracted from the local data by the first network structure of the student model and the features extracted from the local data by the first network structure of the large language teacher model.

[0102] Step S120, input the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and input the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes at least two model substructures except the first network structure, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model.

[0103] Similar to the training process of the teacher model, in this step, the central controller inputs the first feature and the second feature into the second network structure of the student model and the second network structure of the large language teacher model respectively set in the central controller, obtains the third feature output by the second network structure of the student model, and obtains the fourth feature output by the second network structure of the large language teacher model.

[0104] Since the local data is directly input into the first network structure corresponding to the local client, the extraction of data features is achieved and the leakage of local data is avoided.

[0105] Step S130, update the second network structure of the student model according to the first to fourth features, obtain the updated parameters of the second network structure of the student model, and obtain the updated parameters of the first network structure of the student model corresponding to each client. The updated parameters of the first network structure of the student model are obtained by each client updating the first network structure of the student model according to the first to fourth features.

[0106] Similar to the training process of the teacher model, this embodiment updates the second network structure of the student model according to the first to fourth features, obtains the update parameters of the second network structure of the student model, and then obtains the update parameters of the first network structure of the student model corresponding to each client. The obtained parameters are used to determine the end of subsequent iterations.

[0107] Step S140: If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

[0108] In this step, whether the first iteration end condition is met is determined based on the updated parameters of the first network structure of the student model of each client and the updated parameters of the second network structure of the student model. When the first iteration end condition is met, a large language student model generated by the student model corresponding to each client is obtained.

[0109] It should be noted that in steps S110 to S140, the large language teacher model is a trained model and does not need to be updated.

[0110] This method trains the network structure with heavy computational load through the central controller, and the client only trains the network structure with light computational load. It can adapt to the resource limitations of the client device, so that the resource-constrained client can participate in the training of the model. Moreover, through the knowledge distillation method, the knowledge of the teacher model is transferred to the student model, which ensures that the data is not leaked and can significantly reduce the dependence on the central controller.

[0111] In some embodiments of the present application, the first network structure of the student model includes at least an embedding layer, and the second network structure of the student model includes at least a compilation layer; before updating the second network structure of the student model according to the first to fourth features in step S130, the method further includes:

[0112] The updating of the teacher model according to the intermediate features to obtain the update parameters of the first network structure of the teacher model and the update parameters of the second network structure of the teacher model corresponding to each of the clients includes the following steps S410 to S450:

[0113] Step S410: Calculate a first loss function according to the first to fourth features.

[0114] Step S420: Calculate the gradient parameters of the embedding layer and the gradient parameters of the compilation layer of the student model according to the first loss function.

[0115] Updating the second network structure of the student model according to the first to fourth features in step S130 includes:

[0116] Step S430: Update the compilation layer of the student model according to the gradient parameters of the compilation layer.

[0117] After obtaining the update parameters of the second network structure of the student model in step S130, the method further includes:

[0118] Step S440: Send the gradient parameters of the embedding layer of the student model to multiple clients, so that each client updates the embedding layer of the student model according to the gradient parameters of the embedding layer to obtain the updated parameters of the embedding layer of the student model.

[0119] If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including:

[0120] Step S450: If the update parameters of the embedding layer of the student model of each client and the update parameters of the compilation layer of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

[0121] In this embodiment, the student model has at least an embedding layer, which can ensure that the local data of each client is not leaked. After obtaining the first to fourth features, the first loss function is calculated according to the first to fourth features, and then the gradient parameters of the embedding layer and the gradient parameters of the compilation layer of the student model are calculated according to the first loss function, and then the first network structure and the second network structure can be updated by using these gradient parameters, and then it is determined whether the iteration end condition is met.

[0122] In some embodiments of the present application, the first network structure of the student model further includes a head layer; in step S140, if the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, before obtaining the large language student model generated by the student model corresponding to each client, the method further includes the following steps S510 to S550:

[0123] Step S510: obtaining fifth features and sixth features sent by multiple clients respectively, where the fifth feature is a feature extracted by the client from the local data according to the embedding layer of the updated student model, and the sixth feature is a feature extracted by the client from the local data according to the embedding layer of the large language teacher model.

[0124] Step S520: input the fifth feature into the compilation layer corresponding to the updated student model to obtain the seventh feature output by the compilation layer of the updated student model, and input the sixth feature into the compilation layer of the large language teacher model to obtain the eighth feature output by the compilation layer of the large language teacher model.

[0125] Step S530, sending the seventh feature and the eighth feature to multiple clients, so that each client inputs the seventh feature into the head layer of the student model to obtain the ninth feature of the head layer output of the student model, and inputs the eighth feature into the head layer of the large language teacher model to obtain the tenth feature of the head layer output of the large language teacher model, and calculates the second loss function according to the ninth feature and the tenth feature, calculates the first gradient parameter and the second gradient parameter of the student model according to the second loss function, updates the head layer and the embedding layer of the student model according to the first gradient parameter of the student model, and obtains the updated parameters of the head layer and the embedding layer of the student model.

[0126] Step S540: Update the compilation layer of the student model according to the second gradient parameter of the student model to obtain the updated parameter of the compilation layer of the student model.

[0127] If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including:

[0128] Step S550: If the update parameters of the embedding layer, the update parameters of the head layer, and the update parameters of the compilation layer of the student model of each client meet the end condition of the first iteration, a large language student model generated by the student model corresponding to each client is obtained.

[0129] For ease of understanding, the following implementation is provided by taking the central controller as a server as an example:

[0130] Combination Figure 2 , Figure 3 and Figure 4As shown, the student model is initialized on the client k, and the same model structure as the large language teacher model in the above implementation mode is adopted, that is, the large language teacher model and the student model (both BERT models) are divided into an Embedding layer, a Transformer layer, and a Head layer. According to the different complexity of the model substructure, the compilation layer with a higher complexity than the embedding layer and the head layer is deployed on the server side. Each client uses its own local data set to independently train the embedding layer and the head layer of the corresponding student model, i.e., the sub-model. The server side receives the forward propagation and back propagation gradient parameters from the client, and updates the compilation layer accordingly. Therefore, the student model of each client has the same structure as the teacher model, and is composed of an embedding layer and a head layer deployed on the client and a compilation layer on the server. Then, the student model is trained according to the trained large language teacher model to obtain a large language student model, which specifically includes the embedding layer and the compilation layer of the trained student model and the head layer of the trained student model:

[0131] The following first introduces the embedding layer and compilation layer of the training student model;

[0132] First, on client k, the local data D k Input into the embedding layer of the large language teacher model and the student model to obtain the embedding layer output vector T of the large language teacher model e , and the embedding layer output vector T of the student model e , the two vectors T e and T e Sent to the server S respectively.

[0133] The server S then receives the embedding layer output vector T of the large language teacher model from the client e , and the embedding layer output vector S of the student model e Finally, these output vectors are input into the embedding layers of the large language teacher model and the student model respectively to obtain the output vector T of the embedding layer of the large language teacher model t and the output vector S of the embedding layer of the student model t .

[0134] Finally, the two embedding layer output vectors T are calculated e , S e The mean squared error (MSE) between embedding , whose expression is:

[0135]

[0136] where d e The dimension of the embedding layer output vectors for the large language teacher model and the student model, T e(j) and S e (j) is the corresponding value of the jth element of the output vector. Calculate the two coding layer output vectors T t , S t to determine the mean square error between L transformer .

[0137]

[0138] where d t The dimension of the compiled layer output vectors for the large language teacher and student models, T t (j) and S t (j) is the corresponding value of the jth element of the output vector.

[0139] The first loss function of the student model is calculated based on the mean square error of the embedding layer of the student model and the mean square error of the compilation layer of the student model, and its expression is:

[0140] L=L embedding +L transformer

[0141] Finally, according to the first loss function of the student model, the gradient parameters G of the embedding layers of the student model are calculated respectively. e and the gradient parameter G of the compilation layer t , and G e Sent to the client, each client according to G e Update the parameters of the embedding layer of the student model. The server side updates the embedding layer according to G t Update the parameters of the compilation layer of the student model and then train the head layer of the student model.

[0142] The following introduces the training of the first layer of the student model;

[0143] On client k, local data D k Input into the embedding layer of the large language teacher model and the updated student model to obtain the embedding layer output vector T of the large language teacher model e , and the embedding layer output vector S of the updated student model e .

[0144] The embedding layer output vector T of the large language teacher model e and the embedding layer output vector S of the updated student model e Sent to the server S, which receives the embedding layer output vector T of the large language teacher model from the client e , and the embedding layer output vector S of the updated student model e Finally, these vectors are input into the compilation layer of the large language teacher model and the updated student model respectively to obtain the output vector T of the compilation layer of the large language teacher modelt and the output vector S of the compiled layer of the updated student model t .

[0145] Then the output vector T of the compilation layer of the large language teacher model is t and the output vector S of the compiled layer of the updated student model t Send to client k, client k receives the output vector T of the compilation layer of the large language teacher model t and the output vector S of the compilation layer of the updated student model t Input them into the head layer of the large language teacher model and the updated student model respectively to obtain the output vector T of the head layer of the large language teacher model h And the output vector S of the head layer of the updated student model h , calculate T h and S h The cross entropy loss function between logits , which is expressed as follows:

[0146]

[0147] where d h is the output vector dimension of the head layer of the large language teacher model and the updated student model, T h (j) and S h (j) is the corresponding value of the jth element of the output vector.

[0148] In order to calculate the second loss function of the updated student model, the expression of the second loss function L is as follows:

[0149] L=L logits

[0150] Finally, client k calculates the gradient parameter G of the head layer of the updated student model according to the second loss function h , and update the parameters of the head layer of the updated student model according to the update.

[0151] A set of experimental examples is provided below:

[0152] Instantiating a smaller BERT base The student model (layers l = 4, hidden layer dimension d = 312, feedforward size di = 1200, head layer number h = 12) has a total of 14.7M parameters. In contrast, the teacher model used is a standard BERT baseThe model has l=12 layers, hidden layer dimension d=768, feedforward network size di=3072, and h=12 Head layers, with a total of 110M parameters. It can be seen that the number of parameters in the student model is significantly reduced compared to the teacher model, accounting for only 13.4% of the number of parameters in the teacher model.

[0153] (1) Implement the training of the teacher model.

[0154] At this stage, the teacher model was used for fine-tuning. In the federated learning simulation environment, 5 clients were set to participate in the training, the global training rounds were 10 times, each client performed 1 local training round, the learning rate was set to lr = 5e-5, the number of samples per batch training was batchsize = 32, and the FedAvg method was used to aggregate the model. During the training process, the 5 clients completed the training tasks independently before the model was aggregated. The training data set was randomly and evenly divided into 5 parts, and each client used one of them for training.

[0155] (2) Realize the distillation of the student model.

[0156] The training of the student model is divided into two steps. First, when training the Embedding layer and the Transformer layer, the Transformer layer of the student model is trained from the BERT of the teacher model. base In the Transformer layer in , knowledge distillation is performed every 3 layers. The training rounds in this step are fixed to 10 times, and the learning rate lr = 5e-5. Secondly, when training the Head layer, the learning rate is adjusted to lr = 3e-5, and the training rounds are adjusted according to the requirements of the specific task. The training batches of both steps are set to batchsize = 32.

[0157] In addition, the following three benchmark schemes are considered for performance comparison:

[0158] 1) BERT base : Using pre-trained BERT base The model is tested.

[0159] 2) FedBERT: Tested using the BERT teacher model fine-tuned in the first stage.

[0160] 3) Private Model: Deploy the initialized student model on each edge device and train it in their own private data to obtain the private student model on the device for testing. This part adopts the student model structure.

[0161] The same data set partitioning mode as the teacher model training is adopted to ensure that the private data of each client is the same as that of the teacher training.

[0162] In order to comprehensively evaluate the performance of the model, the General Language Understanding Evaluation (GLUE) benchmark was selected as the test platform. The GLUE benchmark is a comprehensive collection of various natural language understanding tasks, which provides a standardized testing environment for the multi-faceted language processing capabilities of the model.

[0163] In this study, three specific downstream tasks in the GLUE benchmark were selected for experimental verification. These tasks include:

[0164] SST-2 (Single Sentence Sentiment Analysis): In this task, the goal is to classify the input sentences into positive or negative categories based on their sentiment. If the sentence expresses a positive sentiment, it is classified and marked as 1; if the sentence expresses a negative sentiment, it is classified and marked as 0. This task has a total of 67,350 training set samples, 873 validation set samples, and 1,821 test set samples, and the performance of the model is judged by accuracy.

[0165] CoLA (Single Sentence Language Acceptability Analysis): In this task, the goal is to judge the legitimacy of the sentence structure of the input sentence based on its grammar. As a binary classification task, 0 represents a sentence that is grammatically unacceptable, while 1 represents a sentence that is grammatically acceptable. This task has a total of 8551 training set samples, 1043 validation set samples, and 1063 test set samples, and the performance of the model is judged by the Matthews correlation coefficient.

[0166] MRPC (Semantic Similarity Task): In this task, the goal is to evaluate the semantic equivalence of sentence pairs, aiming to deeply understand and identify whether two sentences express the same or similar meanings at the semantic level. In this task, 0 and 1 are used as labels, 0 represents that the sentence pair is not semantically similar, and 1 represents that the sentence pair is semantically similar. The dataset used for this task contains 5,800 pairs of sentences, which are extracted from online news sources and accompanied by human annotations, clearly indicating whether each pair of sentences has a paraphrase or semantic equivalence relationship. Due to the long sentence length in the sample and the uneven distribution of categories in the dataset, positive samples account for 68% and negative samples account for 32%. In order to more accurately measure the model performance, accuracy is used as the main evaluation indicator, which directly reflects the proportion of correct classifications of the model.

[0167] Since some training sets of GLUE tasks have less data, the data set was expanded using the data augmentation method in Huawei Tinybert. Through detailed experiments on these tasks, we can deeply understand the performance of the model in different language understanding and reasoning scenarios, thereby ensuring that the evaluation results of the model are highly representative and accurate.

[0168] The model performance and baseline model performance were tested on the SST-2, CoLA, and MRPC tasks in the GLUE task, as shown in Table 1.

[0169] Table 1

[0170]

[0171] BERT base The large number of parameters makes it limited in resource-constrained environments.

[0172] The FedBERT model is large in size and difficult to deploy on the local client after training.

[0173] The private model (PrivateModel) has a small number of parameters and is a small-scale model trained on local data. Its performance on the three tasks is significantly lower than that of the other three models, and its performance is poor.

[0174] Therefore, the student model of this application significantly improves performance while maintaining the same small number of parameters, especially on the SST-2 and CoLA tasks, far exceeding the private model. This result shows that the student model can not only be effectively trained and deployed in a resource-constrained environment, but also achieve excellent task performance, demonstrating its advantages and applicability in practical applications.

[0175] Combination Figure 5As shown, some embodiments of the present application can provide different types of task training sets and local data through different clients to simultaneously train multiple large language teacher models for different tasks. For example, the teacher model for task A is trained to obtain the corresponding large language model, which is used to train the student model that performs task A. The student model that performs task A learns the large language teacher model for task A through knowledge distillation. This embodiment divides the student model and the large language teacher model, and deploys them on a server with sufficient resources and a client with limited resources, respectively, which can ensure that the local data of the client only trains the local network structure, avoiding the risk of local data leakage. In addition, this method trains the network structure with a large computing load through the server, and the client only trains the network structure with a small computing load, which can adapt to the resource limitations of the client device, so that the client with limited resources can participate in the training of the model, and the knowledge of the teacher model is transferred to the student model through knowledge distillation, which ensures that the data is not leaked and can significantly reduce the dependence on the central controller.

[0176] like Figure 6 As shown, some embodiments of the present application provide a large language model training system, the system includes a receiving module 810, a processing module 820, an updating module 830, and an iteration module 840, specifically:

[0177] The receiving module 810 is used to receive first features and second features respectively sent from multiple clients; the first feature is a feature extracted by the client from local data according to a first network structure of a student model, and the second feature is a feature extracted by the client from local data according to a first network structure of a large language teacher model, the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model;

[0178] The processing module 820 is used to input the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and input the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes at least two model substructures except the first network structure, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model;

[0179] An updating module 830 is used to update the second network structure of the student model according to the first to fourth features, obtain update parameters of the second network structure of the student model, and obtain update parameters of the first network structure of the student model corresponding to each client, where the update parameters of the first network structure of the student model are obtained by each client updating the first network structure of the student model according to the first to fourth features;

[0180] The iteration module 840 is used to obtain a large language student model generated by the student model corresponding to each client if the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition.

[0181] It should be noted that the large language model training system provided in this embodiment and the above-mentioned large language model training system method are based on the same inventive concept, so the relevant content of the above-mentioned large language model training method is also applicable to the content of the large language model training system, and therefore, it will not be repeated here.

[0182] like Figure 7 The embodiment of the present application further provides an electronic device, the electronic device comprising:

[0183] At least one hydrogen fuel cell;

[0184] at least one memory;

[0185] at least one processor;

[0186] at least one program;

[0187] The program is stored in the memory, and the processor executes at least one program to implement the large language model training method implemented in the present disclosure.

[0188] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0189] The electronic device according to the embodiment of the present application is described in detail below.

[0190] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present disclosure;

[0191] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1700, and the processor 1600 calls and executes a large language model training method of the embodiment of the present disclosure.

[0192] Input / output interface 1800, used to implement information input and output;

[0193] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0194] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0195] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0196] The embodiment of the present disclosure also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned large language model training method.

[0197] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0198] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0199] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present disclosure, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0200] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0202] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0203] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0207] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs.

[0208] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

[0209] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A large language model training method, characterized in that: For a central controller, the method comprises: Receiving first features and second features respectively sent from a plurality of clients; the first features are features extracted by the client from local data according to a first network structure of a student model, and the second features are features extracted by the client from the local data according to a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model; Inputting the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and inputting the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes the model substructures of the at least two model substructures except the first network structure, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model; The second network structure of the student model is updated according to the first to fourth features, and update parameters of the second network structure of the student model are obtained, and the update parameters of the first network structure of the student model corresponding to each of the clients are obtained by each of the clients updating the first network structure of the student model according to the first to fourth features; If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

2. The large language model training method according to claim 1, characterized in that: Before receiving the first feature and the second feature respectively sent from the plurality of clients, the method further includes: Receiving initial features sent by multiple clients respectively; the initial features are features extracted by the clients from local data according to a first network structure of a teacher model, the teacher model has at least two model substructures, and the first network structure of the teacher model is at least one model substructure of the at least two model substructures; Inputting the initial features into the second network structure of the teacher model to obtain intermediate features output by the second network structure of the teacher model; the second network structure of the teacher model is a model substructure of the at least two model substructures except the first network structure of the teacher model; Update the teacher model according to the intermediate features to obtain update parameters of the first network structure of the teacher model and the second network structure of the teacher model corresponding to each of the clients; If the update parameters of the first network structure of the teacher model corresponding to each of the clients and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each of the clients is obtained.

3. The large language model training method according to claim 2, characterized in that: The first network structure of the teacher model includes an embedding layer and a head layer, and the second network structure of the teacher model is a compilation layer; the initial features are features extracted by the client from local data according to the embedding layer of the teacher model; the updating of the teacher model according to the intermediate features to obtain the update parameters of the first network structure of the teacher model and the update parameters of the second network structure of the teacher model corresponding to each of the clients includes: Sending the intermediate features to the multiple clients, so that each of the clients inputs the intermediate features into the corresponding head layer of the teacher model, obtains the result features output by the head layer of the teacher model, generates the first gradient parameter and the second gradient parameter of the teacher model according to the result features, updates the embedding layer and the head layer of the teacher model according to the first gradient parameter of the teacher model, and obtains the updated parameters of the embedding layer and the head layer of the teacher model; Updating the compilation layer of the teacher model according to the second gradient parameter of the teacher model to obtain an updated parameter of the compilation layer of the teacher model; If the update parameters of the first network structure of the teacher model of each client and the update parameters of the second network structure of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each client is obtained, including: If the update parameters of the embedding layer and the head layer of the teacher model of each of the clients and the update parameters of the compilation layer of the teacher model meet the second iteration end condition, a large language teacher model generated by the teacher model corresponding to each of the clients is obtained.

4. The large language model training method according to claim 1, characterized in that: The first network structure of the student model includes at least an embedding layer, and the second network structure of the student model includes at least a compilation layer; before updating the second network structure of the student model according to the first feature to the fourth feature, the method further includes: Calculate a first loss function according to the first to fourth features; Calculate the gradient parameters of the embedding layer and the gradient parameters of the compilation layer of the student model according to the first loss function; Updating the second network structure of the student model according to the first to fourth features includes: Updating the compilation layer of the student model according to the gradient parameters of the compilation layer; After obtaining the update parameters of the second network structure of the student model, the method further includes: Sending the gradient parameters of the embedding layer of the student model to the multiple clients, so that each of the clients updates the embedding layer of the student model according to the gradient parameters of the embedding layer, and obtains the updated parameters of the embedding layer of the student model; If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including: If the update parameters of the embedding layer of the student model of each client and the update parameters of the compilation layer of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained.

5. The large language model training method according to claim 4, characterized in that: The first network structure of the student model also includes a head layer; before obtaining a large language student model generated by the student model corresponding to each client if the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, the method also includes: Acquire a fifth feature and a sixth feature respectively sent by the multiple clients, wherein the fifth feature is a feature extracted by the client from the local data according to the embedded layer of the updated student model, and the sixth feature is a feature extracted by the client from the local data according to the embedded layer of the large language teacher model; Inputting the fifth feature into the compilation layer corresponding to the updated student model to obtain a seventh feature output by the compilation layer of the updated student model, and inputting the sixth feature into the compilation layer of the large language teacher model to obtain an eighth feature output by the compilation layer of the large language teacher model; Sending the seventh feature and the eighth feature to the multiple clients, so that each of the clients inputs the seventh feature into the head layer of the student model to obtain a ninth feature output by the head layer of the student model, and inputs the eighth feature into the head layer of the large language teacher model to obtain a tenth feature output by the head layer of the large language teacher model, and calculating a second loss function according to the ninth feature and the tenth feature, calculating a first gradient parameter and a second gradient parameter of the student model according to the second loss function, updating the head layer and the embedding layer of the student model according to the first gradient parameter of the student model, and obtaining updated parameters of the head layer and the embedding layer of the student model; Updating the compilation layer of the student model according to the second gradient parameter of the student model to obtain an updated parameter of the compilation layer of the student model; If the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition, a large language student model generated by the student model corresponding to each client is obtained, including: If the update parameters of the embedding layer, the update parameters of the head layer, and the update parameters of the compilation layer of the student model of each client meet the end conditions of the first iteration, a large language student model generated by the student model corresponding to each client is obtained.

6. The large language model training method according to claim 4, characterized in that: The calculating a first loss function according to the first to fourth features includes: Calculate a first mean square error according to the first feature and the second feature; Calculate a second mean square error according to the third feature and the fourth feature; A first loss function is calculated based on the first mean square error and the second mean square error.

7. The large language model training method according to claim 5, characterized in that: The second loss function is calculated according to the ninth feature and the tenth feature; A cross entropy function is calculated according to the ninth feature and the tenth feature, and the cross entropy function is used as the second loss function.

8. A large language model training system, characterized in that: The system comprises: A receiving module, configured to receive first features and second features respectively sent from a plurality of clients; the first features are features extracted by the client from local data according to a first network structure of a student model, and the second features are features extracted by the client from the local data according to a first network structure of a large language teacher model, wherein the student model has at least two model substructures, the first network structure of the student model includes at least one model substructure of the at least two model substructures, and the first network structure of the student model is the same as the first network structure of the large language teacher model; a processing module, configured to input the first feature into the second network structure of the student model to obtain a third feature output by the second network structure of the student model, and input the second feature into the second network structure of the large language teacher model to obtain a fourth feature output by the second network structure of the large language teacher model; the second network structure of the student model includes the model substructures other than the first network structure in the at least two model substructures, and the complexity of the first network structure of the student model is lower than the complexity of the second network structure of the student model, and the second network structure of the student model is the same as the second network structure of the large language teacher model; An updating module, used for updating the second network structure of the student model according to the first to fourth features, obtaining update parameters of the second network structure of the student model, and obtaining update parameters of the first network structure of the student model corresponding to each of the clients, wherein the update parameters of the first network structure of the student model are obtained by each of the clients updating the first network structure of the student model according to the first to fourth features; The iteration module is used to obtain a large language student model generated by the student model corresponding to each client if the update parameters of the first network structure of the student model of each client and the update parameters of the second network structure of the student model meet the first iteration end condition.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a large language model training method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a large language model training method as described in any one of claims 1 to 7.

Images