Model training method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a model training method and device, electronic equipment and storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of large language model, federated learning and deep learning. The specific implementation scheme is: receiving the first model parameters of M target prompt modules respectively issued by a server; taking the first model parameters of the M target prompt modules and the current model parameters of the remaining N-M prompt modules in N prompt modules as initial parameters of a large language model, training the large language model by using a plurality of data samples stored in a federated learning device to obtain second model parameters of the M target prompt modules respectively; and sending the second model parameters of the M target prompt modules to the server, the server being configured to update the model parameters of the N prompt modules respectively based on the second model parameters of the M target prompt modules respectively sent by a plurality of federated learning devices included in a federated learning system.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to the fields of large language models, federated learning, and deep learning. Background Technology

[0002] Large-scale language models refer to language models trained on large-scale text corpora, containing hundreds of billions or more parameters, designed to understand and generate human language, and have been widely used in text summarization, translation, sentiment analysis and other fields. Summary of the Invention

[0003] This disclosure provides a model training method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of this disclosure, a model training method is provided, comprising: receiving first model parameters of each of M target prompting modules sent by a server, wherein a large language model includes N prompting modules, the M target prompting modules belong to the N prompting modules, and N≥M≥1; using the first model parameters of each of the M target prompting modules and the current model parameters of each of the remaining NM prompting modules in the N prompting modules as initial parameters of the large language model, training the large language model using multiple data samples stored in a federated learning device to obtain second model parameters of each of the M target prompting modules; and sending the second model parameters of each of the M target prompting modules to the server, wherein the server is configured to update the model parameters of each of the N prompting modules based on the second model parameters of each of the M target prompting modules sent by the multiple federated learning devices included in the federated learning system.

[0005] According to another aspect of this disclosure, a model training apparatus is provided, comprising: a first receiving module, configured to receive first model parameters of each of M target prompting modules sent by a server, wherein a large language model includes N prompting modules, the M target prompting modules being part of the N prompting modules, wherein N≥M≥1; a first training module, configured to use the first model parameters of each of the M target prompting modules and the current model parameters of each of the remaining NM prompting modules in the N prompting modules as initial parameters of the large language model, and train the large language model using multiple data samples stored in a federated learning device to obtain second model parameters of each of the M target prompting modules; and a first sending module, configured to send the second model parameters of each of the M target prompting modules to the server, wherein the server is configured to update the model parameters of each of the N prompting modules based on the second model parameters of each of the M target prompting modules sent by the multiple federated learning devices included in the federated learning system.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described above.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 The illustration schematically depicts an exemplary system architecture to which model training methods and apparatus can be applied according to embodiments of the present disclosure.

[0012] Figure 2 A flowchart illustrating a model training method according to an embodiment of the present disclosure is shown schematically.

[0013] Figure 3 A schematic diagram of a federated learning system according to an embodiment of the present disclosure is shown.

[0014] Figure 4 A flowchart illustrating a model training method according to another embodiment of the present disclosure is shown schematically.

[0015] Figure 5 A flowchart illustrating a method for determining a target prompting module according to an embodiment of the present disclosure is shown schematically.

[0016] Figure 6 A block diagram of a model training apparatus according to an embodiment of the present disclosure is shown schematically.

[0017] Figure 7 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0018] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] Large Language Models (LLMs) are widely used in natural language processing tasks. The number of parameters in different large language models can range from 110 million to 175 billion. Therefore, updating parameters during the pre-training or fine-tuning process of any large language model will generate huge communication and computation costs. In practical applications, it is difficult to adjust the parameters of the model ontology of large language models.

[0020] The design of the prompt module provides a new approach to fine-tuning large language models. By adding a corresponding prompt module to each layer of the large language model and establishing the relationship between the prompt module and the output of the large language model, the parameters of the large language model ontology can be frozen during the fine-tuning process, and only the parameters of the prompt module are updated during the adjustment process. Therefore, the number of parameters that need to be updated can be significantly reduced.

[0021] Federated learning can be used to update the parameters of the prompt module within a large language model. Federated learning is a distributed machine learning approach that allows multiple user devices to collaboratively train a global model representing all user devices without exchanging user data. However, current methods using federated learning to adjust the prompt module typically require transmitting the entire set of parameters related to the prompt module between devices, resulting in significant communication costs.

[0022] In view of this, embodiments of the present disclosure provide a model training method, apparatus, electronic device, and storage medium for adjusting a large language model in federated learning. The specific implementation scheme is as follows: receiving first model parameters for each of M target prompting modules from a server, wherein the large language model includes N prompting modules, the M target prompting modules belong to the N prompting modules, and N≥M≥1; using the first model parameters of each of the M target prompting modules and the current model parameters of the remaining NM prompting modules in the N prompting modules as initial parameters of the large language model, training the large language model using multiple data samples stored in the federated learning device to obtain second model parameters for each of the M target prompting modules; and sending the second model parameters of each of the M target prompting modules to a server, wherein the server is configured to update the model parameters of each of the N prompting modules based on the second model parameters of each of the M target prompting modules sent by the multiple federated learning devices included in the federated learning system.

[0023] Figure 1 The illustration schematically depicts an exemplary system architecture to which model training methods and apparatus can be applied according to embodiments of the present disclosure.

[0024] It is important to note that Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments, or scenarios. For example, in another embodiment, an exemplary system architecture to which the model training method and apparatus can be applied may include a terminal device, but the terminal device can implement the model training method and apparatus provided by the embodiments of this disclosure without interacting with the server.

[0025] like Figure 1 As shown, the system architecture 100 according to this embodiment can constitute a federated learning system. The system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105.

[0026] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers. Terminal devices 101, 102, and 103 can be used as federated learning devices in a federated learning system. That is, terminal devices 101, 102, and 103 can store data samples and model parameters. During federated learning, terminal devices 101, 102, and 103 can use their stored data samples to train the model built based on their stored model parameters, thereby adjusting the model parameters.

[0027] Network 104 is a medium used to provide a communication link between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Server 105 can be a server providing various services, without limitation. Server 105 can also store model parameters. During federated learning, server 105 can send model parameters to terminal devices 101, 102, and 103 via network 104. Furthermore, server 105 can receive adjusted model parameters returned by terminal devices 101, 102, and 103, and adjust the model parameters stored in server 105 based on the received adjusted model parameters.

[0029] It should be noted that the model training method provided in this embodiment can generally be executed by terminal devices 101, 102, or 103. Correspondingly, the content processing device provided in this embodiment can also be disposed in terminal devices 101, 102, or 103.

[0030] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0031] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0032] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0033] Figure 2 A flowchart illustrating a model training method according to an embodiment of the present disclosure is shown schematically.

[0034] like Figure 2 As shown, the method 200 includes operations S210 to S230.

[0035] In operation S210, the first model parameters of each of the M target prompting modules are received from the server. The large language model includes N prompting modules, and the M target prompting modules belong to the N prompting modules, where N≥M≥1.

[0036] In operation S220, the first model parameters of each of the M target prompting modules and the current model parameters of the remaining NM prompting modules in the N prompting modules are used as the initial parameters of the large language model. The large language model is trained using multiple data samples stored in the federated learning device to obtain the second model parameters of each of the M target prompting modules.

[0037] In operation S230, the second model parameters of each of the M target prompting modules are sent to the server. The server is configured to update the model parameters of each of the N prompting modules based on the second model parameters of each of the M target prompting modules sent by the multiple federated learning devices included in the federated learning system.

[0038] According to embodiments of this disclosure, the server can be a parameter server in a federated learning system. The federated learning system may include a server and multiple federated learning devices. The model parameters of a large language model and the model parameters of the prompting module included in the large language model can both be stored on the server, while the data samples used for pre-training or fine-tuning of the large language model can be stored separately in multiple federated learning devices. During federated learning, each federated learning device can use its own stored data samples to train the model. The trained model parameters can be sent to the server, which then summarizes the model parameters and determines the model parameters for this round of training.

[0039] According to embodiments of this disclosure, before training a large language model begins, or after training the large language model for a preset number of rounds, M target prompting modules can be determined from the N prompting modules included in the large language model for each federated learning device. These M target prompting modules are then associated with the federated learning device. In each subsequent training round, communication between the federated learning device and the server can be limited to the model parameters of these M target prompting modules. The M target prompting modules associated with the federated learning device can be selected from the N prompting modules in any manner. For example, if the federated learning system includes m federated learning devices, the N prompting modules can be evenly distributed according to the number of federated learning devices, i.e., N / m prompting modules are assigned to each federated learning device as target prompting modules associated with that device. Alternatively, the target prompting modules can be determined based on the importance of each prompting module, etc., without limitation.

[0040] According to embodiments of this disclosure, the prompting module can be represented as a neural network layer, and each neural network layer can include one or more neurons, each neuron can have corresponding weights, biases, and other parameters. Accordingly, the model parameters of the prompting module can refer to the set composed of the parameters of each of the first or more neurons included in the prompting module.

[0041] According to embodiments of this disclosure, the first model parameter of the target prompting module may refer to the model parameter of the target prompting module of the large language model determined by the server based on the model parameters fed back by each federated learning device after the previous round of model training.

[0042] According to embodiments of this disclosure, the current model parameters of the prompting module can refer to the model parameters of the prompting module recorded in the federated learning device after the previous round of model training.

[0043] According to embodiments of this disclosure, when training a large language model using multiple data samples stored in a federated learning device, the model ontology of the large language model only performs inference, while the N prompting modules both perform input inference and parameter updates. That is, the parameters of the model ontology of the large language model can be frozen, and only the model parameters of the N prompting modules are adjusted. The second model parameters of the prompting modules can be the model parameters obtained after model parameter tuning using multiple data samples.

[0044] According to embodiments of this disclosure, the same prompting module can be assigned to multiple federated learning devices for parameter tuning. Correspondingly, the server can receive second model parameters of the same prompting module from different federated learning devices. The server can integrate multiple second model parameters of the same prompting module using methods such as weighted summation or averaging to determine the model parameters of the prompting module. These model parameters can be used to distribute to the corresponding federated learning devices in the next round of training.

[0045] According to embodiments of this disclosure, when adjusting prompts for a large language model, each federated learning device in the federated learning system can receive specific prompt layers from the server, namely, the first model parameters of each of the M target prompt modules. Using the first model parameters of the M target prompt modules and the current model parameters of the other NM prompt modules as initial parameters for the current training round, the device adjusts the prompts for the large language model using data samples stored in it, thereby obtaining the second model parameters of each of the M target prompt modules. The federated learning device can return the obtained second model parameters to the server so that the server can integrate the model parameters to obtain a large language model for the next round of model training or for practical applications. By selecting to transmit a limited number of prompt module model parameters between the server and the federated learning device during each round of federated learning, instead of transmitting the model parameters of all prompt modules in related technologies, communication costs can be effectively reduced and the efficiency of prompt adjustment for large language models can be improved.

[0046] The following is for reference. Figures 3-5 In conjunction with specific embodiments, Figure 2The method shown will be further explained.

[0047] Figure 3 A schematic diagram of a federated learning system according to an embodiment of the present disclosure is shown.

[0048] like Figure 3 As shown, the federated learning system can consist of a server 310 and multiple federated learning devices 320. A large language model may include N prompting modules 301. The server 310 can record the model parameters of each of the N prompting modules 301. Similarly, each of the multiple federated learning devices 320 also records the model parameters of each of the N prompting modules 301. The model parameters of each of the N prompting modules 301 recorded in the server 310 may differ from the model parameters of each of the N prompting modules 301 recorded in the federated learning devices 320.

[0049] According to embodiments of this disclosure, the large language model may further include a model ontology 302. The federated learning device 320 may also record model parameters of the model ontology 302 of the large language model. During the training of the large language model using multiple data samples stored in the federated learning device 320, the model parameters of the model ontology 302 can be set to a frozen state. That is, the model parameters of the model ontology 302 may not be adjusted during the federated learning process. The model ontology 302 of the large language model may include N network layers, and the N prompting modules 301 may correspond to each of the N network layers.

[0050] According to embodiments of this disclosure, when performing federated learning using the federated learning system described above, during each training round, each federated learning device 320 can receive updated prompt parameters for a specific layer from the server 310, namely, the first model parameters of each of the M target prompt modules. The federated learning device 320 can adjust the prompts based on the updated prompt parameters for that specific layer and the previously recorded prompt parameters for other layers, using data samples stored in the federated learning device 320. After the prompt adjustment is completed, the adjusted prompt parameters for each layer can be recorded in the federated learning device 320, and the federated learning device 320 can return the adjusted prompt parameters for the specific layer to the server 310. The server 310 can update the prompt parameters for each layer based on the prompt parameters for the specific layer returned by each federated learning device 320.

[0051] According to embodiments of this disclosure, the process by which server 310 updates the cue parameters of each layer based on the cue parameters of each specific layer returned by each federated learning device 320 can be represented as solving the problem of how to generate a set of cue parameters to minimize the global loss, as shown in formula (1):

[0052]

[0053] In formula (1), P can represent the set of model parameters of the prompting modules in all federated learning devices, M can represent the number of federated learning devices, F(M, P) can represent the global loss, and p i F can represent the set of model parameters for the prompting module in the i-th federated learning device. i (M,p i ) can represent the local loss in the i-th federated learning device.

[0054] According to embodiments of this disclosure, a large language model obtained by prompting and adjusting using a federated learning system can be applied to handle various natural language processing tasks. Accordingly, for each federated learning device 320, the data samples stored in the federated learning device 320 can consist of text data and labels for the text data. Specifically, for a specific natural language processing task, for example, for a question-answering task, the text data in the data sample can be the question text, and the labels for the text data can be the answer text; as another example, for a machine translation task, the text data in the data sample can be the original text, and the labels for the text data can be the translated text; as yet another example, for a text classification task, the text data in the data sample can be text that meets the formal requirements, and the labels for the text data can be the category of the text.

[0055] According to embodiments of this disclosure, when adjusting prompts for a large language model, the required prompt modules for transmission can be pre-assigned to each federated learning device. Alternatively, to avoid losing global information, all prompt modules can be selected for transmission of model parameters between the server and the federated learning device during the first few training epochs when gradient changes are greatest. After several training epochs, M target prompt modules can be selected from N prompt modules for transmission of model parameters between the server and the federated learning device. By selecting appropriate prompt modules, communication costs can be reduced without compromising model performance.

[0056] Figure 4 A flowchart illustrating a model training method according to another embodiment of the present disclosure is shown schematically.

[0057] like Figure 4 As shown, the method includes operations S401 to S407.

[0058] In operation S401, determine whether the current training round meets the training mode switching conditions. If the current training round meets the training mode switching conditions, execute operation S402. If the current training round does not meet the training mode switching conditions, execute operation S407.

[0059] In operation S402, M target prompting modules corresponding to the federated learning device are determined from N prompting modules.

[0060] In operation S403, the identification information of each of the M target prompt modules is sent to the server.

[0061] In operation S404, the prompts are adjusted using the first model parameters of each of the M target prompt modules issued by the server, and the adjusted second model parameters of each of the M target prompt modules are sent to the server.

[0062] In operation S405, determine whether the training termination condition is met. If the training termination condition is met, execute operation S406. If the training termination condition is not met, proceed to the next training round and return to execute operation S404.

[0063] In operation S406, the prompt adjustment of the large language model is completed.

[0064] In operation S407, the fourth model parameters of each of the N prompt modules issued by the server are used to adjust the prompts, and the adjusted fifth model parameters of each of the N prompt modules are sent to the server. After completing operation S407, the next training round begins, and the process returns to execute operation S401.

[0065] According to embodiments of this disclosure, the training module switching condition can be expressed as the current training round being greater than a preset training round. The preset training round can be set to a small value, such as 5 or 10, and is not limited thereto. Correspondingly, if the current training round meets the training mode switching condition, it can be expressed as the current training round being greater than the preset training round; if the current training round does not meet the training mode switching condition, it can be expressed as the current training round being less than or equal to the preset training round.

[0066] According to embodiments of this disclosure, model parameters for each of the N prompt modules can be received from the server before determining whether the current training epoch meets the training mode switching conditions. When it is determined that the current training epoch does not meet the training mode switching conditions, the model parameters of each prompt module can be fully transmitted between the server and the federated learning device. Specifically, when the current training epoch does not meet the training mode switching conditions, the server receives the fourth model parameters for each of the N prompt modules; uses the fourth model parameters of each of the N prompt modules as the initial parameters of a large language model, trains the large language model using multiple data samples to obtain the fifth model parameters for each of the N prompt modules; and sends the fifth model parameters of each of the N prompt modules to the server, wherein the server is configured to update the model parameters of each of the N prompt modules based on the fifth model parameters of each of the N prompt modules sent by the multiple federated learning devices.

[0067] According to an embodiment of this disclosure, the preset training round can be represented by t. When it is determined that the current training round meets the training mode switching condition, the third model parameters of each of the N prompt modules issued by the server can be represented as the model parameters of each of the N prompt modules integrated in the server after t rounds of training.

[0068] According to embodiments of this disclosure, training termination conditions may include, but are not limited to, total training loss being less than a loss threshold, training gradient being less than a gradient threshold, and the number of training epochs reaching an epoch threshold, etc., and are not limited thereto.

[0069] According to embodiments of this disclosure, when it is determined that the current training round meets the training mode switching conditions, M target prompting modules corresponding to the federated learning device can be determined from the N prompting modules based on multiple data samples, the initial model parameters of each of the N prompting modules, and the third model parameters of each of the N prompting modules.

[0070] According to embodiments of this disclosure, multiple data samples may refer to data samples stored within the federated learning device. The initial model parameters of each of the N prompting modules may refer to the model parameters of the N prompting modules recorded within the federated learning device before the prompting adjustment begins.

[0071] According to embodiments of this disclosure, after determining M target prompting modules corresponding to the federated learning device, the federated learning device can send identification information representing the M target prompting modules to the server. The server can be configured to determine the model parameters to be sent to the federated learning device based on the identification information. That is, after receiving the identification information sent by a federated learning device, the server will only communicate with that federated learning device regarding the parameters of the specific prompting module determined by the identification information during subsequent training. Specifically, before the start of each subsequent round of training, the server only sends the model parameters of the specific prompting module to the federated learning device, the federated learning device performs prompt adjustments normally, and after adjustment, only returns the model parameters of the specific prompting module to the server.

[0072] According to embodiments of this disclosure, by selecting an appropriate set of levels in federated learning, the size of data packets during communication between the server and the federated learning device can be effectively reduced in each round of training, thereby effectively reducing communication costs.

[0073] According to embodiments of this disclosure, specifically, determining M target prompting modules corresponding to the federated learning device from the N prompting modules based on multiple data samples, initial model parameters of each of the N prompting modules, and third model parameters of each of the N prompting modules may include the following operations:

[0074] Based on the initial model parameters and third model parameters of each of the N prompting modules, the pruning rate is determined; based on multiple data samples and the third model parameters of each of the N prompting modules, the importance evaluation results of each of the N prompting modules are determined; and based on the pruning rate and the importance evaluation results of each of the N prompting modules, M target prompting modules corresponding to the federated learning device are determined from the N prompting modules.

[0075] According to embodiments of this disclosure, the pruning rate can be expressed as the minimum ratio of the number of selected model parameters to the total number of model parameters in the N prompting modules. When the ratio between the sum of the model parameters in the selected prompting module and the sum of the model parameters in the N prompting modules is less than this pruning rate, it can be considered that the selected prompting module is insufficient to represent the N prompting modules; that is, if only the currently selected prompting module is used for parameter transmission, there will be information omissions. Correspondingly, when the ratio between the sum of the model parameters in the selected prompting module and the sum of the model parameters in the N prompting modules is greater than or equal to this pruning rate, it can be considered that the selected prompting module can represent the N prompting modules.

[0076] According to embodiments of this disclosure, the importance evaluation result of the prompt module can be represented as a score. The higher the score, the more important the prompt module is for the federated learning device, and the higher the probability that the prompt module will be selected as the target prompt module.

[0077] According to embodiments of this disclosure, by utilizing the pruning rate and the importance evaluation results of each prompting module to select the target prompting module, a set of levels suitable for the federated learning device can be selected from multiple prompting modules without loss, thereby reducing communication costs without affecting the federated learning effect and model performance.

[0078] According to embodiments of this disclosure, specifically, determining the pruning rate based on the initial model parameters of each of the N prompting modules and the third model parameters of each of the N prompting modules may include the following operations:

[0079] Based on the third model parameters of each of the N prompting modules, a Hessian matrix is ​​generated; based on the Hessian matrix, multiple eigenvalues ​​are calculated; based on the Hessian matrix, the initial model parameters of each of the N prompting modules, and the third model parameters of each of the N prompting modules, pruning constraints are determined; and based on the pruning constraints and multiple eigenvalues, the pruning rate is determined.

[0080] According to embodiments of this disclosure, a large language model can be used as a multivariate function, with each third model parameter as a variable, to calculate the second-order partial derivatives of the large language model with respect to each third model parameter, and to construct a Hessian matrix using the calculated multiple second-order partial derivatives, as shown in formula (2):

[0081]

[0082] In formula (2), Let represent the set of model parameters for each of the N prompting modules after t rounds of training, i.e., the set of third model parameters for each of the N prompting modules; i can represent the i-th federated learning device; x a It can represent the a-th third model parameter, and k can represent the total number of third model parameters in the set; f can represent the generated Hessian matrix; f can represent the large language model.

[0083] According to embodiments of this disclosure, eigenvalue extraction of the Hessian matrix can be achieved using singular value decomposition, characteristic polynomial method, etc., and is not limited thereto.

[0084] According to embodiments of this disclosure, pruning constraints are determined based on the Hessian matrix, the initial model parameters of each of the N hint modules, and the third model parameters of each of the N hint modules. Specifically, this may include the following operations:

[0085] Based on the initial model parameters of each of the N prompting modules, the third model parameters of each of the N prompting modules, and the Hessian matrix, basis functions are generated; the Lipshitz constants of the basis functions are calculated; and based on the Lipshitz constants, pruning constraints are determined.

[0086] According to embodiments of this disclosure, based on the initial model parameters of each of the N prompting modules and the third model parameters of each of the N prompting modules, the change in prompting parameters can be determined, as shown in formula (3):

[0087]

[0088] In formula (3), Δ i w can indicate the change in the prompt parameter. i It can represent the set of initial model parameters for each of the N prompt modules.

[0089] According to embodiments of this disclosure, the adjustment gradient can be determined based on the change in the prompting parameters and the third model parameters of each of the N prompting modules, and the Hessian matrix can be adjusted based on the gradient to obtain the basis function based on the change in the prompting parameters, as shown in formula (4):

[0090]

[0091] In formula (4), It can represent basis functions. It can represent the gradient.

[0092] According to embodiments of this disclosure, after determining the basis functions, the constants used by the basis functions to satisfy the Lipshitz conditions in the real number domain can be calculated to obtain the Lipshitz constants of the basis functions. The specific calculation process for the Lipshitz constants is not limited here.

[0093] According to embodiments of this disclosure, for multiple eigenvalues ​​obtained based on the Hessian matrix, the multiple eigenvalues ​​can be arranged in ascending order to obtain an eigenvalue sequence, which can be represented as follows: It can represent the smallest eigenvalue. This can represent the second smallest eigenvalue, and so on. K i It can represent the number of eigenvalues ​​determined based on the Hessian matrix.

[0094] According to embodiments of this disclosure, the pruning constraint condition determined based on the Lipshitz constant can be defined as the difference between adjacent eigenvalues ​​in the eigenvalue sequence being greater than an integer multiple of the Lipshitz constant. For example, this integer multiple can be chosen as 4 times. Then, the pruning constraint condition can be as shown in formula (5):

[0095]

[0096] In formula (5), It can represent the m-th eigenvalue in the eigenvalue sequence. It can represent the Lipschitz constant.

[0097] According to embodiments of this disclosure, determining the pruning rate based on pruning constraints and multiple eigenvalues ​​may include the following operations:

[0098] Arrange multiple eigenvalues ​​in ascending order to obtain an eigenvalue sequence; determine the target eigenvalue that satisfies the pruning constraint from the eigenvalue sequence to obtain the index of the target eigenvalue; and obtain the pruning rate based on the index of the target eigenvalue and the number of eigenvalues ​​included in the eigenvalue sequence.

[0099] According to embodiments of this disclosure, for each eigenvalue in the eigenvalue sequence, it can be determined whether the eigenvalue satisfies the pruning constraint condition, that is, whether the difference between the eigenvalue and the next eigenvalue is greater than 4 times the Lipshitz constant. For the j-th eigenvalue, if the j-th eigenvalue is the first eigenvalue to satisfy the pruning constraint condition, then the j-th eigenvalue can be determined as the target eigenvalue, and the corresponding index of the target eigenvalue is j.

[0100] According to embodiments of this disclosure, the pruning rate can be obtained based on the index of the target feature value and the number of feature values ​​included in the feature value sequence, as shown in formula (6):

[0101]

[0102] In formula (6), r i It can represent the pruning rate.

[0103] According to embodiments of this disclosure, by utilizing the pruning rate to select the target prompting module, the information loss caused by some transmission model parameters and the impact on federated learning performance can be effectively reduced.

[0104] According to embodiments of this disclosure, the order of operations between determining the pruning rate and determining the importance evaluation results of each of the N prompting modules is not limited, and the two operations can be executed synchronously or asynchronously, which is not limited here.

[0105] According to embodiments of this disclosure, specifically, determining the importance evaluation results of each of the N prompting modules based on multiple data samples and the third model parameters of each of the N prompting modules may include the following operations:

[0106] For each data sample, a data mapping is performed based on a preset activation function to obtain the hidden states of each of the N prompt modules related to the data sample; and based on the hidden states of each of the N prompt modules related to multiple data samples, the importance evaluation results of each of the N prompt modules are determined.

[0107] According to embodiments of this disclosure, the preset activation function can refer to the activation function used when obtaining the model parameters of each prompting module. Based on this preset activation function, the hidden state of each model parameter can be determined. Specifically, for data sample s... i This data sample s i Mapped to a large language model, the q-th model parameter of the l-th prompt module and the data sample s can be determined using a preset activation function fp. i The corresponding hidden state can be represented as fp. q，l (s i Accordingly, for the first prompt module, the l-th prompt module is related to the data sample s. i The relevant hidden state can be represented as K l This can represent the number of model parameters included in the l-th prompt module.

[0108] According to embodiments of this disclosure, since the difficulty of distinguishing input samples during model training is positively correlated with the similarity of hidden states, the correlation between hidden states can be used to evaluate the performance of the model.

[0109] According to embodiments of this disclosure, determining the importance evaluation result of each of the N prompting modules based on the hidden states associated with each of the N prompting modules and multiple data samples can include the following operations:

[0110] For the hint module, the importance score of the hint module is obtained based on the similarity between the hidden state of the hint module and the hidden states of the other N-1 hint modules among the N hint modules; and the importance evaluation result of the hint module is obtained based on the importance scores of the multiple hint modules.

[0111] According to embodiments of this disclosure, for each data sample, a similarity matrix can be constructed based on the cosine similarity between the hidden states of every two cue modules, as shown in formula (7):

[0112]

[0113] In formula (7), cos(h) i，L h i，1 The cosine similarity () between the hidden state of the Lth hint module and the hidden state of the 1st hint module can be represented. It can represent a similarity matrix.

[0114] According to embodiments of this disclosure, for the similarity matrix, multiple similarity feature values ​​can be calculated, and the calculation method for the similarity feature values ​​is not limited herein. The calculated multiple similarity feature values ​​can be represented as {λ i，1 , λ i，2 ,...,λ i，L}, where λ i，1 λ can represent the similarity feature value corresponding to the first prompt module. i，2 It can represent the similarity feature value corresponding to the second prompt module, and so on.

[0115] According to embodiments of this disclosure, calculations and data samples s can be employed. i The calculation method for multiple similarity features is similar, yielding multiple similarity features related to other data samples. Based on the calculated similarity features, the number of similarity features included in the i-th federated learning device can be obtained. i For each data sample, the similarity feature value of the l-th prompt module can be represented as {λ}. 1，l , λ 2，l ,...,λ ni，l Based on the similarity feature value of the l-th prompt module, the importance score of the l-th prompt module can be calculated, as shown in formula (8):

[0116]

[0117] In formula (8), ζ i，l This represents the importance score of the l-th prompt module; λ j，l It can represent the j-th similarity feature value of the l-th prompt module; ∈ can represent a small positive number, for example, ∈ can take the value 1×e -5 By using the importance score calculation method shown in Formula (8), the performance loss caused by outliers in the similarity feature values ​​can be avoided.

[0118] According to embodiments of this disclosure, the global score of each prompt module, i.e., the importance evaluation result of the prompt module, can be determined based on the importance scores of each of the multiple prompt modules, as shown in formula (9):

[0119]

[0120] In formula (9), Y i The importance evaluation result of the i-th prompt module can be represented, and N can represent the total number of data samples stored by each of the multiple federated learning devices.

[0121] According to embodiments of this disclosure, the importance evaluation results obtained in the above manner can explicitly represent the importance of each prompting module in achieving the goal of distinguishing data samples, thereby providing data reference for the subsequent selection process of target prompting modules.

[0122] According to embodiments of this disclosure, after determining the importance evaluation results and pruning rates of each prompting module in each federated learning device, the target prompting module of the federated learning device can be selected. This selection process can be performed by the federated learning device itself, or it can be performed by the server after the federated learning device sends the importance evaluation results and pruning rates of each prompting module to the server; no limitation is made herein.

[0123] According to embodiments of this disclosure, specifically, determining M target cue modules corresponding to the federated learning device from the N cue modules based on the pruning rate and the importance evaluation results of each of the N cue modules may include the following operations:

[0124] Based on the number of parameters of the model parameters of each of the N prompting modules, the number and value of the first parameter are obtained; based on the importance evaluation results of each of the N prompting modules, a set of prompting modules is determined from the N prompting modules; based on the number of parameters of the model parameters of each of the n prompting modules included in the prompting module set, the number and value of the second parameter are obtained, where N≥n≥1; if the ratio between the number and value of the second parameter and the number and value of the first parameter is less than the pruning rate, a new set of prompting modules is determined based on the importance evaluation results of each of the N prompting modules and the set of prompting modules; and if the ratio between the number and value of the second parameter and the number and value of the first parameter is greater than or equal to the pruning rate, M target prompting modules are obtained based on the set of prompting modules.

[0125] According to embodiments of this disclosure, the first parameter quantity and value can be represented as the sum of the number of parameters of the model parameters of each of the N prompting modules. The second parameter quantity and value can be represented as the sum of the number of parameters of the model parameters of each of the n currently existing prompting modules in the prompting module set.

[0126] Figure 5 A flowchart illustrating a method for determining a target prompting module according to an embodiment of the present disclosure is shown schematically.

[0127] like Figure 5 As shown, the method includes operations S501 to S508.

[0128] In operation S501, based on the number of parameters of the model parameters of each of the N prompting modules, the number and value of the first parameter are obtained.

[0129] In operation S502, N prompt modules can be arranged in descending order based on their respective importance evaluation results to obtain an importance sequence.

[0130] In operation S503, initialize t=1 and initialize the prompt module set to an empty set.

[0131] In operation S504, based on the number of parameters of the model parameters of the prompt modules included in the prompt module set, the number and value of the second parameter are obtained.

[0132] In operation S505, the ratio between the number and value of the second parameter and the number and value of the first parameter is calculated.

[0133] In operation S506, determine if the ratio is less than the pruning rate. If the ratio is less than the pruning rate, proceed to operation S507. If the ratio is greater than or equal to the pruning rate, proceed to operation S508.

[0134] In operation S507, the new set of prompt modules is determined to be the intersection of the set of prompt modules and the t-th prompt module in the importance sequence, and t is incremented by 1. After completing operation S507, return to execute operation S504.

[0135] In operation S508, the prompt modules included in the prompt module set are determined to be the target prompt modules.

[0136] According to embodiments of this disclosure, the initial set of prompt modules can be an empty set. Correspondingly, the number and value of the second parameter can be 0, and the ratio between the number and value of the second parameter and the number and value of the first parameter is 0. Clearly, this ratio is less than the pruning rate. At this point, the prompt module that ranks first in the importance sequence can be added to the prompt module set to obtain a new prompt module. That is, the new prompt module set can be represented as the union of the original prompt module set and the prompt module to be added. After determining the new prompt module set, the new number and value of the second parameter can be calculated, which is the number of parameters of the model parameters of the prompt module with the highest importance evaluation result. The ratio between the number and value of the second parameter and the number and value of the first parameter can be compared with the pruning rate. When it is determined that the ratio between the number and value of the second parameter and the number and value of the first parameter is less than the pruning rate, a prompt module can be added to the prompt module set based on the importance evaluation result, and the calculation of the number and value of the second parameter and the comparison of the ratio with the pruning rate can be performed in the next round. When the ratio between the number and value of the second parameter and the number and value of the first parameter is greater than or equal to the pruning rate, it can be determined that the prompt modules in the current prompt module set are sufficient to represent the other prompt modules. At this time, all prompt modules in the prompt module set can be used as the selected target prompt modules.

[0137] Figure 6 A block diagram of a model training apparatus according to an embodiment of the present disclosure is shown schematically.

[0138] like Figure 6 As shown, the model training device 600 may include a first receiving module 610, a first training module 620, and a first sending module 630.

[0139] The first receiving module 610 is used to receive the first model parameters of each of the M target prompting modules sent by the server. The large language model includes N prompting modules, and the M target prompting modules belong to the N prompting modules, where N≥M≥1.

[0140] The first training module 620 is used to train the large language model using the first model parameters of each of the M target prompting modules and the current model parameters of the remaining NM prompting modules in the N prompting modules as the initial parameters of the large language model, and to obtain the second model parameters of each of the M target prompting modules.

[0141] The first sending module 630 is used to send the second model parameters of each of the M target prompting modules to the server. The server is configured to update the model parameters of each of the N prompting modules based on the second model parameters of each of the M target prompting modules sent by the multiple federated learning devices included in the federated learning system.

[0142] According to embodiments of this disclosure, the model training apparatus 600 further includes a second receiving module, a determining module, and a second sending module.

[0143] The second receiving module is used to receive the third model parameters of each of the N prompt modules sent by the server.

[0144] The determination module is used to determine M target prompting modules corresponding to the federated learning device from the N prompting modules, based on multiple data samples, the initial model parameters of each of the N prompting modules, and the third model parameters of each of the N prompting modules, when the training mode switching conditions are met in the current training round.

[0145] The second sending module is used to send identification information representing M target prompting modules to the server, wherein the server is configured to determine the model parameters to be sent to the federated learning device based on the identification information.

[0146] According to embodiments of this disclosure, the determining module includes a first determining submodule, a second determining submodule, and a third determining submodule.

[0147] The first determining submodule is used to determine the pruning rate based on the initial model parameters of each of the N prompting modules and the third model parameters of each of the N prompting modules.

[0148] The second determination submodule is used to determine the importance evaluation results of each of the N prompt modules based on multiple data samples and the third model parameters of each of the N prompt modules.

[0149] The third determination submodule is used to determine M target cue modules corresponding to the federated learning device from the N cue modules based on the pruning rate and the importance evaluation results of each of the N cue modules.

[0150] According to embodiments of this disclosure, the first determining submodule includes a first determining unit, a second determining unit, a third determining unit, and a fourth determining unit.

[0151] The first determining unit is used to generate the Hessian matrix based on the third model parameters of each of the N prompting modules.

[0152] The second determining unit is used to calculate multiple eigenvalues ​​based on the Hessian matrix.

[0153] The third determining unit is used to determine the pruning constraints based on the Hessian matrix, the initial model parameters of each of the N hint modules, and the third model parameters of each of the N hint modules.

[0154] The fourth determining unit is used to determine the pruning rate based on pruning constraints and multiple eigenvalues.

[0155] According to embodiments of this disclosure, the third determining unit includes a first determining subunit, a second determining subunit, and a third determining subunit.

[0156] The first determining sub-unit is used to generate basis functions based on the initial model parameters of each of the N prompting modules, the third model parameters of each of the N prompting modules, and the Hessian matrix.

[0157] The second determining subunit is used to calculate the Lipschitz constant of the basis functions.

[0158] The third determining sub-unit is used to determine the pruning constraints based on the Lipschitz constant.

[0159] According to embodiments of this disclosure, the fourth determining unit includes a fourth determining subunit, a fifth determining subunit, and a sixth determining subunit.

[0160] The fourth sub-unit is used to arrange multiple feature values ​​in ascending order to obtain a feature value sequence.

[0161] The fifth determining subunit is used to determine the target eigenvalue that satisfies the pruning constraint from the eigenvalue sequence, and obtain the index of the target eigenvalue.

[0162] The sixth sub-unit is used to determine the pruning rate based on the index of the target feature value and the number of feature values ​​included in the feature value sequence.

[0163] According to embodiments of this disclosure, the second determining submodule includes a fifth determining unit and a sixth determining unit.

[0164] The fifth determining unit is used to perform data mapping on the data sample based on a preset activation function to obtain the hidden states of N prompt modules related to the data sample.

[0165] The sixth determining unit is used to determine the importance evaluation results of each of the N prompting modules based on the hidden states of each of the N prompting modules and their respective association with multiple data samples.

[0166] According to embodiments of this disclosure, the sixth determining unit includes a seventh determining subunit and an eighth determining subunit.

[0167] The seventh determining subunit is used to obtain the importance score of the prompt module based on the similarity between the hidden state of the prompt module and the hidden states of the other N-1 prompt modules among the N prompt modules.

[0168] The eighth sub-unit is used to obtain the importance evaluation result of the prompt module based on the importance scores of each of the multiple prompt modules.

[0169] According to embodiments of this disclosure, the third determining submodule includes a seventh determining unit, an eighth determining unit, a ninth determining unit, a tenth determining unit, and an eleventh determining unit.

[0170] The seventh determining unit is used to obtain the number and value of the first parameter based on the number of parameters of the model parameters of each of the N prompting modules.

[0171] The eighth determining unit is used to determine the set of prompt modules from the N prompt modules based on the importance evaluation results of each of the N prompt modules.

[0172] The ninth determining unit is used to obtain the number and value of the second parameter based on the number of parameters of the model parameters of each of the n prompting modules included in the prompting module set, where N≥n≥1.

[0173] The tenth determining unit is used to determine a new set of prompting modules based on the importance evaluation results of each of the N prompting modules and the set of prompting modules when the ratio between the number and value of the second parameter and the number and value of the first parameter is less than the pruning rate.

[0174] The eleventh determining unit is used to obtain M target prompting modules based on the prompting module set when the ratio between the number and value of the second parameter and the number and value of the first parameter is greater than or equal to the pruning rate.

[0175] According to embodiments of this disclosure, the model training apparatus 600 further includes a third receiving module, a second training module, and a third transmitting module.

[0176] The third receiving module is used to receive the fourth model parameters of each of the N prompting modules sent by the server when the current training round does not meet the conditions for switching the training mode.

[0177] The second training module is used to train the large language model using the fourth model parameters of each of the N prompting modules as the initial parameters of the large language model, and to obtain the fifth model parameters of each of the N prompting modules.

[0178] The third sending module is used to send the fifth model parameters of each of the N prompting modules to the server. The server is configured to update the model parameters of each of the N prompting modules based on the fifth model parameters sent by each of the multiple federated learning devices.

[0179] According to embodiments of this disclosure, the data sample consists of text data and tags for the text data.

[0180] According to embodiments of this disclosure, the large language model also includes a model ontology, which includes N network layers and N prompting modules, each corresponding to one of the N network layers.

[0181] In the process of training a large language model using multiple data samples stored in the federated learning device, the model parameters of the model ontology are set to a frozen state.

[0182] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0183] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0184] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform the method described above.

[0185] According to an embodiment of this disclosure, a computer program product includes a computer program that, when executed by a processor, implements the method described above.

[0186] Figure 7A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0187] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0188] Multiple components in device 700 are connected to input / output (I / O) interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0189] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as model training methods. For example, in some embodiments, the model training method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the model training method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform model training methods by any other suitable means (e.g., by means of firmware).

[0190] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0191] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0192] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0193] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0194] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0195] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0196] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0197] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A model training method, comprising: The system receives the first model parameters of each of the M target prompting modules sent by the server. The large language model includes N prompting modules, and the M target prompting modules belong to the N prompting modules. The M target prompting modules associated with the federated learning device are determined from the N prompting modules based on the pruning rate and the importance of each prompting module, where N ≥ M ≥ 1. Determining the pruning rate includes: generating a Hessian matrix based on the third model parameters of each of the N prompting modules; calculating multiple eigenvalues ​​based on the Hessian matrix; determining pruning constraints based on the Hessian matrix, the initial model parameters of each of the N prompting modules, and the third model parameters of each of the N prompting modules; and determining the pruning rate based on the pruning constraints and the multiple eigenvalues. Using the first model parameters of each of the M target prompting modules and the current model parameters of the remaining NM prompting modules among the N prompting modules as the initial parameters of the large language model, the large language model is trained using multiple data samples stored in the federated learning device to obtain the second model parameters of each of the M target prompting modules. The data samples include text data and text data labels; and The server sends the second model parameters of each of the M target prompting modules to the server, wherein the server is configured to update the model parameters of each of the N prompting modules based on the second model parameters of each of the M target prompting modules sent by the multiple federated learning devices included in the federated learning system.

2. The method according to claim 1, further comprising: Receive the third model parameters of each of the N prompting modules sent by the server; If the training mode switching conditions are met in the current training round, based on the multiple data samples, the initial model parameters of each of the N prompting modules, and the third model parameters of each of the N prompting modules, the M target prompting modules corresponding to the federated learning device are determined from the N prompting modules. as well as The server is sent identification information representing the M target prompting modules, wherein the server is configured to determine the model parameters to be issued to the federated learning device based on the identification information.

3. The method according to claim 2, wherein, The step of determining the M target prompting modules corresponding to the federated learning device from the N prompting modules based on the multiple data samples, the initial model parameters of each of the N prompting modules, and the third model parameters of each of the N prompting modules includes: The pruning rate is determined based on the initial model parameters of each of the N prompting modules and the third model parameters of each of the N prompting modules; Based on the multiple data samples and the third model parameters of each of the N prompting modules, the importance evaluation results of each of the N prompting modules are determined; and Based on the pruning rate and the importance evaluation results of the N prompting modules, the M target prompting modules corresponding to the federated learning device are determined from the N prompting modules.

4. The method according to claim 3, wherein, The pruning constraints are determined based on the Hessian matrix, the initial model parameters of each of the N hint modules, and the third model parameters of each of the N hint modules, including: Based on the initial model parameters of each of the N prompting modules, the third model parameters of each of the N prompting modules, and the Hessian matrix, a basis function is generated; The Lipschitz constants of the basis functions were calculated; and The pruning constraints are determined based on the Lipschitz constant.

5. The method according to claim 3, wherein, Determining the pruning rate based on the pruning constraints and the multiple feature values ​​includes: Arrange the multiple feature values ​​in ascending order to obtain a feature value sequence; From the eigenvalue sequence, determine the target eigenvalues ​​that satisfy the pruning constraints, and obtain the index of the target eigenvalues; and The pruning rate is obtained based on the index of the target feature value and the number of feature values ​​included in the feature value sequence.

6. The method according to claim 3, wherein, The determination of the importance evaluation results for each of the N prompting modules based on the multiple data samples and the third model parameters of each of the N prompting modules includes: For the data sample, a data mapping is performed on the data sample based on a preset activation function to obtain the hidden states of each of the N prompting modules related to the data sample; and Based on the hidden states of each of the N hint modules and the data samples, the importance evaluation results of each of the N hint modules are determined.

7. The method according to claim 6, wherein, The determination of the importance evaluation result of each of the N hint modules based on the hidden states associated with each of the N hint modules and the multiple data samples includes: For the aforementioned hint module, an importance score is obtained based on the similarity between the hidden state of the hint module and the hidden states of the other N-1 hint modules out of the N hint modules; and Based on the importance scores of each of the multiple prompting modules, the importance evaluation result of the prompting module is obtained.

8. The method according to claim 3, wherein, The process of determining the M target cue modules corresponding to the federated learning device from the N cue modules based on the pruning rate and the importance evaluation results of each of the N cue modules includes: Based on the number of parameters of each of the N prompting modules' model parameters, the number and value of the first parameter are obtained; Based on the importance evaluation results of each of the N prompting modules, a set of prompting modules is determined from the N prompting modules; Based on the number of parameters of the model parameters of each of the n prompting modules included in the prompting module set, the number and value of the second parameter are obtained, where N≥n≥1; If the ratio between the number and value of the second parameter and the number and value of the first parameter is less than the pruning rate, a new set of prompting modules is determined based on the importance evaluation results of each of the N prompting modules and the set of prompting modules; and If the ratio between the number and value of the second parameter and the number and value of the first parameter is greater than or equal to the pruning rate, the M target prompting modules are obtained based on the set of prompting modules.

9. The method according to claim 2, further comprising: If the current training round does not meet the conditions for switching training modes, the fourth model parameters of each of the N prompting modules sent by the server are received. Using the fourth model parameters of each of the N prompting modules as the initial parameters of the large language model, the large language model is trained using the multiple data samples to obtain the fifth model parameters of each of the N prompting modules; as well as The server sends the fifth model parameters of each of the N prompting modules to the server, wherein the server is configured to update the model parameters of each of the N prompting modules based on the fifth model parameters of each of the N prompting modules sent by the plurality of federated learning devices.

10. The method according to any one of claims 1 to 9, wherein, The data sample consists of text data and labels for the text data.

11. The method according to any one of claims 1 to 9, wherein, The large language model also includes a model ontology, which includes N network layers, and each of the N network layers corresponds to one of the N prompting modules. During the training of the large language model using multiple data samples stored in the federated learning device, the model parameters of the model ontology are set to a frozen state.

12. A model training device, comprising: A first receiving module is used to receive the first model parameters of each of the M target prompting modules sent by the server. The large language model includes N prompting modules, and the M target prompting modules belong to the N prompting modules. The M target prompting modules related to the federated learning device are determined from the N prompting modules based on the pruning rate and the importance of each prompting module, where N ≥ M ≥ 1. A first determining submodule is used to determine the pruning rate. The first determining submodule includes: a first determining unit, used to generate a Hessian matrix based on the third model parameters of each of the N prompting modules; a second determining unit, used to calculate multiple eigenvalues ​​based on the Hessian matrix; a third determining unit, used to determine pruning constraints based on the Hessian matrix, the initial model parameters of each of the N prompting modules, and the third model parameters of each of the N prompting modules; and a fourth determining unit, used to determine the pruning rate based on the pruning constraints and the multiple eigenvalues. A first training module is used to train the large language model using the first model parameters of each of the M target prompting modules and the current model parameters of the remaining NM prompting modules out of the N prompting modules as initial parameters, and using multiple data samples stored in a federated learning device to obtain the second model parameters of each of the M target prompting modules. The data samples include text data and text data labels. The first sending module is used to send the second model parameters of each of the M target prompting modules to the server, wherein the server is configured to update the model parameters of each of the N prompting modules based on the second model parameters of each of the M target prompting modules sent by the multiple federated learning devices included in the federated learning system.

13. The apparatus of claim 12, further comprising: The second receiving module is used to receive the third model parameters of each of the N prompting modules sent by the server; The determination module is used to determine the M target prompting modules corresponding to the federated learning device from the N prompting modules based on the multiple data samples, the initial model parameters of each of the N prompting modules, and the third model parameters of each of the N prompting modules, when the training mode switching conditions are met in the current training round. as well as The second sending module is used to send identification information representing the M target prompting modules to the server, wherein the server is configured to determine the model parameters to be sent to the federated learning device based on the identification information.

14. The apparatus according to claim 13, wherein, The determining module includes: The first determining submodule is used to determine the pruning rate based on the initial model parameters of each of the N prompting modules and the third model parameters of each of the N prompting modules; The second determining submodule is used to determine the importance evaluation result of each of the N prompting modules based on the multiple data samples and the third model parameters of each of the N prompting modules; and The third determining submodule is used to determine the M target prompting modules corresponding to the federated learning device from the N prompting modules based on the pruning rate and the importance evaluation results of the N prompting modules.

15. The apparatus according to claim 14, wherein, The third determining unit includes: The first determining subunit is used to generate a basis function based on the initial model parameters of each of the N prompting modules, the third model parameters of each of the N prompting modules, and the Hessian matrix; The second determining subunit is used to calculate the Lipschitz constants of the basis functions; and The third determining subunit is used to determine the pruning constraints based on the Lipschitz constant.

16. The apparatus according to claim 14, wherein, The fourth determining unit includes: The fourth determining subunit is used to arrange the plurality of feature values ​​in ascending order to obtain a feature value sequence; The fifth determining subunit is used to determine the target feature value that satisfies the pruning constraint condition from the feature value sequence, and to obtain the index of the target feature value; and The sixth determining subunit is used to obtain the pruning rate based on the index of the target feature value and the number of feature values ​​included in the feature value sequence.

17. The apparatus according to claim 14, wherein, The second determining submodule includes: The fifth determining unit is used to perform data mapping on the data sample based on a preset activation function to obtain the hidden states of each of the N prompting modules related to the data sample; and The sixth determining unit is used to determine the importance evaluation result of each of the N prompting modules based on the hidden states associated with each of the N prompting modules and the data samples.

18. The apparatus according to claim 17, wherein, The sixth determining unit includes: The seventh determining subunit is used to, for the given prompt module, obtain an importance score for the prompt module based on the similarity between the hidden state of the prompt module and the hidden states of the other N-1 prompt modules among the N prompt modules; and The eighth determining subunit is used to obtain the importance evaluation result of the prompting module based on the importance scores of each of the multiple prompting modules.

19. The apparatus according to claim 14, wherein, The third determining submodule includes: The seventh determining unit is used to obtain the number and value of the first parameter based on the number of parameters of the model parameters of each of the N prompting modules; The eighth determining unit is used to determine a set of prompt modules from the N prompt modules based on the importance evaluation results of each of the N prompt modules; The ninth determining unit is used to obtain the number and value of the second parameter based on the number of parameters of the model parameters of each of the n prompting modules included in the prompting module set, where N≥n≥1; The tenth determining unit is configured to determine a new set of prompting modules based on the importance evaluation results of each of the N prompting modules and the set of prompting modules when the ratio between the number and value of the second parameter and the number and value of the first parameter is less than the pruning rate; and The eleventh determining unit is used to obtain the M target prompting modules based on the set of prompting modules when the ratio between the number and value of the second parameter and the number and value of the first parameter is greater than or equal to the pruning rate.

20. The apparatus of claim 13, further comprising: The third receiving module is used to receive the fourth model parameters of each of the N prompting modules sent by the server when the current training round does not meet the training mode switching conditions. The second training module is used to train the large language model using the fourth model parameters of each of the N prompting modules as the initial parameters of the large language model, and to obtain the fifth model parameters of each of the N prompting modules. as well as The third sending module is used to send the fifth model parameters of each of the N prompting modules to the server, wherein the server is configured to update the model parameters of each of the N prompting modules based on the fifth model parameters of each of the N prompting modules sent by the plurality of federated learning devices.

21. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

23. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images