Federated learning based large language model pre-training method and apparatus

Abstract

The application provides a large language model pre-training method and device based on federated learning, involving a server and multiple participants, a method executed by a first participant comprising: for each target position in each first token sequence of a first training sample set, inputting tokens before the target position as context into a first sub-model and a representation measurement model respectively, the first sub-model predicting a first probability distribution of a token at the target position, the representation measurement model predicting a second probability distribution of the token at the target position, the representation measurement model being derived based on parameter sharing of the first sub-model; calculating a difference loss based on a joint loss function, updating model parameters of the first sub-model based on the difference loss; sending the model parameters of the first sub-model to the server, determining global model parameters based on model parameters uploaded by the multiple participants and sending feedback model parameters to each participant by the server; updating the model parameters of the first sub-model based on the first feedback model parameters sent by the server.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for pre-training large language models based on federated learning. Background Technology

[0002] With the rapid development of large-scale language models (LLMs), self-supervised pre-training based on massive amounts of unlabeled data has become a core technical approach to improve the general representation capabilities of models and the performance of downstream tasks. Large language model pre-training refers to training large language models on large-scale unlabeled corpora through self-supervised learning tasks, enabling them to learn the statistical regularities, semantic structures, and contextual dependencies of language. This results in generalized language representations with good generalization ability and provides effective initialization parameters and semantic representations for downstream natural language processing tasks. Current mainstream large language model pre-training methods primarily rely on Maximum Likelihood Estimation (MLE), which learns the statistical dependencies between lexical units by minimizing the language modeling loss.

[0003] In distributed training scenarios, federated learning (FL) provides an effective privacy-preserving framework for collaborative modeling of multi-party data. By training models on local data and aggregating them on a server, federated learning can achieve knowledge fusion without sharing the original data. However, in large-scale model pre-training scenarios, due to significant differences in the data distribution among participants and the imbalance in data scale and category distribution, traditional federated learning methods often struggle to balance model performance with fairness among participants. Therefore, how to achieve joint model training that balances performance and fairness in an unsupervised pre-training environment using a federated learning framework is a pressing issue that needs to be addressed. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method and apparatus for pre-training large language models based on federated learning, so as to eliminate or improve one or more defects existing in the prior art.

[0005] According to the first aspect, a pre-training method for a large language model based on federated learning is provided. This method involves a server and multiple participants. The method is executed by a first participant among the multiple participants. The large language model localized by the first participant is a first sub-model. The first sub-model is either an initial sub-model or a sub-model updated with parameters based on feedback model parameters sent by the server in the previous training round. The method includes: acquiring a first training sample set, wherein the first training sample set includes multiple first word sequences; for each target position in each first word sequence, taking the words preceding the target position as context, inputting the context into the first sub-model and a pre-constructed representation measurement model respectively, predicting a first probability distribution of words at the target position by the first sub-model, and predicting a second probability distribution of words at the target position by the representation measurement model, wherein the representation measurement model is based on the parameters of the first sub-model. The model is derived from shared data; a difference loss is calculated based on a pre-constructed joint loss function, and the model parameters of the first sub-model are updated based on the difference loss. The joint loss function includes a first loss term and a second loss term. The first loss term is determined based on the first probability distribution and the true word at the target position, and the second loss term is determined based on the first probability distribution and the second probability distribution. The model parameters of the first sub-model are sent to the server, which determines the global model parameters based on the model parameters uploaded by the multiple participants, and determines the feedback model parameters to be fed back to each participant based on the global model parameters and the model parameters uploaded by each participant. The server then sends the first feedback model parameters to the first participant, where the feedback model parameters are different for different participants. The server receives the first feedback model parameters sent by the server and updates the model parameters of the first sub-model based on the first feedback model parameters.

[0006] According to the second aspect, another method for pre-training a large language model based on federated learning is provided. This method involves a server and multiple participants. The method is executed by the server and includes: receiving model parameters of sub-models uploaded by the multiple participants. The first participant among the multiple participants trains the first sub-model in the following manner: for each target position in each first word sequence in its first training sample set, the words preceding the target position are used as context. The context is input into the first sub-model and a pre-built representation measurement model, respectively. The first sub-model predicts the first probability distribution of the words at the target position, and the representation measurement model predicts the first probability distribution of the words at the target position. The model predicts the second probability distribution of words at the target position, wherein the representation measurement model is derived based on the parameter sharing of the first sub-model; the difference loss is calculated based on the pre-constructed joint loss function, and the model parameters of the first sub-model are updated based on the difference loss; the global model parameters are determined based on the model parameters of the sub-models uploaded by the multiple participants; the feedback model parameters are calculated based on the global model parameters and the model parameters uploaded by each participant, wherein the feedback model parameters of different participants are different; the feedback model parameters of each participant are sent to each participant so that each participant updates the model parameters of its local sub-model based on the received feedback model parameters.

[0007] According to a third aspect, a pre-training device for a large language model based on federated learning is provided. Federated learning involves a server and multiple participants. The device is deployed on a first participant, whose local large language model is a first sub-model. This first sub-model is either an initial sub-model or a sub-model updated based on feedback model parameters sent by the server in the previous training round. The device includes: an acquisition unit for acquiring a first training sample set, wherein the first training sample set includes multiple first word sequences; and an input unit for, for each target position in each first word sequence, taking the words preceding the target position as context, inputting the context into the first sub-model and a pre-constructed representation measurement model respectively, whereby the first sub-model predicts a first probability distribution of words at the target position, and the representation measurement model predicts a second probability distribution of words at the target position, wherein the representation measurement model is derived based on parameter sharing of the first sub-model. The system comprises: a calculation unit for calculating the difference loss based on a pre-constructed joint loss function, and updating the model parameters of the first sub-model based on the difference loss; wherein the joint loss function includes a first loss term and a second loss term, the first loss term being determined based on the first probability distribution and the true word at the target position, and the second loss term being determined based on the first probability distribution and the second probability distribution; a sending unit for sending the model parameters of the first sub-model to the server, wherein the server determines global model parameters based on the model parameters uploaded by the multiple participants, and determines feedback model parameters to be fed back to each participant based on the global model parameters and the model parameters uploaded by each participant, and sends the first feedback model parameters to the first participant, wherein the feedback model parameters of different participants are different; and an updating unit for receiving the first feedback model parameters sent by the server and updating the model parameters of the first sub-model based on the first feedback model parameters.

[0008] According to the fourth aspect, another pre-training device for a large language model based on federated learning is provided. Federated learning involves a server and multiple participants. The device is deployed on the server and includes a parameter receiving unit for receiving model parameters of sub-models uploaded by the multiple participants. The first participant among the multiple participants trains the first sub-model in the following manner: for each target position in each first word sequence in its first training sample set, the words preceding the target position are used as context. The context is input into the first sub-model and a pre-built representation measurement model, respectively. The first sub-model predicts the first probability distribution of words at the target position, and the representation measurement model predicts the probability distribution of words at the target position. The second probability distribution of the lexical units, wherein the above-mentioned representation measurement model is derived based on the parameter sharing of the first sub-model; the difference loss is calculated based on the pre-constructed joint loss function, and the model parameters of the first sub-model are updated based on the difference loss; the parameter aggregation unit is used to determine the global model parameters based on the model parameters of the sub-models uploaded by the multiple participants; the parameter calculation unit is used to calculate the feedback model parameters fed back to each participant based on the global model parameters and the model parameters uploaded by each participant, wherein the feedback model parameters of different participants are different; the parameter sending unit is used to send the feedback model parameters of each participant to each participant, so that each participant updates the model parameters of its local sub-model based on the received feedback model parameters.

[0009] According to a fifth aspect, a computing device is provided, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the computing device implements the method described in either the first or second aspect.

[0010] According to a sixth aspect, a computer-readable storage medium is provided that stores a computer program / instructions thereon, wherein the computer program / instructions, when executed by a processor, implement the method as described in any implementation of the first or second aspect.

[0011] The large language model pre-training method and apparatus based on federated learning provided in the embodiments of this specification involve a server and multiple participants. The local large language model of the first participant is a first sub-model. This first sub-model can be an initial sub-model or a sub-model with updated parameters based on the feedback model parameters sent by the server in the previous training round. The first participant can perform the following pre-training steps: First, obtain a first training sample set, which includes multiple first word sequences. Second, for each target position in each first word sequence, use the words before the target position as context, and input the context into the first sub-model and a pre-built representation measurement model respectively. The first sub-model predicts the first probability distribution of the words at the target position, and the representation measurement model predicts the second probability distribution of the words at the target position. The representation measurement model is derived based on the parameter sharing of the first sub-model. Next, the difference loss is calculated based on the pre-constructed joint loss function, and the model parameters of the first sub-model are updated based on the difference loss. The joint loss function can include a first loss term and a second loss term. The first loss term can be determined based on a first probability distribution and the true word units at the target position, and the second loss term can be determined based on both the first and second probability distributions. Then, the model parameters of the first sub-model are sent to the server. The server determines the global model parameters based on the model parameters uploaded by multiple participants, and determines the feedback model parameters to be fed back to each participant based on the global model parameters and the model parameters uploaded by each participant. The server then sends the first feedback model parameters to the first participant, where the feedback model parameters differ between participants. Finally, the first participant receives the first feedback model parameters sent by the server and updates the model parameters of the first sub-model based on them. Thus, without introducing additional supervision or interaction, the first participant guides the first sub-model to learn more accurate representations through the representation measurement model, thereby improving the generalization ability and subsequent training effect of the first sub-model. Furthermore, the server sends different feedback model parameters to different participants, ensuring fairness in the federated learning process.

[0012] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0013] Those skilled in the art will understand that the objectives and advantages achievable with this invention are not limited to those specifically described above, and that the above and other objectives achievable with this invention will become clearer from the following detailed description. Attached Figure Description

[0014] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0015] Figure 1 A flowchart is shown, illustrating a method for pre-training a large language model based on federated learning according to one embodiment. Figure 2 A flowchart of a federated learning-based pre-training method for large language models according to another embodiment is shown; Figure 3 A schematic block diagram of a pre-training apparatus for a large language model based on federated learning according to one embodiment is shown; Figure 4 A schematic block diagram of a pre-training apparatus for a large language model based on federated learning, according to yet another embodiment, is shown. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0017] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0018] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0019] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0020] It is understood that the ordinal numbers such as "first" and "second" mentioned in this specification are only used to distinguish multiple objects of the same or different categories (such as components, steps, parameters, etc.), and do not indicate the priority, importance or order relationship between objects, nor do they constitute a limitation on the technical features.

[0021] As mentioned earlier, how to achieve joint model training that balances performance and fairness in an unsupervised pre-training environment using a federated learning framework is an urgent problem to be solved.

[0022] To address this, the embodiments in this specification provide a pre-training method for large language models based on federated learning. Each participant, without introducing additional supervision or interaction, guides its local large language model to learn more accurate representations through a representation measurement model. Simultaneously, the server sends different feedback model parameters to different participants, ensuring fairness in the federated learning process.

[0023] To facilitate understanding of the embodiments in this specification, some terms used in the embodiments will be explained before the technical solutions are described in detail. These terms include learnability, contribution, potential value, reward, reward fairness, etc.

[0024] Term 1: Learnability.

[0025] Assume the first The local large language model of the participating party is the first The first large language model, The learnability of the vectorized representations output by large language models can be: (1) in, It can represent the first The learnability of the vectorized representations output by large language models; It can represent the first The training sample set possessed by each participant is a set of text sequences; It can represent from the dataset A text sequence is uniformly and randomly sampled as context. ; It can represent a given context The predictability gap at that time; The expected value can be represented as calculated in the following way: from a dataset A text sequence is uniformly and randomly sampled as context. Then for all the contexts sampled calculate And take the average value.

[0026] Here, the predictability gap is a family of functions used to characterize the availability of vectorized representations of the output of a large language model under a representation measurement model. Essentially, it measures the difference in information utilization between a high-capacity model and a low-capacity predictor within a fixed representation space. A high-capacity model can refer to a model with a large number of parameters and high function complexity (e.g., a large language model). A low-capacity predictor can refer to a model with a small number of parameters and low function complexity (e.g., a representation measurement model). Here, the representation measurement model can be derived based on parameter sharing within the large language model. The number of parameters in the representation measurement model is less than the number of parameters in the large language model.

[0027] Predictability gap The smaller the value, the easier it is for the vectorized representation of the large language model output to be utilized by the representation measurement model. Therefore, the predictability gap can be reduced by taking the negative sign. This translates into a learnability metric. The smaller the predictability gap, the higher the learnability.

[0028] Term 2: Contribution.

[0029] In federated learning, each participant in a group of stakeholders pre-trains a large language model using its local dataset. The updated model parameters are then sent to the server. These parameters have value, referred to as contributions in this example. The server can pre-set a test set, based on which it can calculate the contribution of each participant. Assume the... The contribution of each participant is denoted as . The formula is as follows: (2) in, It can represent the set of participants. , Each participant provides its local dataset. . It can represent a subset of participating parties, where , Indicates the participating subset Size. It can represent the union of data sets, that is... In this example, the first... Each participant in its local dataset The large language model obtained from the above training is . It can represent large language models Performance evaluation metrics on the test set, such as F-Score. It can represent a subset of participating parties. It is a collection of participants Remove participants Subsets after that. It can represent a subset of participating parties. The corresponding multiple large language models (excluding large language models) Statistical measures of performance evaluation metrics, such as the mean. It can represent a subset of participating parties. and participating parties The union of . It can represent a subset of participating parties. and participating parties The corresponding multiple large language models (including large language models) ) is a statistical measure of performance evaluation indicators. It can represent the first The large language model corresponding to each participant The learnability of the vectorized representation of the output can be calculated using formula (1).

[0030] In the embodiments of this specification, the contribution depends not only on the performance of the large language model on the test set, but also on the learnability of the vectorized representation output by the large language model, wherein learnability is characterized by the predictability gap.

[0031] Term 3: Reward.

[0032] In different training rounds of federated learning, the server first aggregates the model parameters of the large language model uploaded by multiple participants to obtain global model parameters. Then, the aggregated global model parameters are distributed to different participants according to different parameter proportions for the next training round. That is, different participants may receive different amounts of global model parameters; for example, one participant might receive 90% of the global model parameters, while another participant might receive 50%. In this example, the distributed model parameters are called rewards, and the rewards received by different participants may vary. Term 4: Potential value.

[0033] For the first in federated learning The nth participant, assuming the nth... Each participant uses its local dataset locally. The independently trained model obtained after multiple rounds of independent training is Independent training model The performance evaluation metric (e.g., F-Score) on the test set is If the model is trained independently Performance evaluation metrics Higher than The large language model trained by each participant in the current training round of federated learning Performance evaluation metrics This indicates that the first There is potential for several participants. In this example, the first... The potential value of each participant is denoted as The formula is as follows: (3) in, It can represent an independently trained model Performance evaluation metrics on the test set. It can represent large language models Performance evaluation metrics. It can represent the first The large language model corresponding to each participant The learnability of the vectorized representation of the output can be calculated using formula (1). It can be used to depict the first The vectorized representation output by a large language model is used to utilize information in a low-capacity predictor. Furthermore, the potential value can reflect the ability of the vectorized representation output by a large language model to be fully utilized in current federated training. This ability is closely related to the learnability of its vectorized representation and can therefore be indirectly characterized by the predictability gap.

[0034] Term 5: Reward fairness.

[0035] In the federated learning embodiments of this specification, the rewards received by participants can be determined by their contribution level and / or potential value. For example, participants with higher contribution and potential values ​​should receive more rewards than participants with lower contribution and potential values ​​to reflect fairness in the rewards.

[0036] For example, suppose in the first... There are 1 training rounds Each participating party conducts federal learning, the first... The participating party in the first The contribution of each training round is recorded as Potential value is recorded as Then the first The joint assessment of each participant's contribution and potential value can be denoted as: , The calculation method can be as follows: (4) in, It can represent the mapping relationship between contribution, potential value and joint evaluation. The mapping relationship can be set according to actual needs (e.g., manually). Through the embodiments of this specification, each participant can receive corresponding rewards based on their contribution and potential value. Its formal representation can be as follows: (5) Among them, the The participating party in the first The reward earned at the start of each training round is denoted as Based on the rewards obtained Update the model parameters of the local large language model, and use the updated large language model on the local dataset. After further updating the parameters of the large language model, the updated model parameters are sent to the server. The server then uses the updated large language model to calculate the contribution and potential values, as well as the next training epoch (i.e., the [number]th epoch). A reward of +1 training round. It can represent the mapping relationship between the joint evaluation of the participants and their rewards, and this mapping relationship can be set according to actual needs (e.g., manually set).

[0037] Please see Figure 1 , Figure 1 A flowchart illustrating a pre-training method for a large language model based on federated learning according to one embodiment is shown. It is understood that the federated learning in this embodiment involves a server and multiple participants, who can obtain a pre-trained large language model through multiple rounds of training. Figure 1 The method shown can be executed by the first participant among multiple participants, who can be any of the participants. In this example, it is assumed that the large language model local to the first participant in the current training round is the first sub-model. The first sub-model is either the initial sub-model or a sub-model updated with parameters based on the feedback model parameters sent by the server in the previous training round. Specifically, if the current training round is the first training round, the first sub-model is the untrained initial sub-model. If the current training round is not the first training round, the first sub-model is the sub-model updated with parameters based on the feedback model parameters sent by the server in the previous training round. It can be understood that the server and each participant can be implemented using any device, equipment, platform, or device cluster with computing and processing capabilities.

[0038] like Figure 1 As shown, this federated learning-based large language model pre-training method may include the following steps 101 to 105, specifically: Step 101: Obtain the first training sample set.

[0039] In this embodiment, the first participant may possess a first training sample set, which may include multiple first lexical sequences. These lexical sequences may be unlabeled sequences used for pre-training of a large language model. It is understood that the first training sample set may include lexical sequences from different data categories (e.g., Chinese, mathematics, medicine, law, finance, etc.).

[0040] Step 102: For each target position in each first word sequence, the word before the target position is used as the context. The context is input into the first sub-model and the pre-built representation measurement model respectively. The first sub-model predicts the first probability distribution of the word at the target position, and the representation measurement model predicts the second probability distribution of the word at the target position.

[0041] In this embodiment, for each target position in each first word sequence in the first training sample set, the words preceding the target position can be used as context. Each of the following contexts Given a first sub-model and a pre-built representation measurement model, the first sub-model predicts the first probability distribution of lexical units at the target position. The second probability distribution of the word at the target position is represented by the measurement model. Here, the target position can be any position in the first word sequence, and the first sub-model and the representation measurement model can predict the probability distribution of words at each position in the word sequence.

[0042] In this example, the representation measurement model corresponding to the first sub-model can be derived based on the parameter sharing of the first sub-model. This representation measurement model is not an independently trained model, but rather a structurally weakened model derived from the parameter sharing of the first sub-model. It is primarily used to characterize the availability of the current vectorized representation of the first sub-model under a low-capacity predictor. Unlike the traditional teacher-student distillation framework, the representation measurement model in this embodiment does not participate in independent optimization, is not the final output model, and does not undertake knowledge learning functions. Instead, it serves as an "internal measurer" used to construct teaching signals and inversely constrain the representation learning process of the first sub-model. The representation measurement model in this embodiment does not undertake learning functions; it exists only as an internal measurer. Furthermore, the model parameters of the representation measurement model are entirely derived from the sharing of the first sub-model, rather than being independently trained, thus distinguishing it from the student model in the traditional distillation framework.

[0043] In this embodiment, it is indicated that the measurement model can be generated by imposing structural constraints on the first sub-model. It is also indicated that the number of parameters in the measurement model is less than the number of parameters in the first sub-model. Therefore, while maintaining parameter sharing, a low-capacity predictor is constructed by reducing the model's expressive power to evaluate the learnability of the first sub-model representation.

[0044] In some examples, the first sub-model may include multiple Transformer layers. Furthermore, a representation measurement model can be obtained by performing one or more of the following structural weakening operations on the first sub-model: (1) Reduce the number of Transformer layers in the first sub-model to achieve depth reduction. For example, only retain the first m layers out of multiple Transformer layers.

[0045] (2) Reduce the hidden layer dimension of the first sub-model or reduce the number of attention heads to achieve width reduction.

[0046] (3) Increase the dropout probability of the first sub-model to achieve random deactivation enhancement.

[0047] (4) Limit the scope of attention or use local attention to reduce attention capacity.

[0048] (5) Sparsify or reduce the dimensionality of intermediate features to achieve channel pruning.

[0049] (6) Use a local context window for prediction to restrict the context. Here, restricting the context means that when making a prediction, only a small part of the context before the current word is allowed, rather than the entire preceding context.

[0050] In this example, gradient backpropagation is blocked in the measurement model, and only the parameters of the first sub-model are updated, while the model parameters in the measurement model are shared by the first sub-model.

[0051] For example, in one specific implementation, the measurement model can be constructed as follows: the Transformer structure of the first sub-model is deeply weakened, retaining only the first... Layer (wherein, , (The number of Transformer layers in the first sub-model), while increasing the number of attention heads from... Reduce to (in, The dropout probability is increased from a first probability value to a second probability value, where the second probability value is greater than the first probability value. In this configuration, the representation measurement model shares parameters with the first sub-model, and is used only during forward propagation to generate the predicted probability distribution. It's important to note that this representation measurement model is not trained independently; its parameters are shared by the first sub-model and it is used only during the forward propagation to generate the predicted probability distribution, thus serving as a tool for measuring the predictability of the first sub-model. Through the representation measurement model, a low-capacity predictor can be built from the first sub-model without introducing an explicit model or external supervision signals, thereby characterizing the quality of the current vectorized representation in the "learnability" dimension.

[0052] Step 103: Calculate the difference loss based on the pre-built joint loss function, and update the model parameters of the first sub-model based on the difference loss.

[0053] In this embodiment, a pre-constructed joint loss function can be used. This joint loss function may include a first loss term and a second loss term. The first loss term can be determined based on a first probability distribution and the true word at the target position, and the second loss term can be determined based on the first and second probability distributions. Based on this joint loss function, a difference loss can be calculated, and the model parameters of the first sub-model can be updated based on the difference loss. For example, with the goal of minimizing the difference loss, the model parameters of the first sub-model can be updated using gradient descent.

[0054] For example, the joint loss function can be as follows: (6) in, It can represent the joint loss function. This can represent the first loss term. This can represent the second loss term.

[0055] For the first loss item Assume the training corpus is an unlabeled text sequence dataset. , Represents the dataset The number of Chinese text sequences, where each text sequence can be represented as ,in, It can represent the second token in a text sequence. It can represent the first in a text sequence. Each word element is assumed to originate from an unknown data distribution. In the pre-training of standard large language models, the maximum likelihood estimation criterion can be used to update the model parameters by minimizing the negative log-likelihood loss. Its loss function is the first loss term in the joint loss function, as follows: (7) in, This can represent the model parameters of the first sub-model. Indicates the current word position The previous context, It can be represented that the first sub-model is based on the position of the preceding lexical unit. Previous context prediction of the current lexical position The probability distribution of lexical units. It should be noted that in the pre-training of large language models, the true lexical units at each position... It serves directly as a monitoring signal for that location, therefore, no separate labeling is required.

[0056] In large language model pre-training, the negative log-likelihood loss is calculated as follows: for each position... The model first generates the predicted probability distribution over the entire vocabulary. Then based on the location Real lexical tags Take the corresponding probability value from the distribution and take its negative logarithm. Finally, the summation is performed across all positions. Therefore, the negative log-likelihood loss depends on both the predicted probability distribution and the true word labels.

[0057] For the second loss term, it can be addressed through the predictability gap. Construct a second loss term, where the predictability gap It can be determined based on the first probability distribution and the second probability distribution. In some examples, the second loss term... In It can represent a weighting function. The objective function can be represented as a single function, and the second loss term is the product of the weight function and the objective function. The weight function can be based on the number of training steps. or context Therefore, the objective function can be determined based on the predictability gap. For example, the formula for the objective function can be as follows: (8) in, This objective function can represent the predictability gap. Encouraging large language models to learn representation structures that are more easily utilized by representation measurement models improves the learnability of vectorized representations. With a fixed hypothesis space for the representation measurement model, a decrease in predictability gap implies an improved ability of the representation measurement model to utilize information from the current representation, indicating that the representation is more learnable under constrained learning conditions, i.e., it has higher learnability.

[0058] In some examples, for a given context The predicted probability distribution output by the first sub-model The predicted probability distribution of the measurement model output is represented by [symbol]. The predictability gap is a metric used to characterize the learnability of the first sub-model representation.

[0059] In this example, the predictability gap is not limited to a single form, but can be a class of metrics constructed based on probability distribution uncertainty, distributional variability, or key prediction confidence.

[0060] In one example, the predictability gap It can be determined based on the difference between the entropy of the predicted probability distribution representing the measurement model and the entropy of the predicted probability distribution of the first sub-model.

[0061] For example, the predictability gap can be represented by the Shannon entropy difference between the predicted probability distributions of the measurement model and the first sub-model. Specifically: (9) in, This can represent the gap in predictability. Shannon entropy can represent the predicted probability distribution. It can represent the predicted probability distribution of the measurement model output. Shannon entropy, This can represent the predicted probability distribution of the first sub-model output. The Shannon entropy. The predictability gap is used to measure how easily the representation of the first sub-model can be utilized by the representation measurement model.

[0062] In one example, the predictability gap It can also be determined based on the difference in the predicted probability distribution of the representation measurement model and the predicted probability distribution of the first sub-model.

[0063] For example, the predictability gap The KL (Kullback-Leibler) divergence can be used to determine this, specifically: (10) in, It can represent the KL divergence.

[0064] Alternatively, symmetric divergence (e.g., JS divergence) or optimal propagation distance (e.g., Wasserstein distance) can be used as a metric. Taking JS divergence as an example, specifically: (11) in, It can represent JS divergence.

[0065] In one example, the predictability gap It can also be determined based on the difference in prediction confidence between the first sub-model and the representation measurement model on the first K candidate words.

[0066] For example, the predictability gap The difference in prediction confidence between the first sub-model and the representation measurement model on Top-k candidate words can be specifically defined as follows: (12) In one example, the predictability gap It can also be determined based on the difference in predicted probabilities between the first sub-model and the representation measurement model for the same lexical position.

[0067] For example, targeting real target words The difference in predicted probabilities between the first sub-model and the measurement model is: (13) in, This can be represented as the first sub-model targeting the target word. The predicted probability distribution of location, This can refer to the measurement model targeting the lexical unit. The predicted probability distribution of the location.

[0068] In one example, the predictability gap It can also be obtained by aggregating the hierarchical predictability gaps of the first sub-model and the representation measurement model at multiple levels. The hierarchical predictability gaps corresponding to each level can be calculated based on the prediction probability distributions of the first sub-model and the representation measurement model at each level. The hierarchical predictability gaps at each level can be determined using any of the aforementioned formulas (9) to (13) (e.g., based on entropy difference, KL divergence, or target word probability difference, etc.).

[0069] For example, the first sub-model and the representation measurement model can include multiple Transformer layers, and the predictability gap can be computed and aggregated at multiple levels, such as: (14) in, It can be expressed as the first Layers represent the hierarchical predictability gap in spatial computation. It can represent hierarchical weights.

[0070] In one example, the predictability gap The basic predictability gap can also be obtained by weighting the prediction difficulty based on the sample or context, where the basic predictability gap can be calculated based on the prediction probability distribution of the first sub-model and the representation measurement model.

[0071] For example, the gap in basic predictability can be weighted based on the predictability difficulty of the sample or context. Specifically: (15) in, It can represent a weighting function that is adjusted according to the difficulty of prediction based on the sample or context, for example, it can be constructed based on the uncertainty of the prediction distribution (such as entropy), historical error or confidence. The basic predictability gap can be used to characterize the difference in original predictive power between the first sub-model and the representation measurement model. The basic predictability gap can be determined using any of the aforementioned formulas (9) to (13) (e.g., based on entropy difference, KL divergence, or target word probability difference). The above weighting mechanism can make the first sub-model pay more attention to samples with high uncertainty or difficult learning during training, thereby improving the targeting and efficiency of representation optimization.

[0072] Weighting function in the second loss term The weight function can be determined based on the number of training steps or the context. The method of determination is not limited to a single form.

[0073] In one example, the weight function It can be positively correlated with the number of training steps.

[0074] For example, It can be adjusted according to the number of training steps. Gradually increase, specifically: (16) in, It can represent the number of training steps. It can represent the total number of training steps. It can represent the maximum weight, which can be preset (e.g., manually set). This can represent a weight function determined based on the number of training steps. In this example, equal In this example, during the initial training of the first sub-model, the first loss term in the joint loss function represents the basic language modeling capability. The proportion is relatively large, and in the later stages of training, the objective function representing the teaching constraints is gradually strengthened. The proportion of [something] is such that the first sub-model can prioritize learning basic language modeling skills in the early stages of training, while gradually strengthening teaching constraints in the later stages.

[0075] In one example, the weight function It can be determined based on the predictability gap.

[0076] For example, it can be based on the predictability gap Adjusting the weights, specifically: (17) in, It can represent a pre-defined (e.g., manually defined) non-zero positive number. This can represent a weighting function determined based on context; in this example, equal This example demonstrates how to strengthen instructional constraints when the predictability gap is large and weaken constraints when the predictability gap is small.

[0077] In one example, the weight function It can be determined based on the difficulty of the lexical unit.

[0078] For example, it can be determined based on the difficulty of word prediction (such as prediction entropy or historical error), specifically: (18) in, Entropy can represent the probability distribution of a prediction. Entropy can be used to measure the uncertainty of a prediction, and the higher the difficulty, the greater the weight. This can represent mapping the entropy of the predicted probability distribution to a weight function according to a predefined mapping relationship (e.g., a manually set mapping relationship). The value of . In this example, equal .

[0079] In one example, the weight function It can be determined based on a preset threshold.

[0080] For example, when the predictability gap Below the preset threshold At that time, reduce or turn off the teaching items, specifically: (19) in, This can represent a preset (e.g., manually set) weight value. In this example, equal .

[0081] In one example, the weight function It can be determined based on word frequency.

[0082] For example, weights can be adjusted based on word frequency, giving higher weights to rare words. Specifically: (20) in, It can represent the frequency of word occurrence. In this example, equal .

[0083] Therefore, the embodiments in this specification can achieve a progressive training process from "basic language modeling" to "learnability optimization" by dynamically adjusting the weights of the objective function, thereby adaptively balancing the model fitting ability and the generalizability of the representation at different training stages.

[0084] Step 104: Send the model parameters of the first sub-model to the server. The server determines the global model parameters based on the model parameters uploaded by multiple participants, and determines the feedback model parameters to be fed back to each participant based on the global model parameters and the model parameters uploaded by each participant. Then, the server sends the first feedback model parameters to the first participant.

[0085] In this embodiment, the large language model has the same model structure for all participants and the server. Each participant can upload the updated model parameters of its local large language model to the server, and the server can determine the global model parameters based on the model parameters of multiple participants. For example, the global model parameters can be obtained by weighting the local model parameters of multiple participants according to their respective data volumes using a Federated Averaging (FedAvg) algorithm.

[0086] In some examples, the server can calculate the gradient of each participant's sub-model based on the model parameters uploaded by each participant. At the same time, considering the possibility of imbalanced data categories and data volume, the gradients of each participant are weighted and averaged according to the proportion of data volume and data category. The global model is then updated based on the weighted average gradient to obtain the updated global model parameters.

[0087] Subsequently, the server can determine the feedback model parameters to be sent to each participant based on the global model parameters and the model parameters uploaded by each participant. The feedback model parameters will differ between different participants. As an example, the server can determine the parameter ratio for each participant based on the amount and type of data. For instance, the larger the data volume and the more data categories, the higher the corresponding parameter ratio. For example, a preset mapping relationship can be used to determine the correspondence between data volume, data categories, and parameter ratios. Based on this relationship, the server can determine the parameter ratio for each participant. Then, for each participant, the server can retrieve the corresponding parameter ratio from the global model parameters and send it back to that participant. For example, if a participant's parameter ratio is 70%, the server can retrieve 70% of the model parameters from the global model parameters and send them back to that participant. As another example, the server can determine the parameter ratio of each participant based on the accuracy of the model parameters uploaded by each participant. For example, it can determine the local sub-model of each participant based on the model parameters uploaded by each participant, and test the prediction accuracy of the local sub-model of each participant on the test set of the server. The parameter ratio of each participant can be determined based on the prediction accuracy. For example, the higher the prediction accuracy, the higher the corresponding parameter ratio.

[0088] In some examples, the server may include a test set. Based on this, the server can determine the first feedback model parameters to be fed back to the first participant through the following steps 1) to 4), specifically: Step 1) Determine the sub-model of each participant based on the model parameters of the sub-model uploaded by each participant, and calculate the performance evaluation index of the sub-model of each participant on the test set.

[0089] In this example, the large language model has the same structure for all participants and the server. The server can determine the local sub-model for each participant based on the model parameters of the sub-models uploaded by each participant. Then, the server can calculate the performance evaluation metrics of each participant's sub-model on the test set, such as the F-score.

[0090] Step 2) Calculate the contribution of the first participant based on the performance evaluation metrics of each participant's sub-model on the test set and the learnability of the representation of the first sub-model.

[0091] In this example, each participant can also calculate the representation learnability of its local sub-model and upload the calculated representation learnability along with the model parameters of its local sub-model to the server. For the first participant, the representation learnability of its corresponding first sub-model is based on the predictability gap, which is calculated using the prediction probability distribution of the first sub-model and the representation measurement model. Therefore, the server can calculate the first participant's contribution based on the performance evaluation metrics of each participant's sub-model on the test set and the representation learnability of the first sub-model.

[0092] Step 3) Determine the potential value of the first participant based on the performance evaluation metrics of the first participant's independently trained model, the performance evaluation metrics of the first sub-model, and the representation learnability of the first sub-model.

[0093] In this example, the first participant can train an independent training model locally. This independent training model can be a large language model trained by the first participant using a first training sample set through multiple rounds of training. The model structure of this independent training model is the same as that of the first sub-model. The first participant can upload the model parameters of the independent training model to the server. The server can then determine the first participant's local independent training model and calculate its performance evaluation metrics on the test set. Subsequently, the server can determine the first participant's potential value based on the performance evaluation metrics of the first participant's independent training model, the performance evaluation metrics of the first sub-model, and the learnability of the first sub-model's representation.

[0094] Step 4) Based on the contribution and / or potential value, determine the parameter ratio to be fed back to the first participant, and determine the first feedback model parameters according to the parameter ratio and the global model parameters.

[0095] In this example, the server can determine the parameter ratio fed back to the first participant based on the first participant's contribution and / or potential value. For example, a correspondence between contribution and parameter ratio, or between potential value and parameter ratio, or between contribution and potential value and parameter ratio, can be pre-established (e.g., manually established). Thus, the parameter ratio is determined based on the contribution and / or potential value. Afterward, the server can determine the first feedback model parameters based on the parameter ratio and the global model parameters.

[0096] In some examples, the potential value can be the product of a first ratio and the learnability of the representation of the first sub-model, where the first ratio is the ratio of the difference between the performance evaluation metrics of the independently trained model and the performance evaluation metrics of the first sub-model to the performance evaluation metric of the first sub-model. Furthermore, determining the proportion of parameters fed back to the first participant based on the contribution and the potential value can include: When the potential value is greater than 0, the potential value and 1 are added to obtain the first sum, and the product of the first sum and the contribution is calculated to obtain the first product. The smaller value between 1 and the first product is determined as the parameter ratio. Specifically, for the first... For each participant, when the potential value is greater than 0, the parameter ratio can be... ,in, It can represent the first The contribution of each participant, It can represent the first The potential value of each participant.

[0097] When the potential value is less than or equal to 0, the contribution is determined as a parameter ratio.

[0098] Step 105: Receive the first feedback model parameters sent by the server, and update the model parameters of the first sub-model based on the first feedback model parameters.

[0099] In this embodiment, the first participant can receive the first feedback model parameters sent by the server and update the model parameters of the first sub-model based on the first feedback model parameters. This completes one round of federated learning.

[0100] In summary, without introducing additional supervision or interaction, the first participant guides the first sub-model to learn more accurate representations through the representation measurement model, thereby improving the generalization ability of the first sub-model and the subsequent training effect.

[0101] Please continue reading Figure 2 , Figure 2 A flowchart of a large language model pre-training method based on federated learning according to another embodiment is shown. It is understood that the federated learning in this embodiment involves a server and multiple participants, and the server and participants can obtain a pre-trained large language model through multiple rounds of training. Figure 2 The method shown can be executed by the server.

[0102] like Figure 2 As shown, this federated learning-based large language model pre-training method may include the following steps 201 to 204, specifically: Step 201: Receive model parameters of sub-models uploaded by multiple participants.

[0103] In this embodiment, the server can receive model parameters of the sub-models uploaded by each participant.

[0104] Among the multiple participants, the first participant can train its local first sub-model through the following steps (1) to (2), specifically: Step (1): For each target position in each first word sequence in the first training sample set, the word preceding the target position is used as context. The context is input into the first sub-model and the pre-built representation measurement model, respectively. The first sub-model predicts the first probability distribution of the word at the target position, and the representation measurement model predicts the second probability distribution of the word at the target position. The representation measurement model is derived based on the parameter sharing of the first sub-model.

[0105] Step (2): Calculate the difference loss based on the pre-constructed joint loss function, and update the model parameters of the first sub-model based on the difference loss. The joint loss function may include a first loss term and a second loss term. The first loss term is determined based on the first probability distribution and the true word at the target position, and the second loss term is determined based on the first probability distribution and the second probability distribution.

[0106] Step 202: Determine the global model parameters based on the model parameters of the sub-models uploaded by each participant.

[0107] Step 203: Determine the feedback model parameters to be fed back to each participant based on the global model parameters and the model parameters uploaded by each participant. Note that the feedback model parameters differ between different participants. In some examples, step 203 above may include steps 2031 to 2034, specifically: Step 2031: Determine the sub-model of each participant based on the model parameters of the sub-model uploaded by each participant, and calculate the performance evaluation index of the sub-model of each participant on the test set.

[0108] Step 2032: Calculate the contribution of each participant based on the performance evaluation metrics of each participant's sub-model on the test set and the representation learnability of each participant's corresponding sub-model.

[0109] In this example, learnability is represented based on the predictability gap, which is calculated using the prediction probability distribution of the sub-model and the corresponding representation measurement model.

[0110] Step 2033: Determine the potential value of each participant based on the performance evaluation metrics of the independent training models of each participant, the performance evaluation metrics of the sub-models of each participant, and the representation learnability of the sub-models of each participant.

[0111] Step 2034: Based on the contribution and / or potential value of each participant, determine the parameter ratio to be fed back to each participant, and determine the first feedback model parameters according to the parameter ratio and the global model parameters.

[0112] Step 204: Send the feedback model parameters of each participant to each participant so that each participant can update the model parameters of its local sub-model based on the received feedback model parameters.

[0113] As described above, in the embodiments of this specification, each participant directly affects the model's performance improvement capability under limited training conditions by optimizing the learnability of the model representation, thereby determining its effective contribution and potential capability in federated learning. In this embodiment, the federated aggregation strategy is based on representation learnability and achieves fairness optimization through joint modeling of contribution and potential value.

[0114] Furthermore, the potential value in this embodiment is not determined based on historical contributions or gradient changes, but rather characterized by the difference between the performance of independently trained models and federated training models, thus reflecting the underutilized potential of the model in the current federated environment. In the large model federated aggregation method based on potential fairness proposed in this embodiment, each participant's contribution and potential are comprehensively evaluated before a corresponding reward model is assigned. Here, contribution, potential, and reward are all dynamically calculated, i.e., for each round of federated learning training. Participants Their respective contributions will be calculated. Potential Rewards. For example, Algorithm 1 below illustrates the details of a large-scale federated aggregation method based on potential fairness. In line 1, the global model is initialized first. Based on several non-empty initialization models Perform initialization.

[0115] Lines 2 to 7 are from the participating parties This is described from the perspective of [the relevant authority / entity]. Specifically, line 4 indicates that in each round of training in federated learning, participants […]. Receive the model sent by the server It is understandable that, since all participants share the exact same model structure with the server, there is a one-to-one correspondence between model parameters and the model itself. Under this premise, the server distributing model parameters is equivalent to distributing the complete model, and participants uploading model parameters is equivalent to uploading a local model. In line 5, participants... Use the received model After local training, an updated local model is obtained. In line 6, the updated model will be... Send it to the server for aggregation.

[0116] Lines 8 to 26 describe the process from the server-side perspective, in detail; Line 10 indicates that in each round of training, participants receive... Sending model .

[0117] In lines 11 and 12, the server will connect each participant Uploaded sub-model Perform aggregation to obtain a new global model For the next round (i.e., This prepares for the model distribution (round number). Specifically, the model's gradient aggregation can be divided into two steps, as shown in lines 11 and 12. In line 11, the participants are calculated. gradient ,in, It can represent the first The global model of the round. In line 12, considering both the possibility of imbalanced data categories and imbalanced data volume, the gradients of each participant are... Simultaneously, a weighted average is calculated based on the proportion of data volume and data categories to update the current global model. This yields a global model for the next round. ,in, It can represent a learnability-based adjustment function, which is used to weight the gradient contribution according to the learnability of the model representation. The adjustment function can be set according to actual needs (e.g., manually).

[0118] Lines 13 to 16 are for approximating the subset of each participant. model Because of the data privacy protection principles under federated learning, the server cannot... The model is trained on the data. Therefore, a computational model is required. Approximate model To replace the model For subsequent calculations, in this example, the model Its approximate model They are equivalent. In line 14, the participant subset is calculated using the weighted average method described in line 12. gradient In line 15, a subset of participants is used. gradient To update the approximate model ,get Approximate model .

[0119] In line 17, using each obtained... It can calculate the participants Contribution In line 18, the contribution is normalized to obtain the contribution of each participant. The contribution in the next round is used to obtain a contribution list. In line 19, the server can calculate the participants. Potential value Lines 20 to 24 can be ranked according to contribution. and potential value Determine the model to be sent to the participants.

[0120] Therefore, the federated learning algorithm in this embodiment combines contribution and potential for joint evaluation during the federated aggregation process, so that the model reward allocation matches the actual contribution and unutilized capabilities of the participants, thereby improving the fairness and overall efficiency of multi-party collaborative training.

[0121] According to another embodiment, a pre-training device for a large language model based on federated learning is provided. It is understood that the federated learning in this embodiment involves a server and multiple participants, and the server and multiple participants can obtain a pre-trained large language model through multiple rounds of training. The aforementioned pre-training device for a large language model based on federated learning can be deployed on a first participant, which can be any one of the multiple participants. In this example, it is assumed that the large language model local to the first participant in the current training round is the first sub-model. The first sub-model is either an initial sub-model or a sub-model with updated parameters based on the feedback model parameters sent by the server in the previous training round. Specifically, if the current training round is the first training round, the first sub-model is an untrained initial sub-model. If the current training round is not the first training round, the first sub-model is a sub-model with updated parameters based on the feedback model parameters sent by the server in the previous training round. It is understood that the server and each participant can be implemented using any device, equipment, platform, or device cluster with computing and processing capabilities.

[0122] Figure 3 A schematic block diagram of a large language model pre-training apparatus based on federated learning according to one embodiment is shown. Figure 3 As shown, the federated learning-based large language model pre-training device 300 includes: The acquisition unit 301 is used to acquire a first training sample set, wherein the first training sample set includes multiple first word sequence sequences; The input unit 302 is used to, for each target position in each first word sequence, take the word before the target position as the context, input the context into the first sub-model and the pre-built representation measurement model respectively, predict the first probability distribution of word at the target position by the first sub-model, and predict the second probability distribution of word at the target position by the representation measurement model, wherein the representation measurement model is derived based on the parameter sharing of the first sub-model; The calculation unit 303 is used to calculate the difference loss based on the pre-constructed joint loss function, and update the model parameters of the first sub-model based on the difference loss. The joint loss function includes a first loss term and a second loss term. The first loss term is determined based on the first probability distribution and the true word at the target position, and the second loss term is determined based on the first probability distribution and the second probability distribution. The sending unit 304 is used to send the model parameters of the first sub-model to the server. The server determines the global model parameters based on the model parameters uploaded by the multiple participants, and determines the feedback model parameters to be fed back to each participant based on the global model parameters and the model parameters uploaded by each participant. The server then sends the first feedback model parameters to the first participant. The feedback model parameters of different participants are different. The update unit 305 is used to receive the first feedback model parameters sent by the server and update the model parameters of the first sub-model based on the first feedback model parameters.

[0123] According to another embodiment, a further large language model pre-training device based on federated learning is provided. It is understood that the federated learning in this embodiment involves a server and multiple participants, and the server and participants can obtain a pre-trained large language model through multiple rounds of training. The aforementioned large language model pre-training device based on federated learning can be deployed on the server.

[0124] Figure 4 A schematic block diagram of a pre-training apparatus for a large language model based on federated learning, according to yet another embodiment, is shown. Figure 4 As shown, the federated learning-based large language model pre-training device 400 includes: The parameter receiving unit 401 is used to receive model parameters of the sub-models uploaded by the multiple participants. The first participant among the multiple participants trains the first sub-model in the following way: for each target position in each first word sequence in the first training sample set, the word before the target position is used as context, and the context is input into the first sub-model and a pre-built representation measurement model respectively. The first sub-model predicts the first probability distribution of the word at the target position, and the representation measurement model predicts the second probability distribution of the word at the target position. The representation measurement model is derived based on the parameter sharing of the first sub-model. The difference loss is calculated based on the pre-built joint loss function, and the model parameters of the first sub-model are updated based on the difference loss. The parameter aggregation unit 402 is used to determine the global model parameters based on the model parameters of the sub-models uploaded by the above-mentioned multiple participants; The parameter calculation unit 403 is used to determine the feedback model parameters to be fed back to each participant based on the global model parameters mentioned above and the model parameters uploaded by each participant. The feedback model parameters are different for different participants. The parameter sending unit 404 is used to send the feedback model parameters of each participant to each participant, so that each participant can update the model parameters of its local sub-model based on the received feedback model parameters.

[0125] According to another embodiment, a computing device is also provided, including a processor, a memory, and a computer program / instructions stored in the memory, characterized in that the processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the computing device performs the following functions: Figure 1 or Figure 2 The steps of the method described in the document.

[0126] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program / instructions are stored, characterized in that, when the computer program / instructions are executed by a processor, they implement as follows: Figure 1 or Figure 2 The steps of the method described herein. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0127] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0128] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0129] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A pre-training method for a large language model based on federated learning, the method involving a server and multiple participants, the method being executed by a first participant among the multiple participants, the large language model locally of the first participant being a first sub-model, the first sub-model being an initial sub-model or a sub-model with updated parameters based on feedback model parameters sent by the server in the previous training round, the method comprising: Obtain a first training sample set, wherein the first training sample set includes multiple first word sequence sequences; For each target position in each first word sequence, the word before the target position is taken as the context, and the context is input into the first sub-model and the pre-built representation measurement model respectively. The first sub-model predicts the first probability distribution of the word at the target position, and the representation measurement model predicts the second probability distribution of the word at the target position. The representation measurement model is derived based on the parameter sharing of the first sub-model. The difference loss is calculated based on a pre-constructed joint loss function, and the model parameters of the first sub-model are updated based on the difference loss. The joint loss function includes a first loss term and a second loss term. The first loss term is determined based on the first probability distribution and the true word at the target position, and the second loss term is determined based on the first probability distribution and the second probability distribution. The model parameters of the first sub-model are sent to the server. The server determines the global model parameters based on the model parameters uploaded by the multiple participants, and determines the feedback model parameters to be fed back to each participant based on the global model parameters and the model parameters uploaded by each participant. The server then sends the first feedback model parameters to the first participant. The feedback model parameters of different participants are different. The system receives the first feedback model parameters sent by the server and updates the model parameters of the first sub-model based on the first feedback model parameters.

2. The method according to claim 1, characterized in that, The server includes a test set, and the server determines the first feedback model parameters to be fed back to the first participant through the following steps: The sub-models of each participant are determined based on the model parameters of the sub-models uploaded by each participant, and the performance evaluation metrics of the sub-models of each participant on the test set are calculated. Based on the performance evaluation metrics of each participant's sub-model on the test set and the representation learnability of the first sub-model, the contribution of the first participant is calculated, wherein the representation learnability is obtained based on the predictability gap, which is calculated through the prediction probability distribution of the first sub-model and the representation measurement model. Based on the performance evaluation metrics of the first participant's independent training model, the performance evaluation metrics of the first sub-model, and the representation learnability of the first sub-model, the potential value of the first participant is determined, wherein the independent training model is a large language model obtained by the first participant through multiple rounds of training using the first training sample set. Based on the contribution and / or the potential value, determine the parameter ratio to be fed back to the first participant, and determine the first feedback model parameters according to the parameter ratio and the global model parameters.

3. The method according to claim 2, characterized in that, The potential value is the product of a first ratio and the representation learnability of the first sub-model, where the first ratio is the ratio of the difference between the performance evaluation metric of the independently trained model and the performance evaluation metric of the first sub-model to the performance evaluation metric of the first sub-model. And, based on the contribution level and the potential value, determine the proportion of parameters to be fed back to the first participant, including: When the potential value is greater than 0, the potential value is added to 1 to obtain a first sum, and the product of the first sum and the contribution is calculated to obtain a first product. The smaller value between 1 and the first product is determined as the parameter ratio. When the potential value is less than or equal to 0, the contribution is determined as the parameter ratio.

4. The method according to claim 1, characterized in that, The second loss term is the product of a weight function and an objective function, wherein the objective function is determined based on the predictability gap; wherein the weight function is determined based on the number of training steps or context, and the predictability gap is calculated through the predicted probability distribution of the first sub-model and the representation measurement model.

5. The method according to claim 4, characterized in that, The predictability gap is determined based on the difference between the entropy of the predicted probability distribution representing the measurement model and the entropy of the predicted probability distribution of the first sub-model. Alternatively, it can be determined based on the distribution difference between the predicted probability distribution of the measurement model and the predicted probability distribution of the first sub-model; Alternatively, it can be determined based on the difference in prediction confidence between the first sub-model and the representation measurement model on the first K candidate words; Alternatively, it can be determined based on the difference in prediction probabilities between the first sub-model and the representation measurement model for the same word at the same position; Alternatively, it can be obtained by aggregating the hierarchical predictability gaps between the first sub-model and the representation measurement model at multiple levels, wherein the hierarchical predictability gaps at each level are calculated based on the prediction probability distributions of the first sub-model and the representation measurement model at each level. Alternatively, the basic predictability gap can be obtained by weighting the prediction difficulty based on the sample or context, wherein the basic predictability gap is calculated based on the prediction probability distribution of the first sub-model and the representation measurement model.

6. The method according to claim 1, characterized in that, The first sub-model includes multiple Transformer layers, and the representation measurement model is obtained by performing one or more of the following operations on the first sub-model: Reduce the number of Transformer layers in the first sub-model; Reduce the hidden layer dimension of the first sub-model or reduce the number of attention heads; Increase the dropout probability of the first sub-model; Limit the scope of attention or use localized attention; Sparsification or dimensionality reduction of intermediate features; Prediction is performed using a local context window.

7. A pre-training method for a large language model based on federated learning, the method involving a server and multiple participants, the method being executed by the server, the method comprising: The system receives model parameters of sub-models uploaded by the multiple participants. The first participant among the multiple participants trains the first sub-model in the following manner: for each target position in each first word sequence in its first training sample set, the words preceding the target position are used as context. The context is input into the first sub-model and a pre-built representation measurement model. The first sub-model predicts a first probability distribution of words at the target position, and the representation measurement model predicts a second probability distribution of words at the target position. The representation measurement model is derived based on parameter sharing of the first sub-model. A difference loss is calculated based on a pre-built joint loss function, and the model parameters of the first sub-model are updated based on the difference loss. Based on the model parameters of the sub-models uploaded by the multiple participants, the global model parameters are determined; The feedback model parameters are determined based on the global model parameters and the model parameters uploaded by each participant, wherein the feedback model parameters are different for different participants. The feedback model parameters of each participant are sent to each participant so that each participant can update the model parameters of its local sub-model based on the received feedback model parameters. The multiple participating parties employ the pre-training method as described in claim 1.

8. The method according to claim 7, characterized in that, The server includes a test set. The step of determining the feedback model parameters to be fed back to each participant based on the global model parameters and the model parameters uploaded by each participant includes: The sub-models of each participant are determined based on the model parameters of the sub-models uploaded by each participant, and the performance evaluation metrics of the sub-models of each participant on the test set are calculated. Based on the performance evaluation metrics of each participant's sub-model on the test set and the representation learnability of each participant's corresponding sub-model, the contribution of each participant is calculated. The representation learnability is obtained based on the predictability gap, which is calculated through the prediction probability distribution of the sub-model and the representation measurement model corresponding to the sub-model. Based on the performance evaluation metrics of each participant's independently trained model, the performance evaluation metrics of each participant's sub-model, and the representation learnability of each participant's sub-model, the potential value of each participant is determined. Based on the contribution and / or potential value of each participant, the parameter ratio to be fed back to each participant is determined, and the feedback model parameters are determined according to the parameter ratio and the global model parameters.

9. A computing device, comprising a processor, a memory, and computer programs / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the computing device implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 8.

Images