A method, apparatus and equipment for training a federated large language model
By constructing an initial mask matrix and using sparse reconstruction techniques to perform unstructured pruning on large language models, the problems of large model structure and low training efficiency are solved, and the training speed and accuracy are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHONGJINKE INFORMATION TECH CO LTD
- Filing Date
- 2025-07-08
- Publication Date
- 2026-05-26
AI Technical Summary
Existing methods for training large language models based on federated learning suffer from problems such as large model structures and low training efficiency.
By constructing an initial mask matrix, splitting it into mask blocks, using sparsity and the inverse Hessian matrix to determine the model parameters of non-zero elements, performing unstructured pruning, and combining it with an optimized dataset for training, the model structure is simplified.
Unstructured pruning of large language models was implemented, which improved training speed while ensuring the accuracy of model training.
Smart Images
Figure CN121052335B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, and device for training a federated large language model. Background Technology
[0002] Federated learning is a distributed learning technology that combines traditional cryptography and machine learning. It aims to build a federated learning model based on a distributed dataset for multiple data providers who are unwilling or unable to expose their plaintext data.
[0003] Large language models are Transformer-based deep learning models applied in the field of natural language processing. They typically contain hundreds of billions (or more) parameters trained on large amounts of text data and have a wide range of applications, including machine translation, speech recognition, information retrieval, content generation, and intelligent question answering.
[0004] However, existing methods for training large language models based on federated learning suffer from problems such as large model structures and low training efficiency. Summary of the Invention
[0005] In view of this, the present invention provides a federated large language model training method, apparatus and equipment, the main purpose of which is to solve the problem of large model structure and low training efficiency in the current model.
[0006] To address the aforementioned issues, this application provides a federated large language model training method, applied to the master computing node, comprising:
[0007] For each original model parameter in the initial large language model, construct an initial mask matrix;
[0008] For the current pruning round number, the initial mask matrix is split to obtain the initial mask block corresponding to the current pruning round number;
[0009] Based on the pre-built mask reconstruction dataset, the predetermined sparsity, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix, the current mask block corresponding to the initial mask block is determined, and the current model parameters corresponding to the non-zero, non-target elements in the current mask block are determined, thus obtaining the current model parameter matrix corresponding to the initial mask block.
[0010] Based on the current model parameter matrix corresponding to each pruning round, determine the pruning large language model;
[0011] The pruned large language model is optimized and trained based on a pre-built first optimized dataset to obtain the target large language model.
[0012] Optionally, constructing an initial mask matrix for each original model parameter in the initial large language model specifically includes:
[0013] Determine the corresponding element values for each original model parameter in the initial large language model;
[0014] Based on the values of each element, an initial mask matrix is constructed.
[0015] Optionally, the step of splitting the initial mask matrix for the current pruning round number to obtain the initial mask block corresponding to the current pruning round number specifically includes:
[0016] Determine the target number of columns based on the remaining video memory capacity;
[0017] The initial mask matrix is split based on the target column number to obtain the initial mask block corresponding to the current pruning round number.
[0018] Optionally, determining the target number of columns based on the remaining video memory capacity specifically includes:
[0019] Determine the reference video memory capacity based on the remaining video memory capacity;
[0020] The target number of columns is determined based on the reference memory capacity and the number of elements in each column of the initial mask matrix.
[0021] Optionally, after obtaining the current model parameter matrix, the method further includes:
[0022] Based on a predetermined pruning algorithm, each current model parameter in the current model parameter matrix is compensated to obtain the target model parameter matrix corresponding to the initial mask block, which is used to determine the pruned large language model.
[0023] Optionally, after obtaining the pruned large language model, the method further includes:
[0024] The parameters of each model in the pruned large language model are quantized and compressed to obtain a compressed pruned large language model for optimization training.
[0025] Optionally, after obtaining the pruned large language model, the method further includes: sending the pruned large language model to each slave computing node;
[0026] The process of optimizing and training the pruned large language model based on a pre-constructed first optimized dataset to obtain the target large language model specifically includes:
[0027] The pruned large language model is optimized based on the first optimized dataset to obtain the initial first optimized parameters;
[0028] Receive the initial second optimization parameters obtained by optimizing the pruned large language model from each computing node;
[0029] Based on the first optimization parameters and the second optimization parameters, parameter aggregation is performed to obtain the initial aggregation parameters;
[0030] The initial aggregation parameters are sent to each slave computing node so that each slave computing node can perform the next round of model optimization based on the initial aggregation parameters. The current second optimization parameters sent by each slave computing node are received until the predetermined optimization conditions are met. Then the model optimization stops and the current aggregation parameters are used as the target aggregation parameters to obtain the target large language model.
[0031] To address the aforementioned issues, this application provides a federated large language model training method, applied from computing nodes, including:
[0032] Receive the pruned large language model sent by the master computing node;
[0033] The pruned large language model is optimized and trained based on a pre-built second optimized dataset to obtain the target large language model.
[0034] Optionally, the step of optimizing and training the pruned large language model based on a pre-constructed second optimized dataset to obtain the target large language model specifically includes:
[0035] The pruned large language model is optimized based on the second optimized dataset to obtain the initial second optimized parameters;
[0036] The initial second optimization parameters are sent to the master computing node so that the master computing node can perform parameter aggregation processing based on the first optimization parameters and the second optimization parameters to obtain the initial aggregated parameters.
[0037] The system receives the initial aggregation parameters sent by the master computing node, performs the next round of model optimization based on the initial aggregation parameters, and stops model optimization when the predetermined optimization conditions are met. The received current aggregation parameters are then used as the target aggregation parameters to obtain the target large language model.
[0038] To address the aforementioned problems, this application provides a federalized large language model training device, comprising:
[0039] The building module is used to construct the initial mask matrix for each original model parameter in the initial large language model;
[0040] The splitting module is used to split the initial mask matrix for the current pruning round number to obtain the initial mask block corresponding to the current pruning round number;
[0041] The pruning module is used to determine the current mask block corresponding to the initial mask block based on the pre-built mask reconstruction dataset, the predetermined sparsity, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix. It also determines the current model parameters corresponding to the non-zero, non-target elements in the current mask block, and obtains the current model parameter matrix corresponding to the initial mask block.
[0042] The determination module is used to determine the pruning large language model based on the current model parameter matrix corresponding to each pruning round;
[0043] The first optimization module is used to optimize and train the pruned large language model based on a pre-built first optimization dataset to obtain the target large language model.
[0044] To address the aforementioned problems, this application provides a federalized large language model training device, comprising:
[0045] The second receiving module is used to receive the pruned large language model sent by the main computing node;
[0046] The second optimization module is used to optimize and train the pruned large language model based on a pre-built second optimization dataset to obtain the target large language model.
[0047] To address the aforementioned problems, this application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of any of the above-described federated large language model training methods.
[0048] The federated large language model training method, apparatus, and device in this application construct an initial mask matrix corresponding to the original model parameters. Subsequently, based on the initial mask matrix, the optimal mask matrix and the current model parameters corresponding to each non-zero element in the optimal mask matrix can be solved accurately and quickly. This achieves unstructured pruning of the initial large language model. That is, it realizes unstructured pruning of a portion of the model parameters / weights of the initial large language model by solving the sparse reconstruction problem, which simplifies the model structure of the large language model, improves the training speed of the model, and ensures the accuracy of model training.
[0049] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0050] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0051] Figure 1 This is a flowchart illustrating a federal large language model training method according to an embodiment of this application;
[0052] Figure 2 This is a flowchart illustrating a federal large language model training method according to yet another embodiment of this application;
[0053] Figure 3 This is a structural block diagram of a federal large language model training device according to another embodiment of this application;
[0054] Figure 4 This is a structural block diagram of a federal large language model training device according to another embodiment of this application;
[0055] Figure 5 This is a structural block diagram of an electronic device according to another embodiment of this application. Detailed Implementation
[0056] Various embodiments and features of this application are described herein with reference to the accompanying drawings.
[0057] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.
[0058] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0059] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.
[0060] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.
[0061] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.
[0062] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.
[0063] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.
[0064] This application provides a federated large language model training method, specifically applicable to the master computing node / leading party in federated learning. Specifically, the master computing node can be financial infrastructure. For example... Figure 1 As shown, the method in this embodiment includes the following steps:
[0065] Step S101: Construct an initial mask matrix for each original model parameter in the initial large language model;
[0066] In the specific implementation process, this step involves determining the corresponding element values for each original model parameter in the initial large language model; based on each element value, an initial mask matrix is constructed. That is, as long as an original model parameter exists, the corresponding element value is "1", thus an initial mask matrix with all element values being "1" can be constructed.
[0067] Step S102: For the current pruning round number, split the initial mask matrix to obtain the initial mask block corresponding to the current pruning round number;
[0068] In this step, the total number of pruning rounds can be pre-set based on the number of model parameters and the size of the initial mask matrix, for example, 10 rounds, 20 rounds, etc. The initial mask matrix can be split according to the number of rounds, or the number of split columns / target columns for each pruning round can be pre-determined, and the initial mask matrix can be split according to the number of split columns / target columns.
[0069] Taking the current pruning round 1 as an example, the initial mask matrix can be split to obtain the initial mask block corresponding to the current pruning round 1. Alternatively, taking the current pruning round 5 as an example, based on the initial mask blocks split from the previous 4 rounds, the remaining mask matrix in the initial mask matrix can be determined, and then the remaining mask matrix can be split to obtain the initial mask block corresponding to the current pruning round 5.
[0070] Step S103: Based on the pre-constructed mask reconstruction dataset, the predetermined sparsity rate, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix, determine the current mask block corresponding to the initial mask block, and determine the current model parameters corresponding to the non-zero, non-target elements in the current mask block, and obtain the current model parameter matrix corresponding to the initial mask block.
[0071] In this step, for the initial mask block (all elements are "1"), a predetermined sparsity ratio is used to determine the current mask block (which contains a predetermined proportion of zero elements). For the target mask column in the current mask block, the corresponding feature subset is determined based on the pre-constructed mask reconstruction data. Then, based on this feature subset, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix, the current model parameter column corresponding to the target column is determined. Repeating the above process determines the current model parameter column for each column in the current mask block. That is, each column of the mask in the current mask block corresponds to a column of current model parameters, and each column of model parameters is called the model parameter column. Finally, combining the current model parameter columns yields the current model parameter matrix corresponding to the current mask block / initial mask block.
[0072] In this step, when determining the current mask block corresponding to the initial mask block, and determining the current model parameters corresponding to the non-target elements with non-zero values in the current mask block, it is possible to re-determine which elements in the initial mask block have values of "0" and which elements have values of "1". This allows us to determine which neural network nodes in the initial large language model corresponding to "0" need to be pruned, while the neural network nodes in the initial large language model corresponding to "1" need to be retained.
[0073] Step S104: Determine the pruning large language model based on the current model parameter matrix corresponding to each pruning round number;
[0074] In this step, after obtaining the parameter matrices of each current model, it is possible to determine which neural network nodes in the initial large language model need to be retained and which neural network nodes need to be pruned. The current model parameters / current weights of the neural network nodes that need to be retained are obtained from the corresponding current model parameter matrices. Thus, the pruned large language model is obtained.
[0075] Step S105: Optimize and train the pruned large language model based on the pre-constructed first optimized dataset to obtain the target large language model.
[0076] In the specific implementation process of this step, the pruned large language model can be optimized based on the first optimized dataset to obtain initial first optimized parameters; the initial second optimized parameters obtained by optimizing the pruned large language model can be received from each slave computing node; parameter aggregation processing can be performed based on the first and second optimized parameters to obtain initial aggregated parameters; the initial aggregated parameters can be sent to each slave computing node so that each slave computing node can perform the next round of model optimization based on the initial aggregated parameters, and the current second optimized parameters can be received from each slave computing node until the predetermined optimization conditions are met, at which point the model optimization stops, and the current aggregated parameters are used as the target aggregated parameters to obtain the target large language model.
[0077] The method in this embodiment constructs an initial mask matrix corresponding to the original model parameters. Subsequently, based on the initial mask matrix, the optimal mask matrix and the current model parameters corresponding to each non-zero element in the optimal mask matrix can be solved accurately and quickly. This achieves unstructured pruning of the initial large language model. In other words, it achieves unstructured pruning of a portion of the model parameters / weights of the initial large language model by solving the sparse reconstruction problem, which simplifies the model structure of the large language model, improves the training speed of the model, and ensures the accuracy of model training.
[0078] Another embodiment of this application provides a federated large language model training method, applied to the master computing node. The overall process of model training is as follows:
[0079] Step S201, Installation and initialization of the federated big language model;
[0080] In this step, the financial infrastructure, acting as the lead provider, serves as the master computing node. It deploys the federated large language model collaborative network within its data center and provides a distributed node deployment and installation package, which is sent to each participant (financial institution) / slave computing node in the federated learning process. This enables each participant / slave computing node to deploy and install the components of the federated large language model collaborative network and join the network as a slave computing node. The framework of the federated learning collaborative network can be implemented using other software products with similar functionality, such as FATE, SecretFlow, PaddleFL, TensorFlow Federated, and Pysyft.
[0081] Step S202, data preprocessing;
[0082] In this step, the main computing node connects to its local bond business database to prepare high-quality dialogue data within its own business domain, thereby obtaining the first initial dataset. This first initial dataset contains questions and answers that may be asked in various financial transactions, serving as training data for pruning / masking reconstruction and model recovery / model optimization steps. Specifically, the dialogue data in the first initial dataset includes dialogue data of various types, such as zero-shot hints, few-shot hints, and knowledge-based hints.
[0083] 1. Zero-sample prompts: The institution does not provide task-related examples, but directly prompts the language model to give task-related answers.
[0084] 2. Small sample suggestion: The institution provides a small number of sample questions, such as task descriptions, etc.
[0085] 3. Knowledge Base Hints: The organization provides a knowledge base related to the question and its keywords. Hint words are generated using the contextual content of the knowledge base. The main computing node can extract text and perform semantic understanding on the knowledge base. It then calculates the relevance between vectorized knowledge base document fragments and the input question using a vector distance metric. The most similar document fragments are combined with the input question to form hint words, thereby obtaining dialogue data. The vector distance metric can be implemented using methods such as cosine similarity, dot product, Hamming distance, and other vector distance measures.
[0086] In this step, in addition to bond business dialogue data, the main computing node can prepare general Chinese dialogue data in advance to expand the initial dataset. The sources of the Chinese dialogue data include, but are not limited to, open-source instruction datasets such as Stanford Alpaca, self-instruct, GPT4-for-LLM, alpaca-chinese-dataset, BELLE, and COIG. For non-Chinese dialogue data, other models with similar functionality, such as NLLB-200 or Delta LM, are used to identify the language and translate it into Chinese.
[0087] After the main compute node completes the preparation of the initial dataset, it randomly selects a certain proportion of the data to serve as the mask reconstruction dataset for mask reconstruction and the first optimization dataset for model tuning / optimization training, respectively. For example, the main compute node defaults to selecting 40% of the data as the mask reconstruction dataset D_reconstruction. center Additionally, the 40% of data that is not duplicated constitutes the first optimized dataset, D_recover. center The remainder is test data used to evaluate the accuracy of the algorithm.
[0088] Step S203: Determine the corresponding element values for each original model parameter in the initial large language model; construct the initial mask matrix based on each element value;
[0089] In this step, for each original model parameter of the initial large language model M, an original model parameter matrix containing each original model parameter is constructed, and an initial mask matrix corresponding to this original model parameter matrix is created. Initial mask matrix All elements in the set have a value of "1".
[0090] Step S204: Determine the target number of columns based on the remaining video memory capacity;
[0091] In this step, the main compute node determines the target number of columns as follows: It determines the reference memory capacity GM based on the remaining memory capacity; and it determines the target number of columns B based on the reference memory capacity and the number of elements in each column of the initial mask matrix. k .
[0092] Specifically, the master compute node can adaptively select / determine the target number of columns B based on computing resources such as video memory and the model size. k Taking the determination of the target number of columns based on the remaining video memory capacity as an example, the main compute node first reads the current video memory status / available video memory capacity / remaining video memory capacity. Taking NVIDIA-GPU video memory as an example, the `nvidia-smi` command can be used to subtract the video memory capacity consumed by the system layer, etc., from the total video memory capacity to obtain the available video memory / remaining video memory capacity. Then, based on the remaining video memory capacity and a predetermined ratio, the reference first-slave capacity GM is determined. That is, a portion is reserved for temporary variables and other overhead, and by default, 60% of the remaining video memory capacity is used as the reference video memory capacity GM (in GB).
[0093] After determining the reference memory capacity GM, the master compute node can determine the target number of columns B based on the number of elements in each column of the initial mask matrix. k Assuming the current pruning model uses fp32 data type for storage, and the number of parameters / elements per column is s, then the target number of columns is B. k The calculation can be performed using the following formulas (1) and (2):
[0094] (1)
[0095] (2)
[0096] Where GM represents the reference memory capacity; s represents the number of elements in each column; B represents the set of positive integers; k Indicates the target column number.
[0097] Step S205: For the current number of pruning rounds, split the initial mask matrix based on the target number of columns to obtain the initial mask block corresponding to the current number of pruning rounds;
[0098] In this step, after determining the target number of columns, the initial mask matrix can be adjusted according to the target number of columns. The mask is split to obtain the initial mask block corresponding to each round of pruning. For example, if the initial mask matrix contains 100 columns and the target number of columns is 20, then the initial mask matrix can be split into 5 mask matrix blocks according to the 20-column splitting method. Each round of pruning corresponds to one mask matrix block, which is the initial mask block. A total of 5 rounds of pruning processing are performed.
[0099] Step S206: Based on the pre-built mask reconstruction dataset, the predetermined sparsity, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix, determine the current mask block corresponding to the initial mask block, and determine the current model parameters corresponding to the non-zero, non-target elements in the current mask block, and obtain the current model parameter matrix corresponding to the initial mask block.
[0100] In the specific implementation of this step, the predetermined sparsity rate can be, for example, 10%, 20%, 50%, etc. Taking a sparsity rate of p% as an example, this means that p% of the target elements are determined from the initial mask block, and the element values of these target elements are set to zero; the element values of the remaining 1-p% of non-target elements are still "1", thus reconstructing the current mask block. Simultaneously, this is combined with the pre-constructed mask reconstruction dataset D_reconstruction. center Original model parameter matrix Original model parameter matrix The inverse Hessian matrix is used to further determine the current model parameters corresponding to each non-target element (element with a value of 1), thereby obtaining the current model parameter matrix corresponding to the current mask block / initial mask block.
[0101] In this embodiment, model parameters / weights The optimal value after reconstruction / update can be obtained by solving each column of the sparse reconstruction matrix. The calculations are performed, and the solutions are shown in formulas (3) and (4) below:
[0102] (3)
[0103] (4)
[0104] in, For a subset of the input features, the corresponding weights in the i-th column have not yet been pruned. Specifically, the data D_reconstruction can be reconstructed using a mask. center get; This refers to the original model parameter matrix / original weight matrix corresponding to the split initial mask block; for The inverse Hessian matrix.
[0105] In optimization problems, the Hessian matrix is the second derivative matrix of the objective function at a specific point, containing local information about the curvature of the objective function. Therefore, using its inverse matrix in this step allows for more efficient weight updates, thereby obtaining the current mask block corresponding to the initial mask block, and determining the updated current model parameter column corresponding to each mask column in the current mask block. .
[0106] In this step, for the initial mask block of this pruning round (all elements are "1"), several target elements with zero values can be determined according to a predetermined sparsity rate p%, thus obtaining the first current mask block (the elements with p% values are zero, and the remaining 1-p% elements have "1" values). Then, for each column in the first current mask block, the model parameters / weights are updated to obtain the current model parameter column / current weight column corresponding to each mask column. That is, for the target mask columns in the first current mask block, based on the feature subset (obtained through the mask reconstruction dataset) and the original model parameter matrix corresponding to the initial mask block... The original model parameter matrix (i.e., the original weight matrix) inverse Hessian matrix The process begins by determining the current model parameters / weights corresponding to the elements with a value of "1" in the target mask column, thus obtaining the current model parameter column / weight column corresponding to the target mask column. Repeating this process yields the first current model parameter matrix corresponding to the first current mask block. Then, based on predetermined evaluation metrics, it can be assessed whether the sub-model corresponding to the first current model parameter matrix meets predetermined conditions. If the predetermined conditions are not met, a second current mask block and a second current model parameter matrix are determined, until the predetermined conditions are met. Finally, the Zth current mask block is used as the final current mask block for this round of pruning, and the Zth current model parameter matrix is used as the final current model parameter matrix for this round of pruning.
[0107] In this embodiment, only updating is required. subset of Therefore, it is only necessary to calculate the inverse of the Hessian matrix associated with the subset and update only the corresponding weights, thereby reducing the amount of computation and improving computational efficiency. The steps are as follows:
[0108] Step 1: Sort the subsets in a fixed order, let It includes all features, and each subsequent subset reduces the number of features by one compared to the previous one, that is:
[0109] (5)
[0110] in, The feature corresponding to the index. The inverse of the Hessian matrix used for the subset. .
[0111] Step 2: Using the following formula (6), through Gaussian elimination, from... To efficiently calculate the next inverse Hessian matrix, we remove the first row and first column corresponding to j from the original Hessian matrix.
[0112] (6)
[0113] Step 3: Follow the pruning mask Column iteration calculation and the corresponding inverse Hessian matrix subset of It also updates the weights that need to be pruned, requiring only one calculation in the process. This reduces computational overhead.
[0114] Step S207: Based on the predetermined pruning algorithm, compensate each current model parameter in the current model parameter matrix to obtain the target model parameter matrix corresponding to the initial mask block, which is used to determine the pruned large language model.
[0115] In this step, a compensation matrix corresponding to the current model parameter matrix can be determined based on a predetermined pruning algorithm, OBS (Optimal Brain Surgeon). The current model parameter matrix is then compensated based on this compensation matrix to obtain the compensated target model parameter matrix corresponding to each initial mask block. The compensation matrix contains several compensation values, each corresponding one-to-one with a current model parameter in the current model parameter matrix.
[0116] Step S208: Determine the pruning large language model based on the target model parameter matrix corresponding to each pruning round number;
[0117] In this step, after obtaining the parameter matrices of each target model, these matrices can be merged to obtain the overall target model parameter matrix corresponding to the initial large language model. Based on this overall target model parameter matrix, it can be determined which neural network nodes in the initial large language model need to be retained and which need to be pruned (i.e., neural network nodes with non-zero values need to be retained, and those with zero values need to be pruned). Then, the current model parameters / weights of the neural network nodes that need to be retained are obtained from the overall target model parameter matrix. Thus, the pruned large language model is obtained. .
[0118] Step S209: Quantize and compress the parameters of each model in the pruned large language model to obtain a compressed pruned large language model for optimization training.
[0119] In this step, after obtaining the pruned large language model... Afterwards, further research can be conducted on the pruned large language model. The model parameters / weights are quantized and compressed, for example, N-bit quantization compression (the specific value of N can be selected according to the model size, with the default N=16). This yields the compressed pruned large language model. .
[0120] Step S210: The pruned large language model is optimized and trained based on the pre-constructed first optimized dataset to obtain the target large language model.
[0121] In this step, the initial large language model inevitably loses its original knowledge during model pruning and quantization; on the other hand, the generalization ability of the large language model urgently needs to be enhanced due to the different business corpora from multiple institutions. Therefore, this step performs federated model optimization / recovery to improve the performance of the pruned model. Specifically, fine-tuning can be achieved by using the first optimized dataset D_recover generated in step S202 on the main computing node. center Simultaneously, it jointly utilizes a second optimized dataset D_recover from the compute nodes based on their local locality. nodei Perform federated large language model recovery / optimization. The specific process of model optimization / model recovery is as follows:
[0122] Step S2101: Optimize the pruned large language model based on the first optimized dataset to obtain the initial first optimization parameters;
[0123] In this step, the main compute node can specifically be based on the first optimized dataset D_recover. center For the compressed, pruned large language model Perform model optimization to obtain the initial first optimization parameters.
[0124] Step S2102: Receive the initial second optimization parameters obtained by optimizing the pruned large language model from each computing node;
[0125] In this step, the main computing node obtains the pruned large language model through pruning. Then, this pruned large language model can be... The pruned model is sent to each slave compute node, which is also the master compute node. The model parameters are all sent to each slave computing node. Therefore, each slave computing node can also use the same model optimization method as the master computing node to prune the large language model. Perform model tuning / optimization to obtain the second optimization parameters, and then send the second optimization parameters to the main compute node.
[0126] Step S2103: Perform parameter aggregation processing based on the first optimization parameters and the second optimization parameters to obtain initial aggregation parameters;
[0127] In this step, the master compute node aggregates the parameter fine-tuning results of all compute nodes according to the federated aggregation strategy. Aggregation strategies include, but are not limited to, summation, arithmetic mean, weighted average, maximum value, etc., with the arithmetic mean used by default.
[0128] Step S2104: Send the initial aggregation parameters to each slave computing node so that each slave computing node can perform the next round of model optimization based on the initial aggregation parameters, and receive the current second optimization parameters sent by each slave computing node until the predetermined optimization conditions are met, then stop the model optimization, take the current aggregation parameters as the target aggregation parameters, and obtain the target large language model.
[0129] In this step, after obtaining the initial aggregation parameters, the main computing node will determine the current large language model based on the initial aggregation parameters. Then it can be used as a large language model The initial aggregation parameters are sent to the slave compute nodes. Simultaneously, the master compute node determines the current large language model. Whether the predetermined optimization conditions are met, i.e., determining the current large language model. If convergence is not achieved, and the predetermined optimization conditions are not met, the master computing node will perform the next round of model tuning / optimization based on the current large language model to obtain the current first optimization parameters. It will also receive the current second optimization parameters obtained by each slave computing node based on the next round of model optimization, until the predetermined optimization conditions are met. At this point, the model tuning will stop, and the current aggregation parameters will be used as the target aggregation parameters to obtain the target large language model.
[0130] Another embodiment of this application provides a federated large language model training method, which can be applied to various slave computing nodes / participants in federated learning. Specifically, the slave computing nodes can be financial institutions. Figure 2 As shown, the method in this embodiment includes the following steps:
[0131] Step S301: Receive the pruned large language model sent by the master computing node;
[0132] Step S302: Optimize and train the pruned large language model based on the pre-built second optimized dataset to obtain the target large language model.
[0133] In the specific implementation process, this step involves the computing node receiving the pruned large language model. Then, the local second optimized dataset can be used to optimize / restore the pruned large language model. Specifically, each slave compute node optimizes / restores the pruned large language model based on its local second optimized dataset to obtain initial second optimized parameters; these initial second optimized parameters are sent to the master compute node, which then performs parameter aggregation based on the first and second optimized parameters to obtain initial aggregated parameters; the slave node receives the initial aggregated parameters from the master compute node and performs the next round of model optimization based on them until predetermined optimization conditions are met. At this point, model optimization stops, and the received current aggregated parameters are used as the target aggregated parameters to obtain the target large language model.
[0134] The method in this embodiment receives a pruned large language model obtained based on unstructured pruning from the master computing node, and performs federated model training on the pruned large language model. This simplifies the model structure of the large language model, improves the training speed, and ensures the accuracy of model training.
[0135] Another embodiment of this application provides a federated large language model training method, applied from a computing node, specifically including the following steps:
[0136] Step S401, Installation and initialization of the federated big language model;
[0137] In this step, each slave compute node receives the installation package for the Federated Large Language Model Collaboration Network from the master compute node. Each participating / slave compute node then deploys and installs the components of the Federated Large Language Model Collaboration Network and joins the network as a slave compute node. The framework of the Federated Learning Collaboration Network can be implemented using other software products with similar functionality, such as FATE, SecretFlow, PaddleFL, TensorFlow Federated, and Pysyft.
[0138] Step S402, data preprocessing;
[0139] In this step, the computing node connects to its local bond business database to prepare high-quality dialogue data within its own business domain, thereby obtaining a second initial dataset. This second initial dataset contains questions that may be asked in various financial transactions, along with their answers, serving as training data for importance assessment and model recovery steps. Specifically, the dialogue data in the second initial dataset includes dialogue data of various types, such as zero-shot prompts, few-shot prompts, and knowledge-based prompts.
[0140] 1. Zero-sample prompts: The institution does not provide task-related examples, but directly prompts the language model to give task-related answers.
[0141] 2. Small sample suggestion: The institution provides a small number of sample questions, such as task descriptions, etc.
[0142] 3. Knowledge Base Hints: The organization provides a knowledge base related to the question and its keywords. Hint words are generated using the contextual content of the knowledge base. The main computing node can extract text and perform semantic understanding on the knowledge base. It then calculates the relevance between vectorized knowledge base document fragments and the input question using a vector distance metric. The most similar document fragments are combined with the input question to form hint words, thereby obtaining dialogue data. The vector distance metric can be implemented using methods such as cosine similarity, dot product, Hamming distance, and other vector distance measures.
[0143] After the computing nodes complete the preparation of the second initial dataset, a certain proportion of the data is randomly selected as the second optimization dataset for model optimization / model recovery. By default, 40% of the data is selected as the second optimization dataset D_recover. nodei The rest are test data.
[0144] Step S403: Receive the pruned large language model sent by the master computing node;
[0145] Step S404: Optimize and train the pruned large language model based on the pre-built second optimized dataset to obtain the target large language model.
[0146] In this step, the specific process of model optimization / model recovery is as follows:
[0147] Step S4041: Optimize the pruned large language model based on the second optimized dataset to obtain the initial second optimized parameters;
[0148] In this step, the compute node can specifically be based on the second optimized dataset D_recover. nodei Pruned large language model Model optimization yields initial second optimization parameters.
[0149] Step S4042: Send the initial second optimization parameters to the master computing node so that the master computing node can perform parameter aggregation processing based on the first optimization parameters and the second optimization parameters to obtain the initial aggregated parameters;
[0150] In this step, after obtaining the initial second optimization parameters, the slave compute nodes can send them to the master compute node. The master compute node then aggregates the parameter fine-tuning results from all compute nodes according to the federated aggregation strategy to obtain the initial aggregated parameters.
[0151] Step S4043: Receive the initial aggregation parameters sent by the master computing node, perform the next round of model optimization based on the initial aggregation parameters, and stop model optimization when the predetermined optimization conditions are met. Use the received current aggregation parameters as the target aggregation parameters to obtain the target large language model.
[0152] In this step, after obtaining the initial aggregation parameters, the master compute node determines the current large language model based on these parameters and sends the initial aggregation parameters / current large language model to the slave compute nodes. Simultaneously, the slave compute nodes determine whether the current large language model meets the predetermined tuning conditions, i.e., whether the current large language model has converged. If the predetermined optimization conditions are not met, the slave compute nodes will perform the next round of model optimization based on the current large language model, obtain the current second optimization parameters, and send these second optimization parameters to the master compute node. This process continues until the predetermined optimization conditions are met, at which point model optimization stops. The slave compute nodes then use the current aggregation parameters as the target aggregation parameters to obtain the target large language model.
[0153] The method in this embodiment receives a pruned large language model obtained based on unstructured pruning from the master computing node, and performs federated model training on the pruned large language model. This simplifies the model structure of the large language model, improves the training speed, and ensures the accuracy of model training.
[0154] Another embodiment of this application provides a federated large language model training device, such as... Figure 3 As shown, it includes:
[0155] Module 11 is used to construct the initial mask matrix for each original model parameter in the initial large language model;
[0156] The splitting module 12 is used to split the initial mask matrix for the current pruning round number to obtain the initial mask block corresponding to the current pruning round number;
[0157] The pruning module 13 is used to determine the current mask block corresponding to the initial mask block based on the pre-built mask reconstruction dataset, the predetermined sparsity, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix, and to determine the current model parameters corresponding to the non-zero, non-target elements in the current mask block, and to obtain the current model parameter matrix corresponding to the initial mask block.
[0158] Module 14 is used to determine the pruning large language model based on the current model parameter matrix corresponding to each pruning round.
[0159] The first optimization module 15 is used to optimize and train the pruned large language model based on the pre-built first optimization dataset to obtain the target large language model.
[0160] In this embodiment, the construction module is specifically used to: determine the corresponding element values for each original model parameter in the initial large language model; and construct an initial mask matrix based on each element value.
[0161] In this embodiment, the splitting module specifically includes a column number determination unit and a splitting unit; the column number determination unit is used to: determine the target number of columns based on the remaining video memory capacity; the splitting unit is used to: split the initial mask matrix based on the target number of columns to obtain the initial mask block corresponding to the current pruning round number.
[0162] In this embodiment, the column number determination unit is specifically used to: determine the reference video memory capacity based on the remaining video memory capacity; and determine the target column number based on the reference video memory capacity and the number of elements in each column of the initial mask matrix.
[0163] In this embodiment, the federated large language model training device further includes a compensation module. The compensation module is used to: compensate each current model parameter in the current model parameter matrix based on a predetermined pruning algorithm, so as to obtain a target model parameter matrix corresponding to the initial mask block for determining the pruned large language model.
[0164] In this embodiment, the federated large language model training device further includes a compression module. The compression module is used to: perform quantization and compression processing on each model parameter of the pruned large language model to obtain a compressed pruned large language model for optimized training.
[0165] In this embodiment, the federated large language model training device further includes a first sending module, which is used to send the pruned large language model to each slave computing node.
[0166] The first optimization module is specifically used to: optimize the pruned large language model based on the first optimization dataset to obtain initial first optimization parameters;
[0167] Receive the initial second optimization parameters obtained by optimizing the pruned large language model from each computing node;
[0168] Based on the first optimization parameters and the second optimization parameters, parameter aggregation is performed to obtain the initial aggregation parameters;
[0169] The initial aggregation parameters are sent to each slave computing node so that each slave computing node can perform the next round of model optimization based on the initial aggregation parameters. The current second optimization parameters sent by each slave computing node are received until the predetermined optimization conditions are met. Then the model optimization stops and the current aggregation parameters are used as the target aggregation parameters to obtain the target large language model.
[0170] The federated learning-based model training device in this embodiment constructs an initial mask matrix corresponding to the original model parameters. Subsequently, based on the initial mask matrix, the optimal mask matrix and the current model parameters corresponding to each non-zero element in the optimal mask matrix can be solved accurately and quickly. This achieves unstructured pruning of the initial large language model. That is, it realizes unstructured pruning of a portion of the model parameters / weights of the initial large language model by solving the sparse reconstruction problem, which simplifies the model structure of the large language model, improves the training speed of the model, and ensures the accuracy of model training.
[0171] Another embodiment of this application provides a federated large language model training device, such as... Figure 4 As shown, it includes:
[0172] The second receiving module 21 is used to receive the pruned large language model sent by the main computing node;
[0173] The second optimization module 22 is used to optimize and train the pruned large language model based on the pre-built second optimization dataset to obtain the target large language model.
[0174] In this embodiment, the second optimization module is specifically used to: optimize the pruned large language model based on the second optimization dataset to obtain initial second optimization parameters; send the initial second optimization parameters to the main computing node so that the main computing node can perform parameter aggregation processing based on the first optimization parameters and the second optimization parameters to obtain initial aggregation parameters; receive the initial aggregation parameters sent by the main computing node, perform the next round of model optimization based on the initial aggregation parameters, and stop model optimization when the predetermined optimization conditions are met, and use the received current aggregation parameters as the target aggregation parameters to obtain the target large language model.
[0175] The federated learning-based model training device in this embodiment receives a pruned large language model obtained from unstructured pruning sent by the master computing node, and performs federated model training on the pruned large language model. This simplifies the model structure of the large language model, improves the training speed, and ensures the accuracy of model training.
[0176] Another embodiment of this application provides an electronic device, such as... Figure 5 As shown, it includes at least a memory 1 and a processor 2. The memory 1 stores a computer program, and the processor 2 performs the following method steps when executing the computer program in the memory 1:
[0177] Step 1: Construct an initial mask matrix for each original model parameter in the initial large language model;
[0178] Step 2: For the current pruning round number, split the initial mask matrix to obtain the initial mask block corresponding to the current pruning round number;
[0179] Step 3: Based on the pre-built mask reconstruction dataset, the predetermined sparsity, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix, determine the current mask block corresponding to the initial mask block, and determine the current model parameters corresponding to the non-zero, non-target elements in the current mask block, and obtain the current model parameter matrix corresponding to the initial mask block.
[0180] Step 4: Determine the pruning large language model based on the current model parameter matrix corresponding to each pruning round;
[0181] Step 5: Optimize and train the pruned large language model based on the pre-built first optimization dataset to obtain the target large language model.
[0182] Alternatively, implement the following steps:
[0183] Step 1: Receive the pruned large language model sent by the master computing node;
[0184] Step 2: Optimize and train the pruned large language model based on the pre-built second optimized dataset to obtain the target large language model.
[0185] The specific implementation process of the above method steps can be found in the embodiments of the above arbitrary federated large language model training method, which will not be repeated here.
[0186] In this embodiment, the electronic device constructs an initial mask matrix corresponding to the original model parameters. Subsequently, based on the initial mask matrix, it can accurately and quickly solve for the optimal mask matrix and the current model parameters corresponding to each non-zero element in the optimal mask matrix. This achieves unstructured pruning of the initial large language model. That is, it achieves unstructured pruning of a portion of the model parameters / weights of the initial large language model by solving the sparse reconstruction problem, which simplifies the model structure of the large language model, improves the training speed of the model, and ensures the accuracy of model training.
[0187] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.
Claims
1. A federated large language model training method, applied to the master computing node, characterized in that, include: For each original model parameter in the initial large language model, construct an initial mask matrix; For the current pruning round number, the initial mask matrix is split to obtain the initial mask block corresponding to the current pruning round number; Based on the pre-built mask reconstruction dataset, the predetermined sparsity, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix, the current mask block corresponding to the initial mask block is determined, and the current model parameters corresponding to the non-zero, non-target elements in the current mask block are determined, thus obtaining the current model parameter matrix corresponding to the initial mask block. Based on the current model parameter matrix corresponding to each pruning round, determine the pruning large language model; The pruned large language model is optimized and trained based on a pre-built first optimized dataset to obtain the target large language model. The step of splitting the initial mask matrix for the current pruning round number to obtain the initial mask block corresponding to the current pruning round number specifically includes: Determine the reference video memory capacity based on the remaining video memory capacity; The target number of columns is determined based on the reference memory capacity and the number of elements in each column of the initial mask matrix; The initial mask matrix is split based on the target column number to obtain the initial mask block corresponding to the current pruning round number.
2. The method as described in claim 1, characterized in that, The construction of an initial mask matrix for each original model parameter in the initial large language model specifically includes: Determine the corresponding element values for each original model parameter in the initial large language model; Based on the values of each element, an initial mask matrix is constructed.
3. The method as described in claim 1, characterized in that, After obtaining the current model parameter matrix, the method further includes: Based on a predetermined pruning algorithm, each current model parameter in the current model parameter matrix is compensated to obtain the target model parameter matrix corresponding to the initial mask block, which is used to determine the pruned large language model.
4. The method as described in claim 1, characterized in that, After obtaining the pruned large language model, the method further includes: The parameters of each model in the pruned large language model are quantized and compressed to obtain a compressed pruned large language model for optimization training.
5. A federated large language model training method, applied from computing nodes, characterized in that, include: Receive the pruned large language model sent by the master computing node; The pruned large language model is optimized and trained based on a pre-built second optimized dataset to obtain the target large language model. The pruned large language model is obtained using the federated large language model training method as described in any one of claims 1-4.
6. A federal large language model training device, characterized in that, include: The building module is used to construct the initial mask matrix for each original model parameter in the initial large language model; The split module is used to determine the reference video memory capacity based on the remaining video memory capacity; The target number of columns is determined based on the reference memory capacity and the number of elements in each column of the initial mask matrix; The initial mask matrix is split based on the target column number to obtain the initial mask block corresponding to the current pruning round number; The pruning module is used to determine the current mask block corresponding to the initial mask block based on the pre-built mask reconstruction dataset, the predetermined sparsity, the original model parameter matrix corresponding to the initial mask block, and the inverse Hessian matrix of the original model parameter matrix. It also determines the current model parameters corresponding to the non-zero, non-target elements in the current mask block, and obtains the current model parameter matrix corresponding to the initial mask block. The determination module is used to determine the pruning large language model based on the current model parameter matrix corresponding to each pruning round; The first optimization module is used to optimize and train the pruned large language model based on a pre-built first optimization dataset to obtain the target large language model.
7. A federal large language model training device, characterized in that, include: The second receiving module is used to receive the pruned large language model sent by the main computing node; The second optimization module is used to optimize and train the pruned large language model based on a pre-built second optimization dataset to obtain the target large language model. The pruned large language model is obtained using the federated large language model training method as described in any one of claims 1-4.
8. An electronic device, characterized in that, It includes at least a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of the federated large language model training method according to any one of claims 1-4 or 5.
Citation Information
Patent Citations
Model training method and device, equipment and storage medium
CN115829024A
Structured pruning method and apparatus based on local sparsity constraints
WO2023098544A1