Federal training fine-tuning method and device based on low-rank approximate quantization

Abstract

The application provides a federated training fine-tuning method and device based on low-rank approximation quantization. The method comprises: performing multi-round alternating optimization of the original weight matrix corresponding to the initial local generative language model of the current iteration training round based on quantization and low-rank approximation processing to obtain a target integer weight matrix corresponding to the original weight matrix and a target low-rank approximation matrix; fine-tuning the target low-rank approximation matrix using local data to obtain a trained low-rank approximation matrix corresponding to the current iteration training round; and taking the target integer weight matrix and the trained low-rank approximation matrix as the local quantized generative language model parameters corresponding to the current iteration training round and sending them to an aggregation end device in a federated learning system to which the application belongs for global model aggregation. The application can solve the problem that the current generative language model is difficult to balance the device computing resource consumption, calculation accuracy, communication resource occupancy rate and privacy protection in the federated learning scenario.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a federated training fine-tuning method and apparatus based on low-rank approximate quantization. Background Technology

[0002] With the rapid development of Artificial Intelligence Generated Content (AIGC), especially the increasing maturity of generative language models, large language models have made groundbreaking progress in natural language processing tasks. These models typically contain hundreds of millions or even billions of parameters. However, as the model size increases, the demands on computing resources and storage also increase dramatically, posing significant challenges to their practical deployment. Federated learning, as an emerging distributed learning method, allows multiple participants to train models locally, enabling efficient training and deployment of generative language models while ensuring privacy. Although federated learning can avoid the leakage of sensitive data, the local operation of participating devices still presents challenges such as high computational resource consumption, high computational efficiency, and high communication resource utilization.

[0003] Currently, quantization techniques have emerged to address the computational pressure and high communication resource consumption caused by the limited computing resources of participating devices in federated learning. Quantization converts the high-precision floating-point weights of the model (such as 32-bit floating-point numbers) into low-precision values ​​(such as 8 bits or lower). However, while quantization can reduce model size and computational burden to some extent, it may also introduce a certain loss of precision, especially in extremely low-bit quantization (such as 2-bit quantization), where the impact of quantization error on model performance is more significant.

[0004] Therefore, there is an urgent need to address the problem that existing technical solutions for generative language models in federated learning scenarios cannot meet the balance requirements of computational resource consumption, communication resource utilization, and privacy protection while maintaining high model accuracy. Summary of the Invention

[0005] In view of this, embodiments of this application provide a federated training fine-tuning method and apparatus based on low-rank approximate quantization to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of this application provides a federated training fine-tuning method based on low-rank approximate quantization, comprising:

[0007] The original weight matrix corresponding to the initial local generative language model in the current iteration training round is subjected to multiple rounds of alternating optimization based on quantization and low-rank approximation to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix.

[0008] The target low-rank approximation matrix is ​​fine-tuned using local data to obtain the trained low-rank approximation matrix corresponding to the current iteration training round. The target integer weight matrix and the trained low-rank approximation matrix are used as the local quantized generative language model parameters corresponding to the current iteration training round and sent to the aggregation end device in the federated learning system. The aggregation end device then aggregates the received local quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round.

[0009] In some embodiments of this application, before performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model of the current iteration training round, the method further includes:

[0010] The system receives the initial global generative language model parameters corresponding to the current iteration training round from the aggregation terminal device in its own federated learning system. If the current iteration training round is the preset first iteration training round, the initial global generative language model parameters are generative language model parameters pre-acquired by the aggregation terminal device. If the current iteration training round is not the first iteration training round, the initial global generative language model parameters are the global generative language model parameters corresponding to the previous iteration training round, pre-aggregated by the aggregation terminal device.

[0011] The initial global generative language model parameters are initialized based on local data to obtain the initial local generative language model parameters for the current iteration training round. The initial local generative language model parameters include the original weight matrix corresponding to the initial local generative language model for the current iteration training round.

[0012] In some embodiments of this application, the step of performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model in the current iteration training round to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix includes:

[0013] Based on the preset initial rank and initial quantization bit width, the original weight matrix corresponding to the initial local generative language model in the current iteration training round is initialized with low-rank approximation and quantization to obtain the initial integer weight matrix and the initial low-rank approximation matrix corresponding to the original weight matrix.

[0014] Based on the initial low-rank approximation matrix, the original weight matrix is ​​subjected to multiple rounds of alternating optimization based on quantization and low-rank approximation to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix.

[0015] In some embodiments of this application, the initialization of the original weight matrix corresponding to the initial local generative language model in the current iteration training round based on a preset initial rank and initial quantization bit width, performing low-rank approximation and quantization processing to obtain the initial integer weight matrix and initial low-rank approximation matrix corresponding to the original weight matrix, includes:

[0016] Based on the preset initial rank, singular value decomposition is performed on the original weight matrix of the current iteration training round to obtain the initial low-rank approximate matrix corresponding to the original weight matrix, which is composed of the left singular matrix and the right singular matrix.

[0017] Set the left singular matrix and the right singular matrix in the initial low-rank approximation matrix to 0;

[0018] Based on the preset initial quantization bit width, the initial integer weight matrix is ​​mapped to the integer space to obtain the initial integer weight matrix corresponding to the original weight matrix.

[0019] In some embodiments of this application, the step of performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix according to the initial low-rank approximation matrix to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix includes:

[0020] The quantization residual matrix corresponding to the first round of alternating optimization process is determined based on the original weight matrix and the initial integer weight matrix, and the singular value decomposition is performed on the quantization residual matrix to obtain the low-rank approximate matrix composed of the left singular matrix and the right singular matrix corresponding to the first round of alternating optimization process.

[0021] Based on the low-rank approximation matrix corresponding to the alternating optimization process, at least one alternating optimization step is performed, wherein the alternating optimization step includes: in the current non-first round alternating optimization process, determining the integer weight matrix corresponding to the current non-first round alternating optimization process according to the original weight matrix and the low-rank approximation matrix of the previous round; determining the quantization residual matrix corresponding to the current non-first round alternating optimization process according to the original weight matrix and the integer weight matrix corresponding to the current non-first round alternating optimization process, and performing singular value decomposition on the quantization residual matrix to obtain the low-rank approximation matrix corresponding to the current non-first round alternating optimization process; and jointly optimizing the integer weight matrix and the low-rank approximation matrix corresponding to the current non-first round alternating optimization process based on a preset objective function to obtain the optimized integer weight matrix and the optimized low-rank approximation matrix corresponding to the current non-first round alternating optimization process.

[0022] The optimized integer weight matrix obtained from the last executed alternating optimization step is used as the target integer weight matrix corresponding to the original weight matrix, and the optimized low-rank approximation matrix obtained from the last executed alternating optimization step is used as the target low-rank approximation matrix corresponding to the original weight matrix.

[0023] In some embodiments of this application, the objective function is used to jointly optimize the target integer weight matrix and the target low-rank approximation matrix by minimizing the Frobinius norm between the original weight matrix, the target integer weight matrix, and the target low-rank approximation matrix.

[0024] In some embodiments of this application, the step of fine-tuning the target low-rank approximation matrix using local data to obtain the trained low-rank approximation matrix corresponding to the current training iteration includes:

[0025] The target integer weight matrix is ​​stored in a preset lookup table to map the target integer weight matrix back to simulated high-precision floating point values;

[0026] The target integer weight matrix is ​​fixed, and the target low-rank approximation matrix is ​​updated with gradients using local data to obtain the trained low-rank approximation matrix corresponding to the current training iteration.

[0027] Another aspect of this application provides a federated training fine-tuning apparatus based on low-rank approximate quantization, comprising:

[0028] The alternating optimization module is used to perform multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model in the current iteration training round, so as to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix.

[0029] The optimization and fine-tuning module is used to fine-tune the target low-rank approximation matrix using local data to obtain the trained low-rank approximation matrix corresponding to the current iteration training round. The target integer weight matrix and the trained low-rank approximation matrix are used as the local quantized generative language model parameters corresponding to the current iteration training round and sent to the aggregation end device in the federated learning system where it is located. The aggregation end device then aggregates the received local quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round.

[0030] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the described federated training fine-tuning method based on low-rank approximation quantization.

[0031] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the described federated training fine-tuning method based on low-rank approximation quantization.

[0032] The fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the described federated training fine-tuning method based on low-rank approximation quantization.

[0033] The federated training fine-tuning method based on low-rank approximation quantization provided in this application performs multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model in the current iteration training round to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix. The target low-rank approximation matrix is ​​then fine-tuned using local data to obtain the trained low-rank approximation matrix corresponding to the current iteration training round. The target integer weight matrix and the trained low-rank approximation matrix are used as the local quantized generative language model parameters corresponding to the current iteration training round and sent to the aggregation device in the federated learning system. The aggregation device then aggregates the received local quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round. Training the generative language model under the federated learning framework ensures data privacy protection and guarantees data security during training, preventing the leakage of sensitive user information. Simultaneously, combining low-rank approximation technology with quantization of the generative language model and reducing quantization errors through optimized quantization strategies significantly improves the model's performance under low-precision quantization. By employing an alternating optimization method, alternating between quantization and singular value decomposition, quantization bias can be effectively reduced, improving the initialization quality of the quantized model and thus enhancing fine-tuning performance. Subsequently, combining this with low-rank adaptation fine-tuning technology, performing low-rank adapter fine-tuning after quantization avoids the computational burden of numerous parameter updates in traditional fine-tuning methods, optimizes the utilization efficiency of participating devices' computing resources, and reduces device resource occupancy. Furthermore, by reducing the computational and communication resource consumption of participating devices and achieving data privacy protection, computational accuracy can be further improved, significantly enhancing the generalization ability and adaptability of downstream tasks and ensuring efficient operation in federated learning scenarios.

[0034] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0035] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings:

[0037] Figure 1 This is a schematic diagram of the first process of a federated training fine-tuning method based on low-rank approximation quantization in one embodiment of this application.

[0038] Figure 2 This is a schematic diagram of a second flowchart of a federated training fine-tuning method based on low-rank approximation quantization in one embodiment of this application.

[0039] Figure 3 This is a schematic diagram of the third process of the federated training fine-tuning method based on low-rank approximation quantization in one embodiment of this application.

[0040] Figure 4 This is a schematic diagram illustrating the execution flow of the federated training fine-tuning method based on low-rank approximation quantization in an application example of this application.

[0041] Figure 5 This is a schematic diagram of the structure of a federated training fine-tuning device based on low-rank approximation quantization in one embodiment of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0043] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0044] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0045] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0046] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0047] Generative language models, with their powerful semantic understanding and generation capabilities, are widely used in machine translation, text generation, dialogue systems, and other fields, greatly promoting innovation in the practical application of artificial intelligence technology. However, as the scale of the models increases, the demands for computing resources and storage also increase dramatically, posing significant challenges to their practical deployment.

[0048] To address the aforementioned challenges, particularly in fields involving sensitive data such as healthcare, finance, and law, the application of generative language models faces the dual pressures of privacy protection and data security. Because data in these fields is typically highly privacy-sensitive, traditional centralized training methods cannot meet privacy requirements and struggle to achieve a balance between efficiency and security in centralized data storage and processing. To efficiently train and deploy generative language models while ensuring privacy, increasing research and practical applications are turning to federated learning frameworks. Federated learning, as an emerging distributed learning method, allows multiple participants to train the model locally and share only the updated parameters, without directly sharing the original data. This approach effectively protects data privacy because the data remains on local devices, preventing the leakage of sensitive data.

[0049] To address the computational burden caused by resource constraints in model training equipment, quantization techniques have emerged. Quantization converts the model's high-precision floating-point weights (e.g., 32-bit floating-point numbers) into low-precision values ​​(e.g., 8 bits or lower), effectively reducing the model's storage requirements and computational overhead. This technique can significantly reduce the transmission cost of large models in federated learning scenarios, decrease the size of parameters transmitted in each round of communication, and improve the inference speed of the model on low-resource devices. However, while quantization can significantly reduce model size and computational burden, it may also introduce some precision loss, especially with extremely low-bit quantization (e.g., 2-bit quantization), where the impact of quantization error on model performance is more significant.

[0050] In addition, many researchers have studied federated learning and the quantization of large language models. One study proposed a deep neural network training method within a federated learning framework, emphasizing local computation updates without centrally storing the dataset, thus effectively addressing privacy concerns and reducing communication overhead. Experimental results on multiple datasets show that using iterative model averaging significantly reduces the number of communication rounds and improves training efficiency. Another study proposed the LoRA (Low-Rank Adapter) method, which reduces the number of parameters during fine-tuning by freezing pre-trained model weights and inserting trainable low-rank decomposition matrices at each layer, thereby improving training efficiency. This method is particularly suitable for large language models that need to run in computationally limited environments. It can significantly reduce computational requirements without compromising model performance. In another study, the QLORA (Quantized Low-Rank Adaptation) method combines 4-bit quantization with LoRA. QLoRA is a technique that further reduces memory usage during fine-tuning, further improving the memory efficiency of fine-tuning large-scale quantized models. This allows for efficient fine-tuning even on resource-constrained GPUs without sacrificing task performance. QLORA achieves performance comparable to 16-bit fine-tuning methods while reducing memory footprint through its innovative 4-bit Normal Float data type and dual quantization technology. Normal Float (NF) is a quantization data type optimized for the characteristics of neural network weight distribution. Essentially, the NF type is a vector composed of four floating-point numbers (such as normalfloat4), specifically designed to store key features of high-dimensional data, such as the weight distribution in neural networks. Each component corresponds to a numerical representation of a different dimension, achieving high-precision storage through floating-point numbers.

[0051] While existing quantization techniques have reduced the computational complexity and storage requirements of generative language models to some extent, the accuracy loss and computational resource consumption caused by quantization remain significant bottlenecks in federated learning environments. Traditional quantization methods lead to a significant decline in model performance at low accuracy, especially in complex tasks, where accuracy loss affects the performance of downstream tasks. Furthermore, most existing quantization methods focus on model compression, neglecting how to efficiently handle multi-device collaboration and resource allocation in distributed environments. Although federated learning can effectively protect data privacy, it often struggles to balance efficiency and privacy protection when computational resources are unevenly distributed. Therefore, existing technical solutions cannot meet the balance requirements of computational resource consumption, communication resource utilization, and privacy protection for generative language models in federated learning scenarios while maintaining high model accuracy.

[0052] Therefore, to address the challenge that existing technologies cannot balance computational resource consumption, communication resource utilization, and privacy protection in federated learning scenarios while maintaining high model accuracy, this application provides a federated training fine-tuning method based on low-rank approximation quantization, a device for executing this method, a physical device, and a computer-readable storage medium. To adapt to the resource-constrained reality of client devices in federated learning, an improved low-rank approximation quantization method is employed to optimize parameter updates and storage efficiency of the generative language model. By embedding a low-rank structure in the model's weight matrix, the computational complexity and storage requirements of the model are significantly reduced while maintaining model performance and stability. This technology enables client devices with limited computational resources to participate in federated learning without sacrificing global model quality, thereby expanding the model's applicability.

[0053] The following examples will provide a detailed description.

[0054] Based on this, embodiments of this application provide a federated training fine-tuning method based on low-rank approximation quantization, which can be implemented by a federated training fine-tuning device based on low-rank approximation quantization. See [link to relevant documentation]. Figure 1 The federated training fine-tuning method based on low-rank approximation quantization specifically includes the following:

[0055] Step 100: Perform multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model in the current iteration training round to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix.

[0056] It is understood that the federated training fine-tuning device based on low-rank approximation quantization can be a functional module set in the participant device of federated learning. The participant device can be implemented as a client device or as a server according to actual application requirements. The participant device locally stores local data for training generative language models.

[0057] In one example, the participating device can be an in-vehicle terminal; the generative language model can be a Chinese voice assistant model deployed on the in-vehicle terminal, and the local data includes at least one privacy-sensitive data item collected from the vehicle terminal, such as user voice input and transcribed text, frequently used navigation destinations, schedules, and music preferences. This type of data includes both structured forms (such as user behavior logs in JSON format) and unstructured text (such as natural language command text). In the above typical application scenario, federated learning allows each vehicle to independently train its model using local data without uploading the original data, thereby avoiding data privacy leaks. Furthermore, combined with the low-rank approximation quantization technique of this application, high-precision model parameters can be compressed and their rank reduced under the premise of limited terminal device resources, enabling efficient local deployment and real-time inference of the model. Taking in-vehicle voice assistants as an example, the trained generative language model can perform natural language understanding and personalized command generation tasks online based on local data such as user voice recordings, navigation preferences, and schedules. This includes tasks such as automatically completing navigation destinations, optimizing music playback preference matching, or triggering voice reminders based on schedules. This not only improves response speed and interactive experience but also avoids the leakage of user data uploads.

[0058] In one or more embodiments of this application, the initial local generative language model for the current training iteration is generated by the participating device based on the global generative language model parameters for the current training iteration sent by the aggregating device and local data. The original weight matrix may be included in the global generative language model parameters sent by the aggregating device and sent to the participating device. It is understood that the aggregating device may be implemented as a server or, depending on the actual application requirements, as a client device, etc.

[0059] Step 200: Fine-tune the target low-rank approximation matrix using local data to obtain the trained low-rank approximation matrix corresponding to the current iteration training round. Use the target integer weight matrix and the trained low-rank approximation matrix as the local quantized generative language model parameters corresponding to the current iteration training round and send them to the aggregation end device in the federated learning system where it is located. This allows the aggregation end device to aggregate the received local quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round.

[0060] In step 200, the aggregation device receives the locally quantized generative language model parameters corresponding to the current iteration training round, which are sent by multiple participating devices respectively. Then, it aggregates the locally quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round. If the current iteration training round is not the preset last iteration training round or if the global generative language model parameters have not converged, the global generative language model parameters corresponding to the current iteration training round are used as the initial global generative language model parameters for the next iteration training round and broadcast to each participating device in the federated learning system.

[0061] As described above, the core idea of ​​the federated training fine-tuning method based on low-rank approximation quantization provided in this application is to utilize low-rank approximation and quantization techniques during the federated learning process. This involves alternating between quantization and low-rank approximation to approximate the original pre-trained weights, thereby reducing communication overhead, lowering computational resource consumption, and improving model training performance. Training generative language models within the federated learning framework ensures data privacy protection, guaranteeing data security during training and preventing the leakage of sensitive user information. Simultaneously, combining low-rank approximation techniques with quantization of the generative language model and optimizing quantization strategies to reduce quantization errors significantly improves model performance under low-precision quantization. Alternating optimization methods, such as alternating quantization and singular value decomposition, effectively reduce quantization bias and improve the initialization quality of the quantized model, thus enhancing fine-tuning performance. Subsequently, combining low-rank adaptation fine-tuning techniques after quantization avoids the computational burden of numerous parameter updates in traditional fine-tuning methods, optimizes the utilization efficiency of participating devices' computational resources, and reduces device resource occupancy. Furthermore, it can further improve computational accuracy by reducing the consumption of computing resources and communication resources of participating devices and achieving data privacy protection, thereby significantly improving the generalization ability and adaptability of downstream tasks and ensuring efficient operation in federated learning scenarios.

[0062] To further improve the effectiveness and reliability of federated training fine-tuning based on low-rank approximation quantization, this application provides a federated training fine-tuning method based on low-rank approximation quantization, see [link to relevant documentation]. Figure 2 The federated training fine-tuning method based on low-rank approximation quantization, prior to step 100, specifically includes the following:

[0063] Step 010: Receive the initial global generative language model parameters corresponding to the current iteration training round sent by the aggregation terminal device in the federated learning system to which it is located; wherein, if the current iteration training round is the preset first iteration training round, the initial global generative language model parameters are the generative language model parameters pre-acquired by the aggregation terminal device; if the current iteration training round is not the first iteration training round, the initial global generative language model parameters are the global generative language model parameters corresponding to the previous iteration training round pre-aggregated by the aggregation terminal device.

[0064] Step 020: Initialize the initial global generative language model parameters based on local data to obtain the initial local generative language model parameters for the current iteration training round, wherein the initial local generative language model parameters include the original weight matrix corresponding to the initial local generative language model for the current iteration training round.

[0065] In each round of federated learning, at the start of each training iteration, the server first broadcasts the current global model (i.e., the initial global generative language model parameters) to all available clients. This global model is initialized based on a previously trained or pre-trained model and represents a globally shared set of model parameters. At this stage, the global model broadcast by the server contains high-precision weights from the previous training iteration or pre-training, serving as the starting point for this round of training. Upon receiving the global model, each client first initializes it to obtain an initial local model (i.e., initial local generative language model parameters) to prepare for subsequent local training steps.

[0066] To further reduce quantization bias and improve the initialization quality of the quantized model, thereby enhancing fine-tuning performance, a federated training fine-tuning method based on low-rank approximate quantization is provided in this application embodiment. (See also...) Figure 2 Step 100 of the federated training fine-tuning method based on low-rank approximation quantization specifically includes the following:

[0067] Step 110: Based on the preset initial rank and initial quantization bit width, perform initial low-rank approximation and quantization processing on the original weight matrix corresponding to the initial local generative language model of the current iteration training round to obtain the initial integer weight matrix and the initial low-rank approximation matrix corresponding to the original weight matrix.

[0068] Step 120: Based on the initial low-rank approximation matrix, perform multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix.

[0069] Here, the rank is r, and the initial rank can be denoted as r0. The quantization bit width can be denoted as N, which is also the quantization precision, representing the number of bits used in the weight matrix. The initial quantization bit width can be denoted as N0.

[0070] Building upon this, and to further reduce quantization errors through optimized quantization strategies, thereby significantly improving model performance under low-precision quantization, a federated training fine-tuning method based on low-rank approximate quantization is provided in this application embodiment. (See [link to relevant documentation]). Figure 3 Step 110 of the federated training fine-tuning method based on low-rank approximation quantization specifically includes the following:

[0071] Step 111: Based on the preset initial rank, perform singular value decomposition on the original weight matrix of the current iteration training round to obtain the initial low-rank approximation matrix corresponding to the original weight matrix, which is composed of the left singular matrix and the right singular matrix.

[0072] Specifically, based on r0, in the low-rank approximation process, this application employs low-rank matrix factorization techniques to approximate the pre-trained high-precision weight matrix (i.e., the original weight matrix). Decompose it, where... Let d1 represent the real number field; d2 represent the input feature dimension; and d3 represent the output feature dimension. This is achieved by decomposing the original weight matrix W into two lower-rank matrices, specifically the left singular matrix within the initial lower-rank approximation matrix. Right singular matrix in the initial low-rank approximation matrix It can successfully reduce the model's storage requirements and computational complexity.

[0073] Step 112: Set the left singular matrix and the right singular matrix in the initial low-rank approximation matrix to 0.

[0074] The left singular matrix that constitutes the initial low-rank approximation matrix can be denoted as A0, and the right singular matrix that constitutes the initial low-rank approximation matrix can be denoted as B0. A0 and B0 are set to 0 to ensure that neither quantization nor low-rank approximation introduces additional bias in the initial state.

[0075] Step 113: Based on the preset initial quantization bit width, map the initial integer weight matrix to the integer space to obtain the initial integer weight matrix corresponding to the original weight matrix.

[0076] Specifically, quantization techniques are used to process A0 and B0, which constitute the initial low-rank approximation matrix, compressing the original high-precision floating-point parameters into lower-precision integer values. Then, N-bit quantization maps the weight matrix to integer space, generating a quantized integer weight matrix. In this initialization phase, the initial integer weight matrix Q1 = q corresponding to the original weight matrix is ​​obtained. N (W-0·0T ), where q N (·) is a function that maps high-precision weights to an N-bit quantized matrix; N is used to transform the original weight matrix W, which is a floating-point matrix, into an integer weight matrix Q.

[0077] Building upon this, to further reduce quantization bias and improve the initialization quality of the quantized model, thereby enhancing fine-tuning performance, a federated training fine-tuning method based on low-rank approximate quantization is provided in this application embodiment. (See also...) Figure 3 Step 120 of the federated training fine-tuning method based on low-rank approximation quantization specifically includes the following:

[0078] Step 121: Determine the quantization residual matrix corresponding to the first round of alternating optimization process based on the original weight matrix and the initial integer weight matrix, and perform singular value decomposition on the quantization residual matrix to obtain the low-rank approximate matrix composed of the left singular matrix and the right singular matrix corresponding to the first round of alternating optimization process.

[0079] Specifically, the round number of the alternating optimization process is represented by t. After initialization in step 110 (i.e., t = 0), in the next round of alternating optimization (i.e., t = 1), the quantization residual matrix R1 = W - Q1 corresponding to the first round of alternating optimization is determined based on the original weight matrix W and the initial integer weight matrix Q1. Then, singular value decomposition is performed on the quantization residual matrix R1 corresponding to the first round of alternating optimization to obtain the low-rank approximation matrix corresponding to the first round of alternating optimization. The low-rank approximation matrix corresponding to this round of alternating optimization process The transpose of the left singular matrix A1 and the right singular matrix B1 corresponding to the first round of alternating optimization process constitute.

[0080] In other words, when t=1, Q1 is the quantized weight obtained by the QLoRA method; A1 and B1 obtained by SVD operation on the quantized residual W-Q1 can effectively alleviate the quantization bias and make the model more adaptable to downstream tasks. If the multi-round alternating optimization in step 122 is further performed, that is, the non-first round alternating optimization process corresponding to t>1, the original weight matrix W of high-precision pre-training can be gradually approximated.

[0081] Step 122: Perform at least one alternating optimization step based on the low-rank approximation matrix corresponding to the alternating optimization process, wherein the alternating optimization step includes: in the current non-first round alternating optimization process, determining the integer weight matrix corresponding to the current non-first round alternating optimization process based on the original weight matrix and the low-rank approximation matrix of the previous round; determining the quantization residual matrix corresponding to the current non-first round alternating optimization process based on the original weight matrix and the integer weight matrix corresponding to the current non-first round alternating optimization process, and performing singular value decomposition on the quantization residual matrix to obtain the low-rank approximation matrix corresponding to the current non-first round alternating optimization process; and jointly optimizing the integer weight matrix and the low-rank approximation matrix corresponding to the current non-first round alternating optimization process based on a preset objective function to obtain the optimized integer weight matrix and the optimized low-rank approximation matrix corresponding to the current non-first round alternating optimization process.

[0082] Specifically, when t > 1, at step t, it is necessary to first quantize the original weight matrix W with the low-rank approximation matrix from the previous round. The difference between them yields a new integer weight matrix Q. t t is a positive integer; quantization operations can...

[0083] Represented as:

[0084]

[0085] Where, q N (·) is a function that maps high-precision weights to an N-bit quantization matrix. In this way, this application can progressively project the original high-precision weights W into a finite quantization space, thereby reducing model storage and transmission overhead. After obtaining the integer weight matrix Q... t Then, the quantization residual matrix R is calculated. t =WQ t The singular value decomposition is performed on the matrix to obtain the low-rank approximation matrix corresponding to the current non-first round alternating optimization process. Based on the preset objective function, the integer weight matrix and the low-rank approximation matrix corresponding to the current non-first round alternating optimization process are jointly optimized to obtain the optimized integer weight matrix and the optimized low-rank approximation matrix corresponding to the current non-first round alternating optimization process.

[0086] Step 123: Use the optimized integer weight matrix obtained from the last executed alternating optimization step as the target integer weight matrix corresponding to the original weight matrix, and use the optimized low-rank approximation matrix obtained from the last executed alternating optimization step as the target low-rank approximation matrix corresponding to the original weight matrix.

[0087] Based on this, in order to further optimize the low-rank approximation and quantization processes simultaneously and thus improve fine-tuning performance, in a federated training fine-tuning method based on low-rank approximation quantization provided in this application embodiment, the objective function is used to jointly optimize the target integer weight matrix and the target low-rank approximation matrix by minimizing the Frobinius norm between the original weight matrix, the target integer weight matrix and the target low-rank approximation matrix.

[0088] The objective function takes the following form:

[0089]

[0090] ||·|| F Let Frobinius norm represent the degree of difference between matrices. This is achieved by jointly optimizing the integer weight matrix Q and the low-rank approximation matrix AB. T Here, A represents the left singular matrix, and B represents the right singular matrix. This objective function minimizes the error caused by quantization while maintaining high accuracy. In one or more embodiments of this application, the integer weight matrix can also be referred to as the quantization matrix.

[0091] To further avoid the computational burden caused by numerous parameter updates in traditional fine-tuning methods, optimize the utilization efficiency of the computing resources of participating devices, and reduce device resource occupancy, this application provides a federated training fine-tuning method based on low-rank approximate quantization, see [link to relevant documentation]. Figure 2 or Figure 3 Step 200 of the federated training fine-tuning method based on low-rank approximation quantization specifically includes the following:

[0092] Step 210: Store the target integer weight matrix based on a preset lookup table, so as to map the target integer weight matrix back to the simulated high-precision floating point value.

[0093] Step 220: Fix the target integer weight matrix and use local data to perform gradient update on the target low-rank approximation matrix to obtain the trained low-rank approximation matrix corresponding to the current iteration training round.

[0094] Specifically, this application uses a combination of quantized integer weight matrix and lookup table to perform dequantization, thereby effectively reducing the accuracy loss caused by quantization and significantly reducing computational and storage overhead.

[0095] Step 230: The target integer weight matrix and the trained low-rank approximation matrix are used as the local quantized generative language model parameters corresponding to the current iteration training round and sent to the aggregation device in the federated learning system, so that the aggregation device can aggregate the received local quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round.

[0096] To further illustrate the above embodiments, this application also provides a specific application example of a federated training fine-tuning method based on low-rank approximation quantization, see [link to relevant documentation]. Figure 4 In this application example, the core idea of ​​federated learning, using a server as the aggregator and client devices (which can be abbreviated as clients) as participants, is to leverage low-rank approximation and quantization techniques. During the federated learning process, low-rank approximation and quantization are applied to the client's local model; that is, quantization and low-rank approximation are alternately applied to approximate the original pre-trained weights. This reduces communication overhead, lowers computational resource consumption, and improves model training performance. The overall process includes the following five key steps to achieve quantization fine-tuning of a large language model within the federated learning framework. Specifically, the application example of the federated training fine-tuning method based on low-rank approximation quantization includes the following:

[0097] (I) Initialization Phase

[0098] At the start of each training round of federated learning, the server first broadcasts the current global model θ to all available clients. t (i.e., the initial global generative language model parameters). This global model θ t This initialization is based on previously trained or pre-trained models, representing a globally shared set of model parameters. At this stage, the global model broadcast by the server contains high-precision weights from the previous training round or pre-training, serving as the starting point for this round of training. The global model broadcast operation can be represented as follows:

[0099] θ t ←Server Broadcast(θ t )

[0100] Each client receives the global model θ t Next, it is initialized to prepare for subsequent local training steps. This is done on the client's local dataset D. k Above, the client performs local initialization operations to ensure that its local model is updated based on the global model, thus unifying the model parameters. This includes the initial local model. The initial local generative language model parameters are obtained as follows:

[0101]

[0102] (II) Low-rank approximation stage

[0103] During local training on the client side, in order to effectively reduce the storage and transmission overhead of the model, the model weight matrix is ​​decomposed into a low-rank matrix through low-rank approximation, thereby reducing the storage requirements of the model parameters; at the same time, quantization technology is used to compress high-precision floating-point parameters into lower-precision integer values, further reducing the transmission overhead of the model.

[0104] Specifically, in the low-rank approximation process, this application example uses low-rank matrix factorization to approximate the pre-trained high-precision weight matrix (i.e., the original weight matrix). Decompose the original weight matrix W into two lower-rank matrices. and During the initialization process, the left singular matrix in the initial low-rank approximation matrix is ​​obtained. Right singular matrix in the initial low-rank approximation matrix This successfully reduces the model's storage requirements and computational complexity. The specific low-rank approximation operation is implemented as follows:

[0105] W≈AB T

[0106] In this process, matrices A and B represent the left and right singular matrices of the weight matrix W, respectively. Through the low-rank approximation, the model's storage requirements are significantly reduced, while computational overhead is lowered while maintaining high accuracy.

[0107] Next, quantization techniques are used to process the weight matrices A and B after the low-rank approximation, compressing the original high-precision floating-point parameters into lower-precision integer values. At this point, N-bit quantization maps the weight matrices to integer space, generating quantized integer weight matrices. Where N is the quantization precision, representing the number of bits used in the weight matrix. It transforms the floating-point weight matrix W into an integer weight matrix Q. To simultaneously optimize the low-rank approximation and quantization process, this application proposes a joint optimization objective function, which approximates the weights by minimizing the Frobenius norm between the original weight matrix and the quantized low-rank approximation matrix. The specific objective function is as follows:

[0108]

[0109] Among them ||·|| FLet represent the Frobinius norm, and let represent the degree of difference between the matrices. By jointly optimizing the quantization matrix Q and the low-rank approximation matrices A and B, this objective function can minimize the quantization error while maintaining high accuracy.

[0110] (III) Alternating Optimization Phase

[0111] The method iteratively reduces the quantization bias and approximates the original high-precision pre-trained weights by first quantizing the weight matrix and then performing singular value decomposition (SVD) on the quantization residuals. Specifically, during initialization, the low-rank matrices A0 and B0 are set to zero to ensure that neither quantization nor low-rank approximation introduces additional bias in the initial state. Then, at step t, the original weight matrix W needs to be quantized and approximated with the previous low-rank approximation. The difference between them yields a new quantization matrix Q. t t is a positive integer; the quantization operation can be represented as:

[0112]

[0113] Where q N (·) is a function that maps high-precision weights to an N-bit quantization matrix. In this way, the application examples of this application can progressively project the original high-precision weights W onto a finite quantization space, thereby reducing model storage and transmission overhead. After obtaining the quantized weights Q... t Then, the quantization residual R was calculated. t =WQ t And perform singular value decomposition on it:

[0114]

[0115] Where d = min{d1, d2}, σ t,1 ≥σ t,2 ≥…≥σ t,d The residual matrix R t singular value u t,i and v {t,i} These are the corresponding left and right singular vectors, respectively. Then, an approximation of rank r is obtained using the following formula.

[0116]

[0117]

[0118] We obtain the updated quantization matrix Q at step t. t and low-rank approximation matrix When t=1, Q1 is the quantized weight obtained by the QLoRA method. A1 and B1, obtained by SVD operation on the quantized residual W-Q1, can effectively alleviate quantization bias and make the model more adaptable to downstream tasks. If multiple rounds of alternating optimization are performed for t>1, the original high-precision pre-trained weights W can be gradually approximated.

[0119] (iv) Optimization and Fine-tuning Phase

[0120] The application example in this application combines the quantized integer weight matrix with a lookup table. Dequantization is performed through the lookup table, which effectively reduces the accuracy loss caused by quantization and significantly reduces computational and storage overhead.

[0121] Specifically, the target integer weight matrix obtained in the last round (i.e., round T) after quantization. The weights will be stored in a lookup table Y and dequantized as needed during forward propagation. This process ensures that the model maintains high performance during training and inference even after the weights are quantized. The quantized integer weights M = q N (W) is stored in a lookup table Y[i], where q N (W) represents the result of N-bit quantization of the original high-precision weight matrix W. The lookup table is constructed as follows:

[0122] M = q N (W)

[0123] Furthermore, the lookup table Y[i] is used to map the quantized integer values ​​back to the simulated high-precision floating-point values:

[0124]

[0125] Where F -1 (·) is the inverse function of the quantization mapping, which converts the integer value i in the lookup table into the corresponding simulated high-precision weight. This lookup table ensures that during forward propagation, the integer weights can be effectively dequantized to values ​​close to the original high-precision weights, thereby controlling the precision loss caused by quantization. During backpropagation, only A and B in the target low-rank approximation matrix are updated and optimized using gradients. During fine-tuning, the target integer weight matrix Q... T By maintaining the frozen state, gradient updates during training only apply to the target low-rank approximation matrix, excluding the quantized target integer weight matrix. Most of the computation during training is concentrated in the low-rank adapter part, reducing computational overhead and storage requirements, thus significantly lowering training costs.

[0126] (V) Global Model Aggregation Phase

[0127] After each round of training, all clients k∈S tIt will use the local large language model Uploaded to the server, where Let θ represent the parameters of the large language model after local quantization by client k during the t-th round of training, following multiple rounds of local updates. After receiving the local models from all clients, the server aggregates them and updates the global large language model θ. (t+1) This serves as the initial model for the next round of training. The formula for the aggregation operation is as follows:

[0128]

[0129] Where S t This represents the set of clients participating in the training in the t-th round. The weighting coefficient for each client is defined as the ratio of the dataset size of client k to the total dataset size of all clients, i.e.:

[0130]

[0131] Where |D k | represents the number of samples in the local dataset of client k, ∑ i∈St |D i | indicates the total amount of data from all participating clients.

[0132] Therefore, to address the issues of computational efficiency, storage requirements, and privacy protection for generative language models in federated learning scenarios, this application example trains the generative language model within a federated learning framework while ensuring data privacy. This guarantees data security during training and prevents the leakage of sensitive user information. Simultaneously, low-rank approximation techniques are used to quantize the generative language model, and optimized quantization strategies reduce quantization errors, significantly improving model performance under low-precision quantization. Alternating optimization methods, alternating between quantization and singular value decomposition, effectively reduce quantization bias and improve the initialization quality of the quantized model, thereby enhancing fine-tuning performance. Subsequently, low-rank adaptation fine-tuning is performed after quantization, avoiding the computational burden of numerous parameter updates in traditional fine-tuning methods, optimizing the utilization efficiency of participating devices' computing resources, and reducing device resource occupancy. Furthermore, by reducing the computational and communication resource consumption of participating devices and achieving data privacy protection, computational accuracy is further improved, significantly enhancing the generalization ability and adaptability of downstream tasks and ensuring efficient operation in federated learning scenarios.

[0133] In other words, addressing the challenges of low computational efficiency, high storage requirements, and insufficient privacy protection faced by current generative language models in federated learning scenarios, this application proposes a federated training fine-tuning method based on low-rank approximation quantization, which significantly improves the training efficiency and resource adaptability of generative language models. While ensuring data privacy, this application effectively reduces the computational complexity and storage requirements of the model through low-rank approximation quantization, optimizing the performance on hardware-constrained devices. Through low-rank adaptation fine-tuning, this application not only reduces the computational burden during fine-tuning but also enhances the model's adaptability to specific tasks. In the distributed environment of federated learning, this application improves the utilization of computational resources, reduces training time, and ensures that generative language models can be efficiently and securely fine-tuned and inferred in privacy-sensitive applications.

[0134] From a software perspective, this application also provides a device for performing all or part of the federated training fine-tuning based on low-rank approximation quantization in the aforementioned federated training fine-tuning method based on low-rank approximation quantization, see [link to relevant documentation]. Figure 5 The federated training fine-tuning device based on low-rank approximation quantization specifically includes the following:

[0135] The alternating optimization module 10 is used to perform multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model in the current iteration training round, so as to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix.

[0136] The optimization and fine-tuning module 20 is used to fine-tune the target low-rank approximation matrix using local data to obtain the trained low-rank approximation matrix corresponding to the current iteration training round. The target integer weight matrix and the trained low-rank approximation matrix are used as the local quantized generative language model parameters corresponding to the current iteration training round and sent to the aggregation end device in the federated learning system where it is located. This allows the aggregation end device to aggregate the received local quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round.

[0137] The embodiments of the federated training fine-tuning device based on low-rank approximation quantization provided in this application can be used to execute the processing flow of the federated training fine-tuning method based on low-rank approximation quantization in the above embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the federated training fine-tuning method embodiments based on low-rank approximation quantization described above.

[0138] The low-rank approximation quantization-based federated training fine-tuning device can perform the low-rank approximation quantization-based federated training fine-tuning portion within the client device. The specific implementation can be chosen based on the processing capabilities of the client device and limitations imposed by the user's usage scenario. This application does not impose any limitations in this regard. If all operations are performed within the client device, the client device may further include a processor for the specific processing of the low-rank approximation quantization-based federated training fine-tuning.

[0139] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0140] The server and the client device can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Such network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Furthermore, such network protocols may also include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer Protocol) protocols used on top of the aforementioned protocols.

[0141] As described above, the federated training fine-tuning device based on low-rank approximation quantization provided in this application performs low-rank adapter fine-tuning after quantization. This avoids the computational burden caused by numerous parameter updates in traditional fine-tuning methods, optimizes the utilization efficiency of the computing resources of participating devices, and reduces the resource occupancy rate of the devices. Furthermore, it can further improve computational accuracy while reducing the computing resource consumption and communication resource occupancy rate of participating devices and achieving data privacy protection. This significantly enhances the generalization ability and adaptability of downstream tasks, ensuring efficient operation in federated learning scenarios.

[0142] This application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the federated training fine-tuning method based on low-rank approximation quantization mentioned in the above embodiments. The processor and memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and memory via wired or wireless means.

[0143] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0144] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the federated training fine-tuning method based on low-rank approximation quantization in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the federated training fine-tuning method based on low-rank approximation quantization in the above method embodiments.

[0145] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0146] The one or more modules are stored in the memory, and when executed by the processor, the federated training fine-tuning method based on low-rank approximation quantization in the implementation embodiment is executed.

[0147] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0148] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0149] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0150] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned federated training fine-tuning method based on low-rank approximation quantization. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0151] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned federated training fine-tuning method based on low-rank approximation quantization.

[0152] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0153] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0154] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0155] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A federated training fine-tuning method based on low-rank approximate quantization, characterized in that, include: The original weight matrix corresponding to the initial local generative language model in the current iteration training round is subjected to multiple rounds of alternating optimization based on quantization and low-rank approximation to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix. The target low-rank approximation matrix is ​​fine-tuned using local data to obtain the trained low-rank approximation matrix corresponding to the current iteration training round. The target integer weight matrix and the trained low-rank approximation matrix are used as the local quantized generative language model parameters corresponding to the current iteration training round and sent to the aggregation end device in the federated learning system where it is located. The aggregation end device then aggregates the received local quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round. Before performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model of the current iteration training round, the method further includes: The system receives the initial global generative language model parameters corresponding to the current iteration training round from the aggregation terminal device in its own federated learning system. If the current iteration training round is the preset first iteration training round, the initial global generative language model parameters are generative language model parameters pre-acquired by the aggregation terminal device. If the current iteration training round is not the first iteration training round, the initial global generative language model parameters are the global generative language model parameters corresponding to the previous iteration training round, pre-aggregated by the aggregation terminal device. The initial global generative language model parameters are initialized based on local data to obtain the initial local generative language model parameters for the current iteration training round. The initial local generative language model parameters include the original weight matrix corresponding to the initial local generative language model for the current iteration training round. The step of performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model in the current iteration training round to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix includes: Based on the preset initial rank and initial quantization bit width, the original weight matrix corresponding to the initial local generative language model in the current iteration training round is initialized with low-rank approximation and quantization to obtain the initial integer weight matrix and the initial low-rank approximation matrix corresponding to the original weight matrix. Based on the initial low-rank approximation matrix, the original weight matrix is ​​subjected to multiple rounds of alternating optimization based on quantization and low-rank approximation to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix. The initial low-rank approximation and quantization processing of the original weight matrix corresponding to the initial local generative language model in the current iteration training round, based on a preset initial rank and initial quantization bit width, are performed to obtain the initial integer weight matrix and the initial low-rank approximation matrix corresponding to the original weight matrix, including: Based on the preset initial rank, singular value decomposition is performed on the original weight matrix of the current iteration training round to obtain the initial low-rank approximate matrix corresponding to the original weight matrix, which is composed of the left singular matrix and the right singular matrix. Set the left singular matrix and the right singular matrix in the initial low-rank approximation matrix to 0; Based on the preset initial quantization bit width, the initial integer weight matrix is ​​mapped to the integer space to obtain the initial integer weight matrix corresponding to the original weight matrix; The step of performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix according to the initial low-rank approximation matrix to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix includes: The quantization residual matrix corresponding to the first round of alternating optimization process is determined based on the original weight matrix and the initial integer weight matrix, and the singular value decomposition is performed on the quantization residual matrix to obtain the low-rank approximate matrix composed of the left singular matrix and the right singular matrix corresponding to the first round of alternating optimization process. Based on the low-rank approximation matrix corresponding to the alternating optimization process, at least one alternating optimization step is performed, wherein the alternating optimization step includes: in the current non-first round alternating optimization process, determining the integer weight matrix corresponding to the current non-first round alternating optimization process according to the original weight matrix and the low-rank approximation matrix of the previous round; determining the quantization residual matrix corresponding to the current non-first round alternating optimization process according to the original weight matrix and the integer weight matrix corresponding to the current non-first round alternating optimization process, and performing singular value decomposition on the quantization residual matrix to obtain the low-rank approximation matrix corresponding to the current non-first round alternating optimization process; and jointly optimizing the integer weight matrix and the low-rank approximation matrix corresponding to the current non-first round alternating optimization process based on a preset objective function to obtain the optimized integer weight matrix and the optimized low-rank approximation matrix corresponding to the current non-first round alternating optimization process. The optimized integer weight matrix obtained from the last executed alternating optimization step is used as the target integer weight matrix corresponding to the original weight matrix, and the optimized low-rank approximation matrix obtained from the last executed alternating optimization step is used as the target low-rank approximation matrix corresponding to the original weight matrix. The objective function is used to jointly optimize the target integer weight matrix and the target low-rank approximation matrix by minimizing the Frobinius norm between the original weight matrix, the target integer weight matrix, and the target low-rank approximation matrix; The step of fine-tuning the target low-rank approximation matrix using local data to obtain the trained low-rank approximation matrix corresponding to the current training iteration includes: The target integer weight matrix is ​​stored based on a preset lookup table to map the target integer weight matrix back to simulated high-precision floating point values; The target integer weight matrix is ​​fixed, and the target low-rank approximation matrix is ​​updated with gradients using local data to obtain the trained low-rank approximation matrix corresponding to the current training iteration. The aggregation device aggregates the local quantized generative language model parameters corresponding to the current iteration training round, including: weighting the local quantized generative language model parameters according to the proportion of the sample size of the local dataset corresponding to each client to the total sample size of the dataset of all clients.

2. A federated training fine-tuning device based on low-rank approximate quantization, characterized in that, include: The alternating optimization module is used to perform multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model in the current iteration training round, so as to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix. The optimization and fine-tuning module is used to fine-tune the target low-rank approximation matrix using local data to obtain the trained low-rank approximation matrix corresponding to the current iteration training round. The target integer weight matrix and the trained low-rank approximation matrix are used as the local quantized generative language model parameters corresponding to the current iteration training round and sent to the aggregation end device in the federated learning system where it is located. The aggregation end device then aggregates the received local quantized generative language model parameters corresponding to the current iteration training round to obtain the global generative language model parameters corresponding to the current iteration training round. Before performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model of the current iteration training round, the device also performs: The system receives the initial global generative language model parameters corresponding to the current iteration training round from the aggregation terminal device in its own federated learning system. If the current iteration training round is the preset first iteration training round, the initial global generative language model parameters are generative language model parameters pre-acquired by the aggregation terminal device. If the current iteration training round is not the first iteration training round, the initial global generative language model parameters are the global generative language model parameters corresponding to the previous iteration training round, pre-aggregated by the aggregation terminal device. The initial global generative language model parameters are initialized based on local data to obtain the initial local generative language model parameters for the current iteration training round. The initial local generative language model parameters include the original weight matrix corresponding to the initial local generative language model for the current iteration training round. The step of performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix corresponding to the initial local generative language model in the current iteration training round to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix includes: Based on the preset initial rank and initial quantization bit width, the original weight matrix corresponding to the initial local generative language model in the current iteration training round is initialized with low-rank approximation and quantization to obtain the initial integer weight matrix and the initial low-rank approximation matrix corresponding to the original weight matrix. Based on the initial low-rank approximation matrix, the original weight matrix is ​​subjected to multiple rounds of alternating optimization based on quantization and low-rank approximation to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix. The initial low-rank approximation and quantization processing of the original weight matrix corresponding to the initial local generative language model in the current iteration training round, based on a preset initial rank and initial quantization bit width, are performed to obtain the initial integer weight matrix and the initial low-rank approximation matrix corresponding to the original weight matrix, including: Based on the preset initial rank, singular value decomposition is performed on the original weight matrix of the current iteration training round to obtain the initial low-rank approximate matrix corresponding to the original weight matrix, which is composed of the left singular matrix and the right singular matrix. Set the left singular matrix and the right singular matrix in the initial low-rank approximation matrix to 0; Based on the preset initial quantization bit width, the initial integer weight matrix is ​​mapped to the integer space to obtain the initial integer weight matrix corresponding to the original weight matrix; The step of performing multiple rounds of alternating optimization based on quantization and low-rank approximation on the original weight matrix according to the initial low-rank approximation matrix to obtain the target integer weight matrix and the target low-rank approximation matrix corresponding to the original weight matrix includes: The quantization residual matrix corresponding to the first round of alternating optimization process is determined based on the original weight matrix and the initial integer weight matrix, and the singular value decomposition is performed on the quantization residual matrix to obtain the low-rank approximate matrix composed of the left singular matrix and the right singular matrix corresponding to the first round of alternating optimization process. Based on the low-rank approximation matrix corresponding to the alternating optimization process, at least one alternating optimization step is performed, wherein the alternating optimization step includes: in the current non-first round alternating optimization process, determining the integer weight matrix corresponding to the current non-first round alternating optimization process according to the original weight matrix and the low-rank approximation matrix of the previous round; determining the quantization residual matrix corresponding to the current non-first round alternating optimization process according to the original weight matrix and the integer weight matrix corresponding to the current non-first round alternating optimization process, and performing singular value decomposition on the quantization residual matrix to obtain the low-rank approximation matrix corresponding to the current non-first round alternating optimization process; and jointly optimizing the integer weight matrix and the low-rank approximation matrix corresponding to the current non-first round alternating optimization process based on a preset objective function to obtain the optimized integer weight matrix and the optimized low-rank approximation matrix corresponding to the current non-first round alternating optimization process. The optimized integer weight matrix obtained from the last executed alternating optimization step is used as the target integer weight matrix corresponding to the original weight matrix, and the optimized low-rank approximation matrix obtained from the last executed alternating optimization step is used as the target low-rank approximation matrix corresponding to the original weight matrix. The objective function is used to jointly optimize the target integer weight matrix and the target low-rank approximation matrix by minimizing the Frobinius norm between the original weight matrix, the target integer weight matrix, and the target low-rank approximation matrix; The step of fine-tuning the target low-rank approximation matrix using local data to obtain the trained low-rank approximation matrix corresponding to the current training iteration includes: The target integer weight matrix is ​​stored based on a preset lookup table to map the target integer weight matrix back to simulated high-precision floating point values; The target integer weight matrix is ​​fixed, and the target low-rank approximation matrix is ​​updated with gradients using local data to obtain the trained low-rank approximation matrix corresponding to the current training iteration. The aggregation device aggregates the local quantized generative language model parameters corresponding to the current iteration training round, including: weighting the local quantized generative language model parameters according to the proportion of the sample size of the local dataset corresponding to each client to the total sample size of the dataset of all clients.

3. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the federated training fine-tuning method based on low-rank approximate quantization as described in claim 1.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the federated training fine-tuning method based on low-rank approximation quantization as described in claim 1.

Images