Distributed large language model quantization deployment method, device, equipment and storage medium
By quantizing and low-rank decomposition of the initial weights of large language models, the problems of insufficient computing power of client devices and large overhead of model parameter transmission are solved, and efficient federated learning training is achieved.
Patent Information
- Application Number
- CN202411204590.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-30
AI Technical Summary
In federated learning, client devices have high computing power requirements when deploying large language models, and model parameter transmission leads to high communication overhead and energy consumption, making it difficult to meet real-time performance requirements.
By quantifying the initial weights of the large language model, low-rank decomposition is used to reduce the number of parameters during the training process, and only some parameters are transmitted during the server aggregation process, reducing communication overhead.
It reduces the computational overhead and data transmission of client training, improves the overall training efficiency of federated learning, reduces energy consumption, and meets real-time performance requirements.
Smart Images

Figure CN118735001B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a distributed large language model quantitative deployment method, apparatus, device and storage medium. Background Art
[0002] Federated learning is a distributed machine learning method. In the scenario of federated learning, each client device has local data for local model training. The client uploads the parameter updates of the local model to the server, and the server aggregates these updated parameters to generate global model updates. The aggregated global model updates are sent back to the client, and the client performs the next round of local training based on the new global model. This iterative process continues until the global model converges or reaches the predetermined performance indicators.
[0003] In the related art, if a large language model needs to be deployed on different clients through a federated learning mechanism, the following problems are usually encountered: a large language model usually has hundreds of millions of parameters, and when training on the client side, the computing power of the client device is required to be high. In addition, when the client sends parameters to the server, the amount of data transmitted is too large, resulting in high communication overhead and high energy consumption. Summary of the invention
[0004] The main purpose of the embodiments of the present application is to propose a distributed large language model quantization deployment method, device, equipment and storage medium, reduce the computing power overhead of training large language models in the client, reduce the amount of data transmitted from the client to the server, reduce communication overhead, and reduce transmission energy consumption.
[0005] To achieve the above objectives, a first aspect of an embodiment of the present application proposes a distributed large language model quantization deployment method, including:
[0006] Quantize the initial weights of the large language model deployed in the client to obtain quantized weights, and obtain a first low-rank matrix and a second low-rank matrix related to weight update of the quantized weights and obtained through low-rank decomposition;
[0007] Based on the first low-rank matrix, the second low-rank matrix and the quantized weights, a first weight matrix is obtained, a second weight matrix is obtained by quantizing the first weight matrix, and an optimization target is obtained according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix;
[0008] Solving the optimization objective to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization objective;
[0009] After the server aggregates the first solution matrix and the second solution matrix of each of the clients, the updated first low-rank matrix and the second low-rank matrix are obtained, the optimization target is updated according to the updated first low-rank matrix and the second low-rank matrix, the first solution matrix and the second solution matrix are iteratively updated until the iteration termination condition is reached, the trained large language model is obtained according to the first weight matrix, and the trained large language model is updated in each of the clients.
[0010] In some embodiments, obtaining a first low-rank matrix and a second low-rank matrix related to the weight update of the quantization weight and obtained through low-rank decomposition includes:
[0011] Acquire a local data set corresponding to the client, and use the local data set to train the quantized large language model to obtain an update weight corresponding to the quantized weight;
[0012] Perform low-rank decomposition on the updated weights to obtain the first low-rank matrix and the second low-rank matrix, where the ranks of the first low-rank matrix and the second low-rank matrix are smaller than the rank of the updated weights.
[0013] In some embodiments, obtaining a first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantization weights includes:
[0014] Obtain a product matrix of the first low-rank matrix and the second low-rank matrix;
[0015] The first weight matrix is obtained according to the sum of the quantization weight and the product matrix.
[0016] In some embodiments, the step of performing a quantization process on the first weight matrix to obtain a second weight matrix includes:
[0017] Acquire a first quantization constant and a second quantization constant, and use the first quantization constant to dequantize the second quantization constant to obtain a quantization intermediate value;
[0018] The first weight matrix is dequantized based on the quantized intermediate value to obtain the second weight matrix.
[0019] In some embodiments, obtaining an optimization target according to the first weight matrix, the second weight matrix, the first fine-tuning matrix, and the second fine-tuning matrix includes:
[0020] Obtaining a matrix product of the first fine-tuning matrix and the second fine-tuning matrix to obtain an optimized target matrix;
[0021] Obtaining a quantization difference matrix according to the sum of the optimization target matrix and the second weight matrix;
[0022] The optimization target is obtained by subtracting the quantization difference matrix from the first weight matrix.
[0023] In some embodiments, solving the optimization objective to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization objective includes:
[0024] Obtaining a first estimated value of the first fine-tuning matrix and a second estimated value of the second fine-tuning matrix at a previous iteration step;
[0025] Substituting the first estimated value and the second estimated value into the quantization difference matrix to obtain a quantization estimation matrix of a current iteration step;
[0026] Obtaining a residual matrix according to the difference between the first weight matrix and the quantization estimation matrix, performing singular value decomposition on the residual matrix to obtain singular values and left singular vectors and right singular vectors corresponding to the singular values;
[0027] Obtain the first estimated value and the second estimated value of the current iteration step number according to the singular value, the left singular vector and the right singular vector;
[0028] The iterative process is repeated until a preset number of iteration steps is reached, the first estimated value is obtained as the first solution matrix, and the second estimated value is obtained as the second solution matrix.
[0029] In some embodiments, before obtaining the first low-rank matrix and the second low-rank matrix updated by the server after aggregating the first solution matrix and the second solution matrix of each client, the method further includes:
[0030] quantizing the first weight matrix to obtain updated quantized weights;
[0031] Acquire a local data set corresponding to the client, and use the local data set to train the large language model corresponding to the updated quantization weight to obtain a fine-tuning update weight corresponding to the updated quantization weight;
[0032] The fine-tuning updated weights are subjected to low-rank decomposition to obtain the updated first low-rank matrix and the second low-rank matrix, wherein the ranks of the updated first low-rank matrix and the second low-rank matrix are smaller than the rank of the updated quantization weights.
[0033] In some embodiments, after obtaining the updated first low-rank matrix and the second low-rank matrix, the method further includes:
[0034] Sending the updated first low-rank matrix and the second low-rank matrix to a server;
[0035] A first average low-rank matrix and a second average low-rank matrix obtained by receiving a global average of the first low-rank matrix and the second low-rank matrix of at least one of the clients by a server;
[0036] The first average low-rank matrix is used as the first low-rank matrix, and the second average low-rank matrix is used as the second low-rank matrix.
[0037] In some embodiments, the server aggregates the first solution matrix and the second solution matrix of each of the clients, and the process of updating the first low-rank matrix and the second low-rank matrix includes:
[0038] Obtaining the first solution matrix and the second solution matrix of each of the clients;
[0039] If the dimension of at least one of the first solution matrices is different from that of other first solution matrices, performing data padding to align the data dimensions of each of the first solution matrices;
[0040] If the dimension of at least one of the second solution matrices is different from that of the other second solution matrices, performing data padding to align the data dimensions of each of the second solution matrices;
[0041] The first solution matrix is averaged to obtain the updated first low-rank matrix, and the second solution matrix is averaged to obtain the updated second low-rank matrix.
[0042] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present application proposes a distributed large language model quantization deployment device, including:
[0043] The first processing module is used to quantize the initial weights of the large language model deployed in the client to obtain quantized weights, and obtain a first low-rank matrix and a second low-rank matrix obtained by low-rank decomposition and related to weight update of the quantized weights;
[0044] A second processing module: used to obtain a first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantization weight, perform a quantization process on the first weight matrix to obtain a second weight matrix, and obtain an optimization target according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix;
[0045] Optimization solution module: used for solving the optimization target to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization target;
[0046] Aggregation update module: used to obtain the updated first low-rank matrix and the second low-rank matrix after the server aggregates the first solution matrix and the second solution matrix of each client, update the optimization target according to the updated first low-rank matrix and the second low-rank matrix, iteratively update the first solution matrix and the second solution matrix until the iteration termination condition is reached, obtain the trained large language model according to the first weight matrix, and update the trained large language model in each client.
[0047] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.
[0048] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0049] The distributed large language model quantization deployment method, device, equipment and storage medium proposed in the embodiment of the present application, obtains quantized weights by quantizing the initial weights of the large language model, obtains the first low-rank matrix and the second low-rank matrix, obtains the first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantized weights, quantizes the first weight matrix to obtain the second weight matrix, obtains the optimization target according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix, solves the optimization target, obtains the first solution matrix of the first fine-tuning matrix and the second solution matrix of the second fine-tuning matrix that minimizes the optimization target, obtains the first low-rank matrix and the second low-rank matrix updated after the server aggregates the first solution matrix and the second solution matrix of each client, updates the optimization target according to the updated first low-rank matrix and the second low-rank matrix, iteratively updates the first solution matrix and the second solution matrix until the iteration termination condition is reached, obtains the trained large language model according to the first weight matrix, and updates the trained large language model in each client. The embodiment of the present application first quantizes the initial weights and reduces the number of parameters in the training process based on low-rank decomposition, so that the computational overhead required for training can be reduced accordingly. Reducing the precision of parameters during quantization can further optimize training efficiency and reduce training overhead. During server aggregation, it is not necessary to send all model parameters, but to aggregate some parameters, which can reduce the communication overhead required in the parameter transmission process and improve the overall training efficiency of the federated learning process. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1It is a flowchart of the distributed large language model quantization deployment method provided in an embodiment of the present application.
[0051] Figure 2 A flowchart for obtaining a first low-rank matrix and a second low-rank matrix provided in an embodiment of the present application.
[0052] Figure 3 A flowchart of a process of quantizing a first weight matrix to obtain a second weight matrix is provided in an embodiment of the present application.
[0053] Figure 4 It is a schematic diagram of obtaining an optimization target based on a first weight matrix, a second weight matrix, a first fine-tuning matrix and a second fine-tuning matrix provided in an embodiment of the present application.
[0054] Figure 5 It is a flowchart of solving the optimization target provided by an embodiment of the present application to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimizes the optimization target.
[0055] Figure 6 A schematic diagram of a process in which a server provided in an embodiment of the present application aggregates a first solution matrix and a second solution matrix of each client and updates a first low-rank matrix and a second low-rank matrix.
[0056] Figure 7 It is a schematic diagram of the overall process provided by the embodiment of the present application.
[0057] Figure 8 This is a structural block diagram of a distributed large language model quantization deployment device provided by another embodiment of the present application.
[0058] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0060] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.
[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0062] First, some nouns involved in this application are analyzed:
[0063] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0064] Federated learning is a distributed machine learning method whose main goal is to train models through collaboration and sharing without centralizing data processing. Data is usually distributed across multiple clients instead of being stored centrally on a server. These client devices each have local data for training local models. In federated learning, clients only need to upload parameter updates of local models instead of raw data, thereby protecting data privacy. Model training is usually distributed across multiple clients, and clients use local data to train models and generate updates to local models. Each client then sends its local model updates to a central server, which is responsible for aggregating these updates and generating updates to the global model. The above process is then iterative. The aggregated global model updates are sent back to the client, and the client performs the next round of local training based on the new global model. This iterative process continues until the global model converges or reaches a predetermined performance indicator.
[0065] For example, the input method application of smartphones in mobile devices can use each user's input data to optimize the input method model through federated learning without uploading the user's input content. For example, smart home devices in the Internet of Things can use local data on each device to optimize the control algorithm through federated learning without transmitting data to the cloud. For example, patient data from different hospitals in the medical and health field can be used through federated learning to jointly train medical diagnosis models without sharing patient privacy data.
[0066] With the rapid development of large language models, related technologies have also attempted to deploy these large models on the client side to achieve richer functions. However, deploying large language models on the client side usually encounters the following problems: large language models usually have hundreds of millions to tens of billions of parameters. When training on the client side, the client device (such as mobile phones, tablets, embedded devices, etc.) needs to have strong computing power. In the federated learning scenario, the client needs to communicate with the server frequently to upload and download model updates. Due to the huge number of parameters in large language models, the amount of transmitted data is too large, the communication overhead is high, and the energy consumption is high. In addition, client applications usually require real-time response. When the amount of transmitted data is large, it will also cause insufficient network bandwidth and increased latency, which cannot meet real-time performance requirements.
[0067] Based on this, the embodiments of the present application provide a distributed large language model quantization deployment method, apparatus, device and storage medium. First, by quantizing the initial weights and reducing the number of parameters in the training process based on low-rank decomposition, the computational overhead required for training can be reduced accordingly. Reducing the accuracy of the parameters during the quantization process can further optimize the training efficiency and reduce the training overhead. During the server aggregation process, it is not necessary to send all model parameters, but to aggregate some parameters, which can reduce the communication overhead required in the transmission parameter process and improve the overall training efficiency of the federated learning process.
[0068] The embodiments of the present application provide a distributed large language model quantization deployment method, apparatus, device and storage medium, which are specifically illustrated by the following embodiments. First, the distributed large language model quantization deployment method in the embodiments of the present application is described.
[0069] The distributed large language model quantization deployment method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The distributed large language model quantization deployment method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be a computer program running in the terminal or the server side. For example, the computer program can be a native program or software module in the operating system; it can be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a client that supports large language model training, or it can be a small program, that is, a program that can be run only by downloading it to the browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in. Among them, the terminal communicates with the server through a network. The distributed large language model quantization deployment method can be executed by the terminal or the server, or by the terminal and the server in collaboration.
[0070] In some embodiments, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc. In addition, the terminal may also be an intelligent vehicle-mounted device. The intelligent vehicle-mounted device applies the distributed large language model quantitative deployment method of this embodiment to provide related services to improve the driving experience. The server may be an independent server, or it may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms; it may also be a service node in a blockchain system, and each service node in the blockchain system forms a peer-to-peer (P2P, Peer To Peer, P2P) network. The P2P protocol is an application layer protocol running on the Transmission Control Protocol (TCP) protocol. The terminal and the server may be connected via Bluetooth, Universal Serial Bus (USB) or network communication connection methods, which are not limited in this embodiment.
[0071] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0072] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.
[0073] The following describes the distributed large language model quantization deployment method in the embodiment of the present application.
[0074] Figure 1 is an optional flowchart of the distributed large language model quantization deployment method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps 110 to 140. It can also be understood that this embodiment Figure 1 The order of step 110 to step 140 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.
[0075] Step 110: quantize the initial weights of the large language model deployed in the client to obtain quantized weights, and obtain a first low-rank matrix and a second low-rank matrix obtained through low-rank decomposition and related to weight update of the quantized weights.
[0076] In one embodiment, during the federated learning process, in the initial state, the server sends a pre-trained large language model to each client, and the pre-trained large language model has an initial weight It is understandable that the initial weights of the large language model deployed in each client are the same. Since the large language model has a large number of parameters, the embodiment of the present application selects to fine-tune some parameters in the client, thereby avoiding the problem of large number of parameters and high computational overhead required for training caused by fine-tuning all parameters. The initial weights of the global language model deployed in each client are the same. Quantize to obtain the quantized weight corresponding to each client, where the quantized weight of the i-th client is expressed as .
[0077] The purpose of quantization is to convert continuous or high-precision values (such as floating-point numbers) into lower-precision or discrete representations (such as integers or floating-point numbers with lower digits). The quantization process usually means obtaining a data type with more bits and converting it to fewer bits, such as converting a 32-bit floating-point number to an 8-bit integer. In the embodiment of the present application, quantization may be quantile quantization, which performs matrix quantization based on the selected quantization scale. Different clients may select different quantization scales, which are not limited in this embodiment.
[0078] In one embodiment, considering that during the training process, the neural network corresponding to the large language model actually contains many dense layers that perform matrix multiplication, the weight matrices in these layers are usually full rank. When adapting to a specific task, the learned over-parameterized model actually exists in a low intrinsic dimension, so the pre-trained large language model has a lower "intrinsic dimension" and can still be effectively learned despite being randomly projected to a smaller subspace. Therefore, the embodiment of the present application indirectly trains some dense layers in the neural network by optimizing the rank decomposition matrix of the dense layer that changes during the adaptation process, while keeping the pre-trained weights unchanged. The specific process is to obtain the first low-rank matrix and the second low-rank matrix obtained by low-rank decomposition that are related to the weight update of the quantized weights.
[0079] In one embodiment, referring to Figure 2 , Figure 2 The flowchart for obtaining the first low-rank matrix and the second low-rank matrix provided in the embodiment of the present application specifically includes the following steps:
[0080] Step 210: Obtain a local data set corresponding to the client, and use the local data set to train the quantized large language model to obtain an update weight corresponding to the quantized weight.
[0081] Among them, the quantized weights are applied to the large language model to obtain the quantized large language model. Since each client has a corresponding local data set, the quantized large language model is trained once using the local data set to obtain an updated weight. , the updated weight corresponds to the update of the quantized weight.
[0082] Step 220: Perform low-rank decomposition on the updated weights to obtain a first low-rank matrix and a second low-rank matrix.
[0083] In one embodiment, the weight of the trained large language model is expressed as: , where the update weights The dimension is At this time, considering the low intrinsic dimension mentioned above, the update weight Perform low-rank decomposition and update the weights Convert it to the product of two low-rank matrices to reduce the number of parameters. The specific process is: update the weights Perform low-rank decomposition to obtain the first low-rank matrix and the second low-rank matrix , where the rank of the first low-rank matrix and the second low-rank matrix are both r, and the rank r is less than the rank of the updated weight , expressed as: Therefore, update the weights It is expressed as: , the gradient update of the weights in the training process is changed to the gradient update of the two low-rank matrices, thereby reducing the number of parameters in the training process. The matrix of dimension is transformed into: and The product of two low-rank matrices .
[0084] It is understandable that the low-rank decomposition can be obtained by means of singular value decomposition, which is not limited in this embodiment.
[0085] Step 120: Obtain a first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantization weights, perform a quantization process on the first weight matrix to obtain a second weight matrix, and obtain an optimization target based on the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix.
[0086] In one embodiment, firstly, a first weight matrix is obtained based on the first low-rank matrix, the second low-rank matrix and the quantization weights, specifically: a product matrix of the first low-rank matrix and the second low-rank matrix is obtained, and the first weight matrix is obtained according to the sum of the quantization weights and the product matrix. The product matrix here is the above-mentioned update matrix, so the first weight matrix is expressed as: .
[0087] In one embodiment, under the action of the first weight matrix, assuming that the input data is , the corresponding output data is expressed as:
[0088]
[0089] Next, in one embodiment, the first weight matrix is further quantized to reduce the computational complexity of the training process. Specifically, the first weight matrix is quantized to obtain a second weight matrix.
[0090] First, we take the example of quantizing a 32-bit floating point (FP32) tensor into an Int8 tensor in the range of [−127, 127] to explain the meaning of the following quantization:
[0091]
[0092]
[0093] in, Represents an 8-bit integer tensor, represents a 32-bit floating point tensor, Represents the maximum value in a 32-bit floating point tensor. The quantization constant is also called the quantization scale.
[0094] It can be understood that dequantization corresponds to quantization, expressed as:
[0095]
[0096] in, Represents dequantization.
[0097] In one embodiment, the first weight matrix is subjected to a secondary quantization process. Specifically, the first weight matrix is first quantized to obtain a quantized intermediate weight matrix and the corresponding first quantization constant , and then the first quantization constant generated in the first quantization process Perform a second quantization to obtain a quantized intermediate quantization constant and the second quantization constant For example, nonlinear 4-bit quantization is used for the first quantization, and the quantization result is a 4-bit integer, represented as NF4 quantization. FP8 quantization is used for the second quantization, and the quantization result is a half-precision floating point number. Compared with 32-bit floating point numbers, half-precision floating point numbers can reduce the model's memory usage and speed up the calculation process while reducing accuracy. It can be understood that the above quantization algorithm is only for illustration.
[0098] The first quantization process mentioned above is expressed as:
[0099]
[0100] in, Represents quantization, represents the first quantization constant, represents the first weight matrix, and represents the intermediate weight matrix after quantization.
[0101] The second quantization process is expressed as:
[0102]
[0103] in, represents the first quantization constant, represents the second quantization constant, Represents an intermediate quantization constant.
[0104] Combining the above two steps, the secondary quantization process is expressed as:
[0105]
[0106] After secondary quantization, a corresponding dequantization process needs to be performed in the output data, and the model weights corresponding to the dequantization are recorded as the second weight matrix. Figure 3 , Figure 3 The flowchart of the quantization process of the first weight matrix to obtain the second weight matrix provided in the embodiment of the present application specifically includes the following steps:
[0107] Step 310: Obtain a first quantization constant and a second quantization constant, and use the first quantization constant to dequantize the second quantization constant to obtain a quantization intermediate value.
[0108] Among them, since the second quantization constant is obtained by quantizing the first quantization constant, the corresponding dequantization process is expressed as:
[0109] C=
[0110] Among them, C represents the quantized intermediate value, represents the first quantization constant, It is understood that dequantization corresponds to quantization, which is obtained according to the quantization algorithm actually selected.
[0111] Step 320: Dequantize the first weight matrix based on the quantized intermediate value to obtain a second weight matrix.
[0112] Among them, the second weight matrix is expressed as:
[0113]
[0114] in, Represents double dequantization.
[0115] At this time, the second weight matrix is updated in combination. At this time, according to the above low-rank decomposition method, the first fine-tuning matrix related to the weight update of the i-th client in the second weight matrix is obtained: and the second fine-tuning matrix , the output data obtained at this time is expressed as:
[0116]
[0117] Although the quantization process can reduce the computational complexity, it will bring about error loss. When data is converted from high precision to low precision, some information must be discarded. For example, when a 32-bit floating point number is quantized into an 8-bit integer, a lot of details and precise values are lost, and the information loss process is irreversible. In addition, quantization involves approximation of the original value, and each value is approximated to the nearest quantization level. Therefore, it is also necessary to fine-tune the error caused by quantization to reduce the error.
[0118] Therefore, in the embodiment of the present application, the output data under the action of the first weight matrix is subtracted from the output data under the action of the second weight matrix to define the error caused by quantization, which is expressed as:
[0119]
[0120] From the above, it can be seen that the error caused by the first weight matrix and the second weight matrix as a whole can be defined as:
[0121]
[0122] In one embodiment, the optimization target can be obtained according to the error. Figure 4 , Figure 4 It is a schematic diagram of obtaining an optimization target according to a first weight matrix, a second weight matrix, a first fine-tuning matrix, and a second fine-tuning matrix provided in an embodiment of the present application, which specifically includes the following steps:
[0123] Step 410: Obtain the matrix product of the first fine-tuning matrix and the second fine-tuning matrix to obtain an optimized target matrix.
[0124] Among them, the optimization target matrix is expressed as: .
[0125] Step 420: Obtain a quantization difference matrix according to the sum of the optimization target matrix and the second weight matrix.
[0126] Among them, the quantization difference matrix is expressed as: .
[0127] Step 430: Subtract the quantization difference matrix from the first weight matrix to obtain an optimization target.
[0128] The optimization objective is expressed as:
[0129]
[0130] Step 130: Solve the optimization target to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization target.
[0131] In one embodiment, the optimization objective is solved by minimization, which is expressed as:
[0132]
[0133] The meaning of the above formula is that in the first fine-tuning matrix and the second fine-tuning matrix Under the condition of , the optimization objective reaches the minimum value.
[0134] In one embodiment, referring to Figure 5 , Figure 5 The flowchart of solving the optimization target and obtaining the first solution matrix of the first fine-tuning matrix and the second solution matrix of the second fine-tuning matrix that minimizes the optimization target provided by the embodiment of the present application specifically includes the following steps:
[0135] Step 510: Obtain a first estimated value of the first fine-tuning matrix and a second estimated value of the second fine-tuning matrix at a previous iteration step.
[0136] Assuming that the current iteration number is t, the previous iteration number is t-1, and the first estimated value of the first fine-tuning matrix corresponding to the current iteration number is , the second estimate of the second fine-tuning matrix is , the first estimate of the first fine-tuning matrix at the previous iteration is , the second estimate of the second fine-tuning matrix is .
[0137] It can be understood that if the current iteration step is the first step, the first estimated value is the first initial value , the second estimated value is the second initial value . The first initial value and the second initial value may be zero.
[0138] Step 520: Substitute the first estimated value and the second estimated value into the quantization difference matrix to obtain the quantization estimation matrix of the current iteration step.
[0139] Among them, the quantization estimation matrix corresponding to the t-th iteration step is It is expressed as:
[0140]
[0141] Where T stands for transpose.
[0142] Step 530: Obtain a residual matrix according to the difference between the first weight matrix and the quantization estimation matrix, perform singular value decomposition on the residual matrix, and obtain singular values and left singular vectors and right singular vectors corresponding to the singular values.
[0143] Among them, the residual matrix corresponding to the t-th iteration step is It is expressed as:
[0144] =
[0145] Then the residual matrix Perform singular value decomposition to obtain the singular value and the left singular vector and right singular vector corresponding to the singular value, expressed as:
[0146]
[0147]
[0148] in, represents the jth singular value, represents the number of singular values, represents the maximum dimension of the first weight matrix, represents the maximum dimension of the second weight matrix, represents the left singular vector corresponding to the jth singular value, Denotes the right singular vector corresponding to the j-th singular value.
[0149] Among them, the left singular vector is the column vector of the matrix U, and the right singular vector is the column vector of the matrix V. Assume that the residual matrix The dimension is m×n. The core of singular value decomposition is to transform the residual matrix It is expressed as the product of three special matrices, namely =UΣV, where U is an m×m unitary matrix, Σ is an m×n semidefinite diagonal matrix, and V is the conjugate transposed matrix of the n×n unitary matrix. In this decomposition, the diagonal elements of Σ are the residual matrix The singular values of .
[0150] Step 540: Obtain a first estimated value and a second estimated value of the current iteration step number according to the singular values, the left singular vectors and the right singular vectors.
[0151] Among them, the first estimate of the number of iterations at t is It is expressed as:
[0152]
[0153] Second estimate It is expressed as:
[0154]
[0155] in, is the residual matrix rank.
[0156] Step 550: Repeat the iteration process until a preset number of iteration steps is reached, obtain the first estimated value as the first solution matrix, and obtain the second estimated value as the second solution matrix.
[0157] Among them, the total number of iteration steps can be set in advance. When the number of iteration steps reaches the preset number of iteration steps, the iteration is terminated to obtain the first estimated value and the second estimated value corresponding to the preset number of iteration steps, and the first estimated value is used as the first solution matrix and the second estimated value is used as the second solution matrix.
[0158] In one embodiment, the preset number of iteration steps may be a fixed number of steps set according to the model size and complexity of the large language model. For example, five iterations are performed, and the first solution matrix and the second solution matrix are obtained according to the first estimated value and the second estimated value after the fifth iteration. A dynamically changing preset number of iteration steps may also be set, and the residual matrix of the two previous and next iteration results is If the change in is lower than a certain set threshold, the iteration is stopped. It can be understood that the embodiment of the present application does not limit the preset number of iteration steps, and can be set according to actual conditions.
[0159] According to the above process, each client obtains the first solution matrix and the second solution matrix of the client, where the first solution matrix is equivalent to the first fine-tuning matrix corresponding to the last iteration process. , the second solution matrix is equivalent to the second fine-tuning matrix corresponding to the last iteration process , so the first solution matrix can be expressed as , the second solution matrix is expressed as .
[0160] In one embodiment, in order to further reduce the impact of local data, two fine-tuning methods are adopted. The first fine-tuning method is described in detail as follows.
[0161] The first solution matrix in the client is obtained by the above process and the second solution matrix At this time, the first solution matrix and the second solution matrix are fixed and unchanged, and the large model deployed on the client can be fine-tuned. The fine-tuning process is specifically as follows: quantize the first weight matrix to obtain updated quantization weights, obtain the local data set corresponding to the client, use the local data set to train the large language model corresponding to the updated quantization weights, obtain the fine-tuning update weights corresponding to the updated quantization weights, perform low-rank decomposition on the fine-tuning update weights, and obtain the updated first low-rank matrix and the second low-rank matrix. The ranks of the updated first low-rank matrix and the second low-rank matrix are less than the ranks of the updated quantization weights.
[0162] In one embodiment, the first weight matrix As a whole, similar to the initial weights , then quantize the first weight matrix to get the updated quantized weight , the updated quantized weights correspond to an updated large language model. Then obtain the local data set of the corresponding client to retrain the updated large language model to obtain the new updated weights , the updated weight corresponds to the update of the quantized weight, and the weight of the trained large language model is expressed as: According to the low-rank decomposition process as described above, the new update weights Perform low-rank decomposition and update the new weights It is converted into the product of two low-rank matrices to reduce the number of parameters. After low-rank decomposition, a new first low-rank matrix and a new second low-rank matrix are obtained.
[0163] It is understandable that, since a low-rank decomposition process is performed, the ranks of the new first low-rank matrix and the new second low-rank matrix are less than the rank of the updated quantization weights. The above method further improves the accuracy of the result of the large language model in the client.
[0164] The second fine-tuning method further reduces local data bias based on the first fine-tuning method, as described below.
[0165] In one embodiment, after the first low-rank matrix and the second low-rank matrix are updated using the second low-rank decomposition process, the updated first low-rank matrix and the second low-rank matrix in different clients can also be distributed globally, thereby further reducing the bias caused by local data. Therefore, in an embodiment of the present application, the updated first low-rank matrix and the second low-rank matrix are sent to the server, and the first low-rank matrix and the second low-rank matrix trained by different local data sets are globally averaged in the server to obtain a first average low-rank matrix and a second average low-rank matrix. Then the client receives the first average low-rank matrix and the second average low-rank matrix obtained by globally averaging the first low-rank matrix and the second low-rank matrix of at least one client, and uses the first average low-rank matrix as the first low-rank matrix, and uses the second average low-rank matrix as the second low-rank matrix.
[0166] That is, the first low-rank matrix after the server updates different clients and the second low-rank matrix After global averaging, the global first average low-rank matrix is obtained and the second average low-rank matrix , and then the first average low-rank matrix and the second average low-rank matrix Return to each client to replace the original first low-rank matrix in the client and the second low-rank matrix In the subsequent process, the global related low-rank matrix is used for updating, thereby avoiding the data bias problem caused by local data during training.
[0167] In one embodiment, since the above two different fine-tuning methods will adjust the first low-rank matrix and the second low-rank matrix Update, assuming that the updated first low-rank matrix is expressed as , the updated second low-rank matrix is expressed as , after using it to replace the relevant parameters, the corresponding output data is expressed as:
[0168]
[0169] The optimization objective is further expressed as:
[0170]
[0171] The solution is also obtained through the minimization process, which is expressed as:
[0172]
[0173] Solve the new optimization target in the above manner to obtain a new first solution matrix and a second solution matrix.
[0174] Step 140: After obtaining the first solution matrix and the second solution matrix of each client from the server and aggregating them, the updated first low-rank matrix and the second low-rank matrix are updated, the optimization target is updated according to the updated first low-rank matrix and the second low-rank matrix, the first solution matrix and the second solution matrix are iteratively updated until the iteration termination condition is reached, the trained large language model is obtained according to the first weight matrix, and the trained large language model is updated in each client.
[0175] In one embodiment, the data gap between different clients is large and each has its own limitations. Therefore, the embodiment of the present application integrates the bias caused by the heterogeneous data of different clients by aggregation. After each client has a corresponding first solution matrix and a second solution matrix, the first solution matrix and the second solution matrix of each client are sent to the server to perform parameter aggregation in the federated learning process and update the global large language model. The subsequent client training process can be based on the globally updated first low-rank matrix and the second low-rank matrix, thereby solving the bias problem caused by local data.
[0176] Reference Figure 6 , Figure 6 A schematic diagram of a process in which a server provided in an embodiment of the present application aggregates a first solution matrix and a second solution matrix of each client and updates a first low-rank matrix and a second low-rank matrix, specifically comprising the following steps:
[0177] Step 610: Obtain the first solution matrix and the second solution matrix of each client.
[0178] Step 620: If the dimension of at least one first solution matrix is different from that of other first solution matrices, perform data padding to align the data dimension of each first solution matrix.
[0179] Step 630: If the dimension of at least one second solution matrix is different from that of other second solution matrices, perform data padding to align the data dimension of each second solution matrix.
[0180] Among them, due to the different resource conditions of different clients, the quantization algorithm and the corresponding rank in the low-rank decomposition will be adaptively selected, resulting in the size and structure of the first solution matrix and the second solution matrix may be different. Therefore, in the embodiment of the present application, the server will align the dimensions of the obtained first solution matrix and the second solution matrix respectively. The alignment method is to fill the low-dimensional matrix with zeros to obtain the first solution matrix and the second solution matrix with the same dimension. For example, according to 16 bits, the aligned first solution matrix is expressed as: , the first solution matrix after alignment is expressed as: .
[0181] Step 640: Calculate the average of the first solution matrix to obtain an updated first low-rank matrix, and calculate the average of the second solution matrix to obtain an updated second low-rank matrix.
[0182] Assuming that the federated learning system has N clients, the first low-rank matrix The update process is expressed as:
[0183]
[0184] The second low-rank matrix The update process is expressed as:
[0185]
[0186] The updated first low-rank matrix and the second low-rank matrix can reflect the parameter update status of all clients, so the server re-sends the updated first low-rank matrix and the second low-rank matrix to all clients for the federated learning process.
[0187] After the client receives the updated first low-rank matrix and the second low-rank matrix, the optimization target is updated according to the updated first low-rank matrix and the second low-rank matrix in the manner of the above embodiment, and then the first solution matrix and the second solution matrix are updated according to the solution method of the optimization target, and then the parameters are aggregated again on the server, and so on for multiple iterations until the iteration termination condition is reached. At this time, the iteration termination condition can be reaching the set number of iterations, or it can be that the large language model converges and the parameters are stable, for example, during the iteration process, the first low-rank matrix and the second low-rank matrix tend to be stable, and the change amount is less than the preset change amount.
[0188] After the iteration is terminated, a global first low-rank matrix and a second low-rank matrix can be obtained. Then, each client obtains a first weight matrix that integrates its own training process and the training process of other clients based on the quantization matrix, the first low-rank matrix and the second low-rank matrix. The trained large language model can be obtained using the first weight matrix.
[0189] In one embodiment, referring to Figure 7 , Figure 7 It is a schematic diagram of the overall process provided by the embodiment of the present application. Figure 7 First, the initial weights of the large language model deployed in the client are quantized to obtain the quantized weights. Then, the local data set corresponding to the client is obtained, and the quantized large language model is trained using the local data set to obtain the updated weights corresponding to the quantized weights, and the updated weights are low-rank decomposed to obtain the first low-rank matrix and the second low-rank matrix. Then, the first weight matrix is obtained based on the first low-rank matrix, the second low-rank matrix and the quantized weights, and then the second weight matrix is obtained according to the first weight matrix, and the optimization target is obtained according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix. The optimization target is iteratively solved to determine whether the preset number of iterations is reached until the first solution matrix of the first fine-tuning matrix and the second solution matrix of the second fine-tuning matrix are obtained. Then, each client sends the first solution matrix and the second solution matrix to the server for aggregation, and updates the first low-rank matrix and the second low-rank matrix. The server then sends the updated first low-rank matrix and the second low-rank matrix to the client to update the optimization target of the client until the update results of the first low-rank matrix and the second low-rank matrix in the server meet the iteration termination condition, thereby obtaining the large language model in each client, and updating the trained large language model in each client.
[0190] As can be seen from the above, in the parameter update process of federated learning in the embodiment of the present application, the client sends the first solution matrix and the second solution matrix to the server, and the server returns the updated first low-rank matrix and the second low-rank matrix to the client, and the amount of parameters transmitted is significantly reduced. In addition, since the quantization and low-rank decomposition processes are used in the training process, the model size and computational complexity of the large language model are compressed, so that the memory requirements of the large language model deployed in the client are reduced, so that the training process does not require more computing resources. By optimizing the solution process of the target, the loss caused by the quantization error is minimized, and the performance of the large language model is maintained, so that the large language model with better performance can be deployed and applied in the client, and the response efficiency of the large language model is improved.
[0191] The technical solution provided by the embodiment of the present application is to obtain quantized weights by quantizing the initial weights of the large language model, obtain the first low-rank matrix and the second low-rank matrix, obtain the first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantized weights, and obtain the second weight matrix by quantizing the first weight matrix. The optimization target is obtained according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix, and the optimization target is solved to obtain the first solution matrix of the first fine-tuning matrix and the second solution matrix of the second fine-tuning matrix that minimizes the optimization target. The server aggregates the first solution matrix and the second solution matrix of each client, and then updates the first low-rank matrix and the second low-rank matrix. The optimization target is updated according to the updated first low-rank matrix and the second low-rank matrix, and the first solution matrix and the second solution matrix are iteratively updated until the iteration termination condition is reached. The trained large language model is obtained according to the first weight matrix, and the trained large language model is updated in each client. The embodiment of the present application first quantizes the initial weights and reduces the number of parameters in the training process based on low-rank decomposition, so that the computational overhead required for training can be reduced accordingly. Reducing the accuracy of the parameters during the quantization process can further optimize the training efficiency and reduce the training overhead. During the server aggregation process, there is no need to send all model parameters, but rather aggregate some parameters. This can reduce the communication overhead required in the parameter transmission process and improve the overall training efficiency of the federated learning process.
[0192] The present application also provides a distributed large language model quantization deployment device, which can implement the above-mentioned distributed large language model quantization deployment method, referring to Figure 8 , the device comprises:
[0193] The first processing module 810 is used to quantize the initial weights of the large language model deployed in the client to obtain quantized weights, and obtain a first low-rank matrix and a second low-rank matrix related to weight update of the quantized weights and obtained through low-rank decomposition.
[0194] The second processing module 820 is used to obtain a first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantization weights, perform a quantization process on the first weight matrix to obtain a second weight matrix, and obtain an optimization target according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix.
[0195] Optimization solution module 830 is used to solve the optimization target to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization target.
[0196] Aggregation update module 840: used to obtain the updated first low-rank matrix and second low-rank matrix after the server aggregates the first solution matrix and the second solution matrix for each client, update the optimization target according to the updated first low-rank matrix and the second low-rank matrix, iteratively update the first solution matrix and the second solution matrix until the iteration termination condition is reached, obtain the trained large language model according to the first weight matrix, and update the trained large language model in each client.
[0197] The specific implementation of the distributed large language model quantization deployment device of this embodiment is basically the same as the specific implementation of the distributed large language model quantization deployment method described above, and will not be repeated here.
[0198] The present application also provides an electronic device, including:
[0199] at least one memory;
[0200] at least one processor;
[0201] at least one program;
[0202] The program is stored in the memory, and the processor executes the at least one program to implement the distributed large language model quantization deployment method implemented in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.
[0203] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0204] The processor 901 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0205] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the distributed large language model quantization deployment method of the embodiment of this application;
[0206] Input / output interface 903, used to implement information input and output;
[0207] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and
[0208] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);
[0209] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .
[0210] An embodiment of the present application also provides a storage medium, which is a storage medium that stores a computer program. When the computer program is executed by a processor, the above-mentioned distributed large language model quantization deployment method is implemented.
[0211] As a non-transitory storage medium, the memory can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0212] The distributed large language model quantization deployment method, device, equipment and storage medium proposed in the embodiment of the present application, obtains quantized weights by quantizing the initial weights of the large language model, obtains the first low-rank matrix and the second low-rank matrix, obtains the first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantized weights, quantizes the first weight matrix to obtain the second weight matrix, obtains the optimization target according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix, solves the optimization target, obtains the first solution matrix of the first fine-tuning matrix and the second solution matrix of the second fine-tuning matrix that minimizes the optimization target, obtains the first low-rank matrix and the second low-rank matrix updated after the server aggregates the first solution matrix and the second solution matrix of each client, updates the optimization target according to the updated first low-rank matrix and the second low-rank matrix, iteratively updates the first solution matrix and the second solution matrix until the iteration termination condition is reached, obtains the trained large language model according to the first weight matrix, and updates the trained large language model in each client. The embodiment of the present application first quantizes the initial weights and reduces the number of parameters in the training process based on low-rank decomposition, so that the computational overhead required for training can be reduced accordingly. Reducing the precision of parameters during quantization can further optimize training efficiency and reduce training overhead. During server aggregation, it is not necessary to send all model parameters, but to aggregate some parameters, which can reduce the communication overhead required in the parameter transmission process and improve the overall training efficiency of the federated learning process.
[0213] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0214] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0215] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0216] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0217] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0218] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0219] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0220] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0221] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0222] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store programs.
[0223] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.
Claims
1. A distributed large language model quantization deployment method, characterized in that: include: Quantize the initial weights of the large language model deployed in the client to obtain quantized weights, and obtain a first low-rank matrix and a second low-rank matrix related to weight update of the quantized weights and obtained through low-rank decomposition; Based on the first low-rank matrix, the second low-rank matrix and the quantized weights, a first weight matrix is obtained, a second weight matrix is obtained by quantizing the first weight matrix, and an optimization target is obtained according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix; Solving the optimization objective to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization objective; Obtain the first low-rank matrix and the second low-rank matrix updated after the server aggregates the first solution matrix and the second solution matrix of each of the clients, update the optimization target according to the updated first low-rank matrix and the second low-rank matrix, iteratively update the first solution matrix and the second solution matrix until an iteration termination condition is reached, obtain the trained large language model according to the first weight matrix, and update the trained large language model in each of the clients; The obtaining of the optimization target according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix includes: Obtaining a first fine-tuning matrix and a second fine-tuning matrix based on the second weight matrix, obtaining a matrix product of the first fine-tuning matrix and the second fine-tuning matrix, and obtaining an optimized target matrix; Obtaining a quantization difference matrix according to the sum of the optimization target matrix and the second weight matrix; Subtracting the quantization difference matrix from the first weight matrix to obtain the optimization target; The step of solving the optimization objective to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization objective includes: Obtaining a first estimated value of the first fine-tuning matrix and a second estimated value of the second fine-tuning matrix at a previous iteration step; Substituting the first estimated value and the second estimated value into the quantization difference matrix to obtain a quantization estimation matrix of a current iteration step; Obtaining a residual matrix according to the difference between the first weight matrix and the quantization estimation matrix, performing singular value decomposition on the residual matrix to obtain singular values and left singular vectors and right singular vectors corresponding to the singular values; Obtain the first estimated value and the second estimated value of the current iteration step number according to the singular value, the left singular vector and the right singular vector; The iterative process is repeated until a preset number of iteration steps is reached, the first estimated value is obtained as the first solution matrix, and the second estimated value is obtained as the second solution matrix.
2. The distributed large language model quantization deployment method according to claim 1 is characterized in that: The obtaining of a first low-rank matrix and a second low-rank matrix related to the weight update of the quantization weight and obtained through low-rank decomposition includes: Acquire a local data set corresponding to the client, and use the local data set to train the quantized large language model to obtain an update weight corresponding to the quantized weight; Perform low-rank decomposition on the updated weights to obtain the first low-rank matrix and the second low-rank matrix, where the ranks of the first low-rank matrix and the second low-rank matrix are smaller than the rank of the updated weights.
3. The distributed large language model quantization deployment method according to claim 2 is characterized in that: The obtaining a first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantization weights includes: Obtain a product matrix of the first low-rank matrix and the second low-rank matrix; The first weight matrix is obtained according to the sum of the quantization weight and the product matrix.
4. The distributed large language model quantization deployment method according to claim 1, characterized in that: The step of performing a quantization process on the first weight matrix to obtain a second weight matrix includes: Acquire a first quantization constant and a second quantization constant, and use the first quantization constant to dequantize the second quantization constant to obtain a quantization intermediate value; The first weight matrix is dequantized based on the quantized intermediate value to obtain the second weight matrix.
5. The distributed large language model quantization deployment method according to claim 1, characterized in that: Before obtaining the first low-rank matrix and the second low-rank matrix updated by the server after aggregating the first solution matrix and the second solution matrix of each client, the method further includes: quantizing the first weight matrix to obtain updated quantized weights; Acquire a local data set corresponding to the client, and use the local data set to train the large language model corresponding to the updated quantization weight to obtain a fine-tuning update weight corresponding to the updated quantization weight; The fine-tuning updated weights are subjected to low-rank decomposition to obtain the updated first low-rank matrix and the second low-rank matrix, wherein the ranks of the updated first low-rank matrix and the second low-rank matrix are smaller than the rank of the updated quantization weights.
6. The distributed large language model quantization deployment method according to claim 5, characterized in that: After obtaining the updated first low-rank matrix and the second low-rank matrix, the method further includes: Sending the updated first low-rank matrix and the second low-rank matrix to a server; A first average low-rank matrix and a second average low-rank matrix obtained by receiving a global average of the first low-rank matrix and the second low-rank matrix of at least one of the clients by a server; The first average low-rank matrix is used as the first low-rank matrix, and the second average low-rank matrix is used as the second low-rank matrix.
7. The distributed large language model quantization deployment method according to claim 1, characterized in that: The server aggregates the first solution matrix and the second solution matrix of each of the clients, and the process of updating the first low-rank matrix and the second low-rank matrix includes: Obtaining the first solution matrix and the second solution matrix of each of the clients; If the dimension of at least one of the first solution matrices is different from that of other first solution matrices, performing data padding to align the data dimensions of each of the first solution matrices; If the dimension of at least one of the second solution matrices is different from that of the other second solution matrices, performing data padding to align the data dimensions of each of the second solution matrices; The first solution matrix is averaged to obtain the updated first low-rank matrix, and the second solution matrix is averaged to obtain the updated second low-rank matrix.
8. A distributed large language model quantization deployment device, characterized in that: include: The first processing module is used to quantize the initial weights of the large language model deployed in the client to obtain quantized weights, and obtain a first low-rank matrix and a second low-rank matrix obtained by low-rank decomposition and related to weight update of the quantized weights; A second processing module: used to obtain a first weight matrix based on the first low-rank matrix, the second low-rank matrix and the quantization weight, perform a quantization process on the first weight matrix to obtain a second weight matrix, and obtain an optimization target according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix; Optimization solution module: used for solving the optimization target to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization target; Aggregation update module: used for obtaining the updated first low-rank matrix and the second low-rank matrix after the server aggregates the first solution matrix and the second solution matrix of each client, updating the optimization target according to the updated first low-rank matrix and the second low-rank matrix, iteratively updating the first solution matrix and the second solution matrix until the iteration termination condition is reached, obtaining the trained large language model according to the first weight matrix, and updating the trained large language model in each client; The obtaining of the optimization target according to the first weight matrix, the second weight matrix, the first fine-tuning matrix and the second fine-tuning matrix includes: Obtaining a first fine-tuning matrix and a second fine-tuning matrix based on the second weight matrix, obtaining a matrix product of the first fine-tuning matrix and the second fine-tuning matrix, and obtaining an optimized target matrix; Obtaining a quantization difference matrix according to the sum of the optimization target matrix and the second weight matrix; Subtracting the quantization difference matrix from the first weight matrix to obtain the optimization target; The step of solving the optimization objective to obtain a first solution matrix of the first fine-tuning matrix and a second solution matrix of the second fine-tuning matrix that minimize the optimization objective includes: Obtaining a first estimated value of the first fine-tuning matrix and a second estimated value of the second fine-tuning matrix at a previous iteration step; Substituting the first estimated value and the second estimated value into the quantization difference matrix to obtain a quantization estimation matrix of a current iteration step; Obtaining a residual matrix according to the difference between the first weight matrix and the quantization estimation matrix, performing singular value decomposition on the residual matrix to obtain singular values and left singular vectors and right singular vectors corresponding to the singular values; Obtain the first estimated value and the second estimated value of the current iteration step number according to the singular value, the left singular vector and the right singular vector; The iterative process is repeated until a preset number of iteration steps is reached, the first estimated value is obtained as the first solution matrix, and the second estimated value is obtained as the second solution matrix.
9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the distributed large language model quantization deployment method according to any one of claims 1 to 7 when executing the computer program.
10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the distributed large language model quantization deployment method according to any one of claims 1 to 7 is implemented.
