A federated fine-tuning method, apparatus, and related products based on interval compression

Abstract

This invention discloses a federated fine-tuning method, apparatus, and related products based on interval compression. The method includes: dividing the original gradient data according to target partition parameters to obtain partition gradient data corresponding to at least two partition position codes; for each partition position code, performing sparsification processing on the partition gradient data corresponding to the partition position code to obtain sparse position codes and sparse gradient data, and quantizing the sparse gradient data to obtain target gradient data corresponding to the partition position code; determining target transmission data based on at least two partition position codes and the sparse position codes and target gradient data corresponding to each partition position code; and sending the target transmission data to a server so that the server can fine-tune the general pre-trained model based on the target transmission data received from at least two clients, thereby improving the data compression effect in the federated fine-tuning process.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a federated fine-tuning method, apparatus and related products based on interval compression. Background Technology

[0002] With the rapid development of artificial intelligence technology, general pre-trained models are now massive in scale, have numerous parameters, and possess powerful learning and reasoning capabilities, demonstrating unprecedented capabilities in many fields.

[0003] Fine-tuning a general pre-trained model requires significant computational resources and data. Federated Learning (FL), as a distributed learning approach, can utilize local data distributed across various clients to fine-tune the general pre-trained model. Specifically, after obtaining the update gradients of the general pre-trained model using local data, the client sends these update gradients to the server. The server then uses the aggregated update gradients to fine-tune the general pre-trained model.

[0004] The federated fine-tuning framework involves frequent communication between the client and the server. The large scale of the general pre-trained model makes the drawbacks of the traditional data compression algorithm used in the data transmission process increasingly obvious, and the compression effect is not ideal, resulting in low efficiency of federated fine-tuning of the general pre-trained model. Summary of the Invention

[0005] This invention provides a federated fine-tuning method, apparatus, and related products based on interval compression to solve the problem of poor compression performance of traditional compression algorithms when faced with large data volumes, save communication resources occupied by data transmission processes, and improve the federated fine-tuning efficiency of general pre-trained models.

[0006] According to one embodiment of the present invention, a federated fine-tuning method based on interval compression is provided, the method comprising:

[0007] Based on the target partitioning parameters, the original gradient data is divided to obtain at least two partitioning location codes corresponding to the partition gradient data.

[0008] For each partition location code, the partition gradient data corresponding to the partition location code is sparsified to obtain sparse location code and sparse gradient data, and the sparse gradient data is quantized to obtain the target gradient data corresponding to the partition location code.

[0009] The target transmission data is determined based on the at least two partition location codes, the sparse location codes corresponding to each partition location code, and the target gradient data.

[0010] The target transmission data is sent to the server so that the server can fine-tune the general pre-trained model based on the target transmission data received from at least two clients.

[0011] According to another embodiment of the present invention, a federated fine-tuning device based on interval compression is provided, the device comprising:

[0012] The original gradient data partitioning module is used to partition the original gradient data according to the target partitioning parameters to obtain partition gradient data corresponding to at least two partition position codes.

[0013] The target gradient data determination module is used to perform sparsification processing on the partition gradient data corresponding to the partition position code for each partition position code to obtain sparse position code and sparse gradient data, and to perform quantization processing on the sparse gradient data to obtain the target gradient data corresponding to the partition position code.

[0014] The target transmission data determination module is used to determine the target transmission data based on the at least two partition position codes, the sparse position codes corresponding to each partition position code, and the target gradient data.

[0015] The target transmission data sending module is used to send the target transmission data to the server so that the server can fine-tune the general pre-trained model based on the target transmission data sent by at least two clients respectively.

[0016] According to another embodiment of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the federated fine-tuning method based on interval compression as described in any embodiment of the present invention.

[0020] According to another embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the federated fine-tuning method based on interval compression as described in any embodiment of the present invention.

[0021] According to another embodiment of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the federated fine-tuning method based on interval compression as described in any embodiment of the present invention.

[0022] The technical solution of this invention divides the original gradient data into multiple partition gradient data according to the target partition parameters, and performs sparsification and quantization processing on each partition gradient data in sequence. This reduces the position encoding cost of traditional sparsification algorithms as the amount of data increases, and solves the problem that traditional quantization algorithms cannot provide sufficient compression ratio as the amount of data increases. It saves the communication resources occupied by the data transmission process and improves the efficiency of federated fine-tuning of general pre-trained models.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a federated fine-tuning method based on interval compression, provided as an embodiment of the present invention;

[0026] Figure 2 A flowchart illustrating another federated fine-tuning method based on interval compression provided in one embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram illustrating a specific example of a federated fine-tuning method based on interval compression, provided as an embodiment of the present invention.

[0028] Figure 4 A flowchart illustrating another federated fine-tuning method based on interval compression provided in one embodiment of the present invention;

[0029] Figure 5 A schematic diagram of a federated fine-tuning device based on interval compression provided in one embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0031] Quantization algorithms compress data by converting the high-precision representation of gradient values ​​in gradient data into a low-precision representation. However, quantization algorithms have limited effectiveness in achieving high compression rates, especially when dealing with the gradient data of large-scale general pre-trained models, where they cannot provide sufficiently high compression ratios.

[0032] Taking the example of each gradient value in the gradient data being represented by 32 bits, and the quantization algorithm reducing it to y bits, the compression ratio Λ is defined as the ratio of the number of bits used to represent the original gradient value to the number of bits used to represent the compressed gradient value. The compression ratio Λ of the quantization algorithm... Q =32 / y. According to existing research, when the number of bits used to represent the gradient value is less than 6, the performance of general pre-trained models will significantly decrease. Therefore, the upper limit of the compression ratio of quantization algorithms is approximately Λ. Q =32 / 6≈5.4. Therefore, when faced with a large gradient scale, the compression ratio of the quantization algorithm is limited, making it difficult to simultaneously meet the compression requirements and the performance requirements of general pre-trained models.

[0033] Sparsity algorithms compress data by retaining the key gradient values ​​and removing the rest, thus reducing the number of gradients that need to be transmitted. The TopK algorithm is a common example; specifically, it aims to retain the K largest absolute gradient values ​​in the gradient data. However, when applied, sparsity algorithms generate positional codes to mark the location of the retained gradient values ​​within the gradient data. As the number of gradients increases, the representation cost of these positional codes also increases, limiting the compression effectiveness of sparsity methods.

[0034] Taking gradient data with d gradients as an example, the TopK algorithm only retains the K gradient values ​​with the largest absolute values. Ignoring the cost of positional encoding, the compression ratio Λ of the TopK algorithm is... S =d / K, this compression ratio is far higher than that of quantization algorithms. Unlike quantization algorithms, which directly transmit all gradient values, sparsification algorithms need to transmit both the retained gradient values ​​and the positional encoding of each gradient value. The advantage of sparsification algorithms is weakened by the additional positional encoding. Specifically, for gradient data with d gradients, the positional encoding in sparsification algorithms requires at least... Bit representation, therefore, considering the cost of positional encoding, the compression ratio of the TopK algorithm is... Where y represents the number of bits used to represent the gradient value.

[0035] When dealing with the massive gradient scale of a general pre-trained model, the number of gradients, d, is extremely large, resulting in a very low bit depth for positional encoding. There is a risk of exceeding the number of bits y in the gradient value representation. For example, suppose y = 16, when the number of gradients d = 10...6 At this point, the positional encoding uses 20 bits. Therefore, regardless of how K is adjusted, the communication efficiency of the TopK algorithm will be affected.

[0036] In summary, when faced with the massive gradient scale of general pre-trained models, quantization algorithms, while simple, have limited compression ratios, while sparsification algorithms, due to the additional positional encoding, may actually increase the amount of data transmitted, resulting in low communication efficiency.

[0037] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0038] It should be noted that the terms "first," "second," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0039] Figure 1 This is a flowchart illustrating a federated fine-tuning method based on interval compression, provided as an embodiment of the present invention. This embodiment is applicable to situations where a federated fine-tuning framework is used to fine-tune a general pre-trained model. The method can be executed by a federated fine-tuning device based on interval compression, which can be implemented in hardware and / or software and can be configured in a terminal device. Figure 1 As shown, the method includes:

[0040] S110. Based on the target partitioning parameters, the original gradient data is divided to obtain at least two partitioning location codes corresponding to the partitioning gradient data.

[0041] Specifically, the target partitioning parameter is used to characterize the set value under a specified partitioning dimension. In an optional embodiment, the target partitioning parameter is the number of target partitions, the number of target gradients, or at least two gradient start and end ranges.

[0042] The number of target partitions refers to the number of gradient data in each partition. For example, if the number of gradients corresponding to the original gradient data is 100, then when the number of target partitions is 20, the number of gradients corresponding to the gradient data in each partition is 5.

[0043] The target gradient number is the number of gradients corresponding to the partition gradient data, which represents the number of gradient values ​​contained in the partition gradient data. For example, assuming that the original gradient data has 100 gradients, when the target gradient number is 4, the number of partition gradient data is 25.

[0044] The gradient start and end ranges include the start and end position encodings of the gradient values ​​in the original gradient data. For example, assuming that the number of gradients corresponding to the original gradient data is 100, the two gradient start and end ranges included in the target partition parameters can be [0,59] and [60,99], respectively.

[0045] In one specific embodiment, the target partitioning parameters can be customized according to actual needs, and the number of gradients corresponding to different partition gradient data can be the same or different.

[0046] Specifically, the partition location code is used to uniquely identify the partition gradient data. For example, the partition location code can be an index-based location code or a hash-based location code. The encoding form used for the partition location code is not limited here.

[0047] S120. For each partition location code, the partition gradient data corresponding to the partition location code is sparsified to obtain sparse location code and sparse gradient data. The sparse gradient data is then quantized to obtain the target gradient data corresponding to the partition location code.

[0048] In one alternative embodiment, the sparsity algorithms used for encoding different partition positions can be the same or different. Exemplary examples include the TopK algorithm and a thresholding algorithm, but are not limited to the example scenario.

[0049] Specifically, the sparse gradient data contains at least one gradient value retained after the partitioned gradient data has been sparsified. The sparse location encoding contains at least one gradient location code, which is used to mark the position information of the gradient value in the sparse gradient data within the partitioned gradient data. The gradient location code in the sparse location encoding corresponds one-to-one with the gradient value in the sparse gradient data.

[0050] For example, suppose the partition gradient data is [10,20,1,2,3,4,5,30], and the sparsity K=3 for the TopK algorithm, then the sparse gradient data is [10,20,30], and the sparse position encoding is [0,1,7].

[0051] In one optional embodiment, the quantization algorithms used for encoding different partition positions can be the same or different. Exemplary quantization algorithms include scalar quantization, vector quantization, product quantization (PQ), and quantized stochastic gradient descent (QSGD), but are not limited to the example scenario.

[0052] Specifically, the gradient position encoding in sparse positional encoding corresponds one-to-one with the gradient value in the target gradient data.

[0053] In traditional sparsity algorithms, the retained gradient values ​​are selected from the global scope of the original gradient data, which requires the positional encoding to cover the entire range of the gradient vector. This embodiment significantly reduces the positional encoding requirements for each partition of the gradient data by dividing the original gradient data into multiple partitions and applying the sparsity algorithm independently within each partition. Specifically, taking original gradient data with d gradients as an example, this embodiment divides it into q partitions, where the i-th partition contains b... i There are gradient values, and they satisfy... Because the number of gradients in each partition's gradient data is b i Typically much smaller than the number of gradients d in the original gradient data, therefore the number of bits used to represent the gradient position encoding is... The number of bits required for positional encoding is less than that required in traditional sparsity algorithms. This reduces the cost of positional encoding of gradient values ​​in traditional sparsification algorithms.

[0054] S130. Determine the target transmission data based on at least two partition location codes, the sparse location codes corresponding to each partition location code, and the target gradient data.

[0055] In one optional embodiment, determining the target transmission data based on at least two partition location codes and the sparse location code and target gradient data corresponding to each partition location code includes: for each partition location code, encapsulating the partition location code, the sparse location code and target gradient data corresponding to the partition location code in the data payload of the data packet; and generating the target transmission data based on the data packet corresponding to each partition location code.

[0056] Specifically, in the data payload of the data packet in this embodiment, the partition location encoding, gradient location encoding, and gradient value correspond one-to-one.

[0057] For example, assuming the target gradient data contains 3 gradient values, and correspondingly the sparse location coding contains 3 gradient location codes, then the data payload of the data packet contains 3 data groups, each containing a partition location code, a gradient location code, and a gradient value.

[0058] In another optional embodiment, the target transmission data is determined based on at least two partition location codes and the sparse location code and target gradient data corresponding to each partition location code, including: for each partition location code, encapsulating the partition location code in the data packet header and encapsulating the sparse location code and target gradient data corresponding to the partition location code in the data payload of the data packet; and generating the target transmission data based on the data packet corresponding to each partition location code.

[0059] Specifically, in this embodiment, the gradient position code and gradient value correspond one-to-one in the data payload of the data packet. Taking the example above, the data packet header contains the partition position code, and the data payload contains three data groups, each containing the gradient position code and gradient value.

[0060] The advantage of this setup is that it avoids adding partition location encoding to each gradient value in the target gradient data, thereby significantly reducing the additional overhead of data packets and further improving data transmission efficiency.

[0061] In one optional embodiment, the partition location code is encapsulated in the packet header of the data packet, and the sparse location code corresponding to the partition location code and the target gradient data are encapsulated in the data payload of the data packet, including: obtaining the data packet capacity corresponding to the data packet; rounding up the ratio between the occupied capacity corresponding to the sparse location code and the target gradient data and the data packet capacity as the partition data packet traffic; encapsulating the partition location code in the packet header of the data packet of the partition data packet traffic, and encapsulating the sparse location code and the target gradient data in the data payload of the data packet of the partition data packet traffic according to the data packet capacity.

[0062] Specifically, packet capacity refers to the amount of data that the data payload of a packet can encapsulate, and partition packet flow refers to the number of packets allocated to partition location encoding.

[0063] For example, suppose the number of gradients in the gradient data of the i-th partition is b. i There are K gradients in the target gradient data. i If the gradient value in the target gradient data has y′ bits, then the gradient position encoding has y′ bits. The capacity occupied by the bit, sparse position encoding and target gradient data If we represent the packet size as P, then the partition packet traffic Correspondingly, (r i The gradient allocation amounts for -1) data packets are P, and the gradient allocation amount for 1 data packet is M. i -P(r i -1). Here, gradient allocation represents the number of gradient values ​​encapsulated in the data packet.

[0064] S140. Send the target transmission data to the server so that the server can fine-tune the general pre-trained model based on the target transmission data sent by at least two clients.

[0065] For example, general pre-trained models include Large Language Models (LLMs), general models for the visual domain, general models for the multimodal domain, or general models for the audio domain, but are not limited to the example cases.

[0066] In one alternative embodiment, the server fine-tunes the general pre-trained model within a federated fine-tuning framework, using Low-Rank Adaptation (LoRA) technology.

[0067] In the federated fine-tuning framework, multiple clients collaboratively train a shared, general pre-trained model while maintaining data privacy. Each client uses local data to fine-tune the low-rank matrix in the LoRA technique without updating the entire general pre-trained model. Specifically, the LoRA technique significantly reduces the number of gradients that need to be updated and the computational overhead by introducing two low-rank matrices to represent the update gradients of the general pre-trained model.

[0068] The fine-tuning process of a general pre-trained model based on LoRA technology includes four stages: parameter distribution, local update, parameter aggregation, and iterative iteration. In the parameter distribution stage, during each iteration of fine-tuning training, the server distributes the current global model data of the general pre-trained model (including the low-rank matrix provided by LoRA technology) to all clients participating in the training. All clients use the same general pre-trained model as their initial model.

[0069] During the local update phase, each client uses local data to fine-tune the low-rank matrices. Specifically, the client only adjusts the two low-rank matrices introduced by the LoRA technique, without updating the complete model parameters of the general pre-trained model. The updated low-rank matrices represent the gradient data of the general pre-trained model determined by the client based on the local data. After several rounds of local training, each client uploads the calculated gradient data to the server. Specifically, the gradient data transmission method adopts S110-S130 in this embodiment.

[0070] During the model aggregation phase, after receiving gradient data from all clients, the server performs an aggregation operation to obtain global gradient data. For example, the server averages the gradient data uploaded by all clients and uses the resulting average gradient data to fine-tune the general pre-trained model. The fine-tuned global model data is then redistributed to each client for the next iteration.

[0071] During the iterative phase, the above steps will be repeated in multiple training rounds until the performance of the general pre-trained model reaches the expected target.

[0072] LoRA technology ensures that the amount of gradients uploaded from the client to the server in each iteration is only 0.1% to 1% of the total model parameters of a general pre-trained model.

[0073] The technical solution of this embodiment divides the original gradient data into multiple partition gradient data according to the target partition parameters, and performs sparsification and quantization processing on each partition gradient data in sequence. This not only reduces the position encoding cost caused by the traditional sparsification algorithm as the amount of data increases, but also solves the problem that the traditional quantization algorithm cannot provide sufficient compression ratio as the amount of data increases. This saves the communication resources occupied by the data transmission process and improves the efficiency of federated fine-tuning of the general pre-trained model.

[0074] Figure 2This is a flowchart of another federated fine-tuning method based on interval compression provided in one embodiment of the present invention. This embodiment further refines the "determining the target transmission data based on at least two partition location codes and the sparse location codes and target gradient data corresponding to each partition location code" in the above embodiment. In this embodiment, determining the target transmission data based on at least two partition location codes and the sparse location codes and target gradient data corresponding to each partition location code includes: obtaining the maximum data packet traffic of the communication device corresponding to the client; using the ratio between the maximum data packet traffic and the number of partitions of the partition location codes as the partition data packet traffic corresponding to each partition location code; for each partition location code, determining the data packet of the partition data packet traffic based on the partition location code and the sparse location codes and target gradient data corresponding to the partition location code; and generating the target transmission data based on the data packet of the maximum data packet traffic. Figure 2 As shown, the method includes:

[0075] S210. Based on the target partitioning parameters, the original gradient data is divided to obtain at least two partitioning location codes corresponding to the partitioning gradient data.

[0076] S220. For each partition location code, the partition gradient data corresponding to the partition location code is sparsified to obtain sparse location code and sparse gradient data. The sparse gradient data is then quantized to obtain the target gradient data corresponding to the partition location code.

[0077] S210-S220 in this embodiment are the same as those in the above embodiment. Figure 1 The S110-S120 shown are the same or similar, and will not be described again in this embodiment.

[0078] S230. Obtain the maximum data packet traffic of the communication device corresponding to the client.

[0079] This embodiment is more applicable to situations where the number of gradients corresponding to at least two target gradient data points is similar. Specifically, the difference in the number of gradients corresponding to each pair of target gradient data points is less than a difference threshold. For example, the difference threshold can be 20, but it is not limited to the example scenario.

[0080] Specifically, maximum data packet flow represents the maximum number of data packets that a communication device can upload to a server in a single instance.

[0081] S240. The ratio between the maximum data packet traffic and the number of partitions in the partition location code is used as the partition data packet traffic corresponding to each partition location code.

[0082] For example, if the maximum packet traffic is represented by R and the number of partitions corresponding to the partition location code is q, then the partition packet traffic r = R / q.

[0083] S250. For each partition location code, determine the data packets of the partition data packet traffic based on the partition location code, the sparse location code corresponding to the partition location code, and the target gradient data.

[0084] In an optional embodiment, determining the data packet of the partitioned data packet traffic based on the partition location code and the sparse location code and target gradient data corresponding to the partition location code includes: encapsulating the partition location code and the sparse location code and target gradient data corresponding to the partition location code in the data payload of the data packet of the partitioned data packet traffic.

[0085] Specifically, in the data payload of the data packet in this embodiment, the partition location encoding, gradient location encoding, and gradient value correspond one-to-one.

[0086] In another optional embodiment, determining the data packet of the partitioned data packet traffic based on the partition location code and the sparse location code and target gradient data corresponding to the partition location code includes: encapsulating the partition location code in the data packet header of the partitioned data packet traffic; and randomly encapsulating the sparse location code and target gradient data in the data payload of the partitioned data packet traffic.

[0087] In another optional embodiment, determining the data packets of the partitioned data packet traffic based on the partition location code, the sparse location code corresponding to the partition location code, and the target gradient data includes: encapsulating the partition location code in the data packet header of the partitioned data packet traffic; using the ratio between the number of gradients in the target gradient data and the partitioned data packet traffic as the gradient allocation amount corresponding to the data packet; wherein, the gradient allocation amount represents the number of gradient values ​​encapsulated in each data packet; and encapsulating the sparse location code and the target gradient data in the data payload of the partitioned data packet traffic according to the gradient allocation amount.

[0088] Specifically, in the data payload of the data packet in this embodiment, the gradient position encoding and gradient value correspond one-to-one, and the gradient number of the target gradient data represents the number of gradient values ​​contained in the target gradient data.

[0089] For example, the number of gradients for the i-th target gradient data is k. i This indicates that the gradient allocation amount k for each data packet corresponding to the i-th partition location encoding is... i，c =k i / r.

[0090] S260. Generate target transmission data based on the data packet with the maximum data packet flow.

[0091] Figure 3This is a schematic diagram illustrating a specific example of a federated fine-tuning method based on interval compression, provided by an embodiment of the present invention. Specifically, the position encoding of the original gradient data with gradient number d has a representation bit depth of d bits. The gradient value is represented by y bits. The original gradient data is divided into q partitions, each with b gradient values. The corresponding partition location encoding is represented by y bits. In sparse positional coding, the number of bits used to represent each gradient position is 1. The target gradient data contains K gradient values.

[0092] During the gradient intervalization stage, the compression ratio This can be expressed as the following formula:

[0093]

[0094] During the gradient compression phase, the compression ratio This can be expressed as the following formula:

[0095]

[0096] Where y′ represents the number of bits used to represent the gradient values ​​in the target gradient data.

[0097] During the gradient packetization stage, because the partition location encoding is encapsulated in the packet header, the compression ratio is... This can be expressed as the following formula:

[0098]

[0099] S270. Send the target transmission data to the server so that the server can fine-tune the general pre-trained model based on the target transmission data sent by at least two clients.

[0100] S270 in this embodiment is the same as that in the above embodiment. Figure 1 The S140 shown is the same or similar, and will not be described again in this embodiment.

[0101] The technical solution of this embodiment, by uniformly distributing data packets to each partition location and uniformly encapsulating the gradient values ​​in the target gradient data within the data packets, avoids situations where the data packets are too large or too small under the constraint of fixed communication traffic. This ensures the stability of the data transmission process and the server-side data processing process, and also avoids the differences in server-side processing logic caused by inconsistent data packet sizes.

[0102] Figure 4This is a flowchart illustrating another federated fine-tuning method based on interval compression provided in one embodiment of the present invention. This embodiment further refines the federated fine-tuning method based on interval compression in the above embodiment. Figure 4 As shown, the method includes:

[0103] S310. Using the total compression error of the original gradient data as the optimization objective, construct the objective optimization function.

[0104] In this embodiment, the target partitioning parameter is the number of target partitions, and the general pre-trained model is a large language model. Specifically, both the global gradient vector and the interval-based gradient vector of the large language model satisfy the Gaussian distribution characteristic. Therefore, the gradient vector corresponding to the t-th fine-tuning training can be modeled as having zero mean and a standard deviation of σ. t The Gaussian distribution.

[0105] Taking the TopK algorithm as an example, the original gradient data u corresponding to the t-th fine-tuning training is... t The l-th gradient value in the absolute value descending sequence satisfies the following relationship:

[0106]

[0107] Among them, erf -1 Let d represent the inverse function of the error function, and let d represent the original gradient data u. t The number of gradients, l∈{1,2,...,d}.

[0108] Correspondingly, the original gradient data u t After the gradient data is divided into multiple partitions, for the gradient data u of the i-th partition... t，i The l-th gradient value in the absolute value sequence satisfies the following relationship:

[0109]

[0110] Among them, b i Represents the partition gradient data u t，i The number of gradients, l∈{1, 2, ..., b} i}

[0111] In this embodiment, the objective optimization function represents the sum of the partition compression errors corresponding to the at least two partition location codes. The partition compression error represents the compression error of the objective gradient data relative to the partition gradient data, and the partition compression error includes sparsity error and quantization error.

[0112] Specifically, sparsification introduces sparsity error by retaining only the K gradient values ​​with the largest absolute values ​​in the partitioned gradient data and discarding the rest. Quantization introduces quantization error due to the loss of precision in the gradient values.

[0113] In an optional embodiment, the objective optimization function satisfies the following formula:

[0114]

[0115] Among them, Γ t Let q represent the objective optimization function corresponding to the t-th fine-tuning training of the client and the large language model, and let γ represent the number of partitions. t，i G represents the partition compression error corresponding to the position encoding of the i-th partition under the t-th fine-tuning. S G represents the sparsification error. Q Indicates quantization error, u t，i Let u represent the gradient data of the i-th partition under the t-th fine-tuning. t，i {l} represents the partition gradient data u t，i The corresponding l-th gradient value in the descending absolute value sequence, ζ represents the sparse gradient data corresponding to the i-th partition location encoding. i This represents the error parameter introduced by the quantization algorithm corresponding to the i-th partition position encoding.

[0116] Specifically, the total compression error satisfies the following relationship:

[0117]

[0118] in, This represents all the target gradient data corresponding to the t-th fine-tuning training. This represents the target gradient data under the t-th fine-tuning. This represents the expected function.

[0119] In an optional embodiment, when the quantization process uses the PQ algorithm, the error parameter ζ corresponding to the c-th data packet under the i-th partition location encoding is... i，c Satisfy the following formula:

[0120]

[0121] Where, k i，c b represents the gradient allocation amount of the c-th data packet corresponding to the i-th partition location code. i y' represents the number of gradients corresponding to the gradient data of the i-th partition, y′ represents the number of bits required to represent the gradient values ​​in the target gradient data, and P represents the data packet capacity.

[0122] Accordingly, in this embodiment, the partition compression error γ t，i Satisfy the following formula:

[0123]

[0124] Where, r i This represents the partition data packet traffic corresponding to the i-th partition location code.

[0125] S320. Determine the constraints based on the maximum data packet traffic and data packet capacity of the communication device corresponding to the client.

[0126] In this embodiment, the constraints include that the gradient allocation amount of each data packet under the partition location encoding is the same, the occupied capacity corresponding to the gradient allocation amount is less than or equal to the data packet capacity, the number of gradients corresponding to each partition gradient data is the same, and the partition data packet traffic corresponding to each partition location encoding is the same.

[0127] In one optional embodiment, the constraints include:

[0128] k i，c (log2b i +y′)≤P,b i =d / q, r i =R / q,k i，c =k i / r i

[0129] Where, k i，c b represents the gradient allocation amount of the c-th data packet corresponding to the i-th partition position encoding in the gradient allocation matrix k. i This represents the number of gradients corresponding to the gradient data of the i-th partition. y′ represents the number of bits required to represent the gradient values ​​in the target gradient data, P represents the data packet size, d represents the number of gradients corresponding to the original gradient data, q represents the number of partitions, and k i This represents the sparseness of the partition corresponding to the i-th partition position code, 1≤k i ≤b i r i This represents the partition packet traffic corresponding to the i-th partition location code, 1≤r i ≤R, where R represents the maximum data packet flow.

[0130] Specifically, k i，c Satisfying Relationships Where, 1≤k i，c ≤k i .

[0131] S330. Under the condition of satisfying the constraints, the objective optimization function is minimized to obtain the number of objective partitions and the objective gradient assignment matrix.

[0132] Specifically, based on the objective function and constraints, the minimization solution can be expressed as follows: form:

[0133]

[0134] In an optional embodiment, minimizing the objective optimization function to obtain the target number of partitions and the target gradient assignment matrix includes: in each iteration, substituting the gradient assignment matrix from the previous iteration into the objective optimization function to obtain a first optimization function, and minimizing the first optimization function to obtain the number of partitions in the current iteration; substituting the number of partitions in the current iteration into the objective optimization function to obtain a second optimization function, and minimizing each partition optimization function corresponding to the second optimization function to obtain the gradient assignment matrix in the current iteration; wherein, the partition optimization function characterizes the partition compression error; until the objective optimization function converges, the number of partitions in the current iteration is taken as the target number of partitions, and the gradient assignment matrix in the current iteration is taken as the target gradient assignment matrix.

[0135] Theoretical analysis proves that, The problem is a non-convex optimization problem. Therefore, the minimization process is divided into problem decomposition and alternating solution. Specifically, the minimization process will be... It can be formally decomposed into a convex optimization problem with respect to the number of partitions q. Form and convex optimization problem with respect to gradient assignment matrix k form.

[0136] in, Formal characterization, with a fixed gradient assignment matrix k, The objective function in the form can be transformed into the first optimization function, which optimizes the number of partitions q to achieve the desired result. The first optimization function in the form is minimized. For example, The form can be represented as follows:

[0137]

[0138] For example, the solution method used to minimize the first optimization function can be gradient descent, but it is not limited to the example case.

[0139] in, Formal representation under a fixed number of partitions q, The objective function in the form can be transformed into a second optimization function, which is achieved by optimizing the gradient assignment matrix k. The second optimization function in the form is minimized. Since k i and k i，cIt only affects the compression error of the i-th partition and is independent of the compression errors of other partitions. Therefore, the optimization of the gradient assignment matrix k can be performed independently for each partition's gradient data. For example, with q = q * hour, The form can be represented as follows:

[0140] in, This represents the maximum gradient allocation amount for the data packet.

[0141] For the quantization error G in the partition optimization function Q (k i ), can be represented as:

[0142]

[0143] in, This represents the set of gradient values ​​assigned to the c-th data packet corresponding to the i-th partition location code in the target gradient data. This represents the quantization error of the c-th data packet corresponding to the i-th partition location encoding.

[0144] For example, the sequential minimum optimization algorithm is used to iteratively adjust the gradient allocation of data packets to obtain the minimum quantization error G. Q (k i Specifically, in two consecutive data packets, the c-th and (c-1)-th, when k i，c +k i，c-1 When =X, By adjusting k i，c To minimize quantization error.

[0145] S340. Based on the target gradient allocation matrix, determine the sparsity of each partition location encoding corresponding to the sparsification process.

[0146] Specifically, for each partition location code, the sum of the gradient assignments of the r data packets corresponding to the partition location code in the target gradient assignment matrix is ​​used as the partition sparsity corresponding to the sparsification process of the partition location code.

[0147] In this embodiment, a gradient compression scheme is obtained by minimizing the objective optimization function. Specifically, the gradient compression scheme includes the number of objective partitions and the partition sparsity corresponding to the position encoding of each partition.

[0148] S350. Based on the number of target partitions, divide the original gradient data to obtain at least two partition position codes corresponding to the partition gradient data.

[0149] For example, for the original gradient data with a gradient number of d, when the number of target partitions is q, the gradient number corresponding to the partition gradient data is b = d / q.

[0150] S360. For each partition location code, according to the partition sparsity corresponding to the partition location code, the partition gradient data corresponding to the partition location code is sparsified to obtain sparse location code and sparse gradient data. The sparse gradient data is then quantized to obtain the target gradient data corresponding to the partition location code.

[0151] For example, when the partition sparsity corresponding to the i-th partition position code is k i When the sparse position encoding, sparse gradient data, and target gradient data each correspond to a data size of k, the sparse position encoding, sparse gradient data, and target gradient data are all k. i .

[0152] S370. Determine the target transmission data based on at least two partition location codes, the sparse location codes corresponding to each partition location code, and the target gradient data.

[0153] S380. Send the target transmission data to the server so that the server can fine-tune the large language model based on the target transmission data sent by at least two clients.

[0154] S370-S380 in this embodiment are the same as those in the above embodiments. Figure 1 S130-S140 shown are the same as or similar to those in the above embodiments. Figure 2 The S230-S270 shown are the same or similar, and will not be described again in this embodiment.

[0155] The number of gradients and the sparsity of the partitioned gradient data are crucial. If the number of gradients is too large, the cost of positional encoding will still be high; if the gradient data is too small, it will be difficult to sparsify the gradients sufficiently. If the partition sparsity is too large, the compression effect will be poor; if the partition sparsity is too small, it will easily affect the model performance of large language models.

[0156] The technical solution of this embodiment is based on the fact that the gradient vector of the large language model conforms to the Gaussian distribution characteristic. The total compression error of the original gradient data is used as the optimization objective to construct the objective optimization function. The constraints are determined according to the maximum data packet traffic and data packet capacity of the communication device corresponding to the client. Under the condition of satisfying the constraints, the objective optimization function is minimized to obtain the gradient compression scheme. This achieves a balance between sparsity effect and quantization effect, and at the same time, a balance between compression effect and model performance of the large language model.

[0157] The above embodiments were extensively tested on public datasets. The experimental results show that, compared with the federated fine-tuning method using traditional data compression algorithms, the large language model obtained by the federated fine-tuning method based on interval compression provided in this embodiment improves the accuracy by 6.42% to 18.87% and reduces communication traffic by 17.07% to 44.44%, achieving the goal of simultaneously improving the fine-tuning effect of the large language model and the compression effect in the communication process.

[0158] The following are embodiments of the federated fine-tuning device based on interval compression provided in this invention. This device and the federated fine-tuning method based on interval compression in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the federated fine-tuning device based on interval compression, please refer to the content of the federated fine-tuning method based on interval compression in the above embodiments.

[0159] Figure 5 This is a schematic diagram of a federated fine-tuning device based on interval compression, provided as an embodiment of the present invention. Figure 5 As shown, the device includes: a raw gradient data partitioning module 410, a target gradient data determination module 420, a target transmission data determination module 430, and a target transmission data sending module 440.

[0160] The original gradient data partitioning module 410 is used to partition the original gradient data according to the target partitioning parameters to obtain partition gradient data corresponding to at least two partition position codes respectively.

[0161] The target gradient data determination module 420 is used to perform sparsification processing on the partition gradient data corresponding to the partition position code for each partition position code to obtain sparse position code and sparse gradient data, and to perform quantization processing on the sparse gradient data to obtain the target gradient data corresponding to the partition position code.

[0162] The target transmission data determination module 430 is used to determine the target transmission data based on at least two partition position codes and the sparse position code and target gradient data corresponding to each partition position code;

[0163] The target transmission data sending module 440 is used to send the target transmission data to the server so that the server can fine-tune the general pre-trained model based on the target transmission data sent by at least two clients.

[0164] The technical solution of this embodiment divides the original gradient data into multiple partition gradient data according to the target partition parameters, and performs sparsification and quantization processing on each partition gradient data in sequence. This not only reduces the position encoding cost caused by the traditional sparsification algorithm as the amount of data increases, but also solves the problem that the traditional quantization algorithm cannot provide sufficient compression ratio as the amount of data increases. This saves the communication resources occupied by the data transmission process and improves the efficiency of federated fine-tuning of the general pre-trained model.

[0165] In an optional embodiment, the target transmission data determination module 430 includes:

[0166] The maximum data packet traffic acquisition unit is used to acquire the maximum data packet traffic of the communication device corresponding to the client.

[0167] The partition packet traffic determination unit is used to take the ratio between the maximum packet traffic and the number of partitions for each partition location code as the partition packet traffic corresponding to each partition location code;

[0168] The packet determination unit is used to determine the packet of the partition packet traffic based on the partition location code, the sparse location code corresponding to the partition location code, and the target gradient data for each partition location code.

[0169] The target transmission data generation unit is used to generate target transmission data based on the data packets with the maximum data packet flow.

[0170] In one optional embodiment, the data packet determining unit is specifically used for:

[0171] Encapsulate the partition location code in the header of the partition packet traffic packets;

[0172] The ratio between the number of gradients in the target gradient data and the traffic of the partitioned data packets is used as the gradient allocation amount for the corresponding data packet; where the gradient allocation amount represents the number of gradient values ​​encapsulated in each data packet.

[0173] Based on the gradient allocation, the sparse location encoding and target gradient data are encapsulated in the data payload of the partitioned packet traffic.

[0174] In an optional embodiment, the target partitioning parameter is the number of target partitions, the general pre-trained model is a large language model, and correspondingly, the device further includes:

[0175] The objective optimization function construction module is used to construct the objective optimization function by taking the total compression error of the original gradient data as the optimization objective.

[0176] The constraint determination module is used to determine the constraints based on the maximum data packet traffic and data packet capacity of the communication device corresponding to the client.

[0177] The objective optimization function solution module is used to minimize the objective optimization function under the constraints to obtain the number of objective partitions and the objective gradient assignment matrix; wherein, the matrix parameter k in the objective gradient assignment matrix... i，c The gradient allocation represents the number of gradient values ​​encapsulated in the c-th data packet corresponding to the i-th partition location encoding.

[0178] The partition sparsity determination module is used to determine the partition sparsity corresponding to the sparsification processing of each partition position encoding based on the target gradient allocation matrix.

[0179] The objective optimization function represents the sum of the partition compression errors corresponding to at least two partition location codes. The partition compression error represents the compression error of the objective gradient data relative to the partition gradient data. The partition compression error includes sparsity error and quantization error. The constraints include that the gradient allocation of each data packet under the partition location code is the same, the occupied capacity corresponding to the gradient allocation is less than or equal to the data packet capacity, the number of gradients corresponding to each partition gradient data is the same, and the partition data packet traffic corresponding to each partition location code is the same.

[0180] In an optional embodiment, the objective optimization function satisfies the following formula:

[0181]

[0182] Among them, Γ t Let q represent the objective optimization function corresponding to the t-th fine-tuning training of the client and the large language model, and let γ represent the number of partitions. t，i G represents the partition compression error corresponding to the position encoding of the i-th partition under the t-th fine-tuning. S G represents the sparsification error. Q Indicates quantization error, u t，i Let u represent the gradient data of the i-th partition under the t-th fine-tuning. t，i {l} represents the partition gradient data u t，i The corresponding l-th gradient value in the descending absolute value sequence, ζ represents the sparse gradient data corresponding to the i-th partition location encoding. i This represents the error parameter introduced by the quantization algorithm corresponding to the i-th partition position encoding.

[0183] In one optional embodiment, the constraints include:

[0184] k i，c (log2b i +y′)≤P,b i =d / q, r i =R / q,ki，c =k i / r i

[0185] Where, k i，c b represents the gradient allocation amount of the c-th data packet corresponding to the i-th partition location code. i y' represents the number of gradients corresponding to the gradient data of the i-th partition, y′ represents the number of bits required to represent the gradient values ​​in the target gradient data, P represents the data packet size, d represents the number of gradients corresponding to the original gradient data, q represents the number of partitions, and k represents the number of partitions. i Let represent the sparseness of the partition corresponding to the i-th partition location code, r represent the data packet traffic of the partition corresponding to the i-th partition location code, and R represent the maximum data packet traffic.

[0186] In one optional embodiment, the objective optimization function solving module is specifically used for:

[0187] In each iteration, the gradient assignment matrix from the previous iteration is substituted into the objective optimization function to obtain the first optimization function, and the first optimization function is minimized to obtain the number of partitions in the current iteration.

[0188] Substitute the number of partitions in the current iteration into the objective optimization function to obtain the second optimization function, and minimize the partition optimization function corresponding to the second optimization function to obtain the gradient assignment matrix of the current iteration; where the partition optimization function represents the partition compression error;

[0189] When the objective optimization function converges, the number of partitions in the current iteration is taken as the target number of partitions, and the gradient assignment matrix of the current iteration is taken as the target gradient assignment matrix.

[0190] The federated fine-tuning device based on interval compression provided in this embodiment of the invention can execute the federated fine-tuning method based on interval compression provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0191] Figure 6 This is a schematic diagram of an electronic device provided according to one embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0192] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor 11. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0193] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information or data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0194] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the federated fine-tuning method based on interval compression provided in the above embodiments.

[0195] In some embodiments, the interval compression-based federated tuning method provided in the above embodiments can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the interval compression-based federated tuning method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the interval compression-based federated tuning method by any other suitable means (e.g., by means of firmware).

[0196] Various embodiments of the systems and techniques described above herein can be implemented in the following systems or combinations thereof: digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0197] Computer programs for implementing the interval-based compression federated fine-tuning method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0198] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable storage medium. Examples of machine-readable storage media include, based on an electrical connection of at least one wire, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0199] To provide interaction with a user, the systems and techniques described herein can be implemented on a terminal device having: a display device for displaying information to the user (e.g., a cathode-ray tube (CRT) or liquid crystal display (LCD) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the terminal device. Other types of devices can also provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0200] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0201] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0202] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0203] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A federated fine-tuning method based on interval compression, characterized in that, include: Based on the target partitioning parameters, the original gradient data is divided to obtain at least two partitioning location codes corresponding to the partition gradient data. For each partition location code, the partition gradient data corresponding to the partition location code is sparsified to obtain sparse location code and sparse gradient data, and the sparse gradient data is quantized to obtain the target gradient data corresponding to the partition location code. The target transmission data is determined based on the at least two partition location codes, the sparse location codes corresponding to each partition location code, and the target gradient data. The target transmission data is sent to the server so that the server can fine-tune the general pre-trained model based on the target transmission data received from at least two clients.

2. The method according to claim 1, characterized in that, The step of determining the target transmission data based on the at least two partition position codes, the sparse position codes corresponding to each partition position code, and the target gradient data includes: Get the maximum data packet traffic of the communication device corresponding to the client; The ratio between the maximum data packet traffic and the number of partitions in the partition location code is used as the partition data packet traffic corresponding to each partition location code; For each partition location code, the data packets of the partition data packet traffic are determined based on the partition location code, the sparse location code corresponding to the partition location code, and the target gradient data. Generate the target transmission data based on the data packets with the largest data packet flow.

3. The method according to claim 2, characterized in that, The step of determining the data packets of the partition data packet traffic based on the partition location code, the sparse location code corresponding to the partition location code, and the target gradient data includes: The partition location code is encapsulated in the header of the data packets of the partition data packet traffic; The ratio between the number of gradients in the target gradient data and the traffic of the partitioned data packets is used as the gradient allocation amount corresponding to the data packets; wherein, the gradient allocation amount represents the number of gradient values ​​encapsulated in each data packet; Based on the gradient allocation, the sparse location encoding and the target gradient data are encapsulated in the data payload of the data packets in the partitioned data packet traffic.

4. The method according to any one of claims 1-3, characterized in that, The target partitioning parameter is the number of target partitions, the general pre-trained model is a large language model, and correspondingly, the method further includes: The total compression error of the original gradient data is used as the optimization objective to construct an objective optimization function. Determine the constraints based on the maximum data packet traffic and data packet capacity of the communication device corresponding to the client; Under the condition that the constraints are satisfied, the objective optimization function is minimized to obtain the number of objective partitions and the objective gradient assignment matrix; wherein, the matrix parameter k in the objective gradient assignment matrix... i，c The gradient allocation represents the number of gradient values ​​encapsulated in the c-th data packet corresponding to the i-th partition location encoding. Based on the target gradient allocation matrix, determine the partition sparsity corresponding to the sparsification processing of each partition position encoding; Wherein, the objective optimization function represents the sum of the partition compression errors corresponding to at least two partition location codes, the partition compression error represents the compression error of the objective gradient data relative to the partition gradient data, the partition compression error includes sparsity error and quantization error, and the constraints include that the gradient allocation amount of each data packet under the partition location code is the same, the occupied capacity corresponding to the gradient allocation amount is less than or equal to the data packet capacity, the number of gradients corresponding to each partition gradient data is the same, and the partition data packet traffic corresponding to each partition location code is the same.

5. The method according to claim 4, characterized in that, The objective optimization function satisfies the following formula: Among them, Γ t Let q represent the objective optimization function corresponding to the t-th fine-tuning training of the client and the large language model, and let γ represent the number of partitions. t，i G represents the partition compression error corresponding to the position encoding of the i-th partition under the t-th fine-tuning. S G represents the sparsification error. Q Indicates quantization error, u t，i Let u represent the gradient data of the i-th partition under the t-th fine-tuning. t，i {l} represents the partition gradient data u t，i The corresponding l-th gradient value in the descending absolute value sequence, ζ represents the sparse gradient data corresponding to the i-th partition location encoding. i This represents the error parameter introduced by the quantization algorithm corresponding to the i-th partition position encoding.

6. The method according to claim 4, characterized in that, The constraints include: k i，c (log2b i +y′)≤P,b i =d / q,r i =R / q,k i，c =k i / r i ; Where, k i，c b represents the gradient allocation amount of the c-th data packet corresponding to the i-th partition position encoding in the gradient allocation matrix k. i y' represents the number of gradients corresponding to the gradient data of the i-th partition, y′ represents the number of bits required to represent the gradient values ​​in the target gradient data, P represents the data packet capacity, d represents the number of gradients corresponding to the original gradient data, q represents the number of partitions, and k represents the number of partitions. i r represents the partition sparsity corresponding to the i-th partition position code. i R represents the partition data packet traffic corresponding to the i-th partition location code, and R represents the maximum data packet traffic.

7. The method according to claim 4, characterized in that, The process of minimizing the objective optimization function to obtain the number of objective partitions and the objective gradient assignment matrix includes: In each iteration, the gradient assignment matrix from the previous iteration is substituted into the objective optimization function to obtain the first optimization function, and the first optimization function is minimized to obtain the number of partitions in the current iteration. Substituting the number of partitions in the current iteration into the objective optimization function yields the second optimization function. Minimizing each partition optimization function corresponding to the second optimization function yields the gradient assignment matrix for the current iteration. The partition optimization function represents the partition compression error. When the objective optimization function converges, the number of partitions in the current iteration is taken as the target number of partitions, and the gradient assignment matrix of the current iteration is taken as the target gradient assignment matrix.

8. A federated fine-tuning device based on interval compression, characterized in that, include: The original gradient data partitioning module is used to partition the original gradient data according to the target partitioning parameters to obtain partition gradient data corresponding to at least two partition position codes. The target gradient data determination module is used to perform sparsification processing on the partition gradient data corresponding to the partition position code for each partition position code to obtain sparse position code and sparse gradient data, and to perform quantization processing on the sparse gradient data to obtain the target gradient data corresponding to the partition position code. The target transmission data determination module is used to determine the target transmission data based on the at least two partition position codes, the sparse position codes corresponding to each partition position code, and the target gradient data. The target transmission data sending module is used to send the target transmission data to the server so that the server can fine-tune the general pre-trained model based on the target transmission data sent by at least two clients respectively.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the federated fine-tuning method based on interval compression as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the federated fine-tuning method based on interval compression as described in any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the federated fine-tuning method based on interval compression according to any one of claims 1-7.

Images