Machine learning model fine tuning method and system, server and client

Abstract

The invention relates to the technical field of machine learning, and provides a machine learning model fine tuning method and system, a server and a client. In the method, firstly, a server compresses all network layers of a machine learning model to obtain a compressed target network layer, and sends the target network layer to a client; then, the client optimizes and updates the model parameters of the target network layer based on a local training data set, and sends the optimized and updated model parameters back to the server; and finally, the server performs fine adjustment on the machine learning model according to the received optimized and updated model parameters. By adopting the method, the number of transmitted model parameters can be reduced in the model fine tuning process, so that the leakage risk of sensitive information is reduced, and the privacy security of data is improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a machine learning model fine-tuning method, system, server and client. Background Art

[0002] In order to improve the processing ability of machine learning models such as Transformer in specific tasks, it is often necessary to fine-tune the machine learning models. At present, considering the protection of data privacy, the federated learning method is usually used for fine-tuning, which is also called federated fine-tuning. The basic process of federated fine-tuning is that the server distributes the initial model to each client, and each client uses the local training data set to independently train the initial model. After training, the updated model parameters are transmitted back to the server, and the server completes the model aggregation according to the updated model parameters of each client, and then redistributes the updated model to each client, and continuously optimizes the model parameters in this cycle. Since the training data set of the above model fine-tuning process is only stored locally on the client, there is no need to transmit the training data set between the client and the server, so the privacy of the client data can be protected. However, the model parameters transmitted in the federated fine-tuning method may also lead to the leakage of sensitive information. Some attackers can infer sensitive information about the training data set from the transmitted model parameters. It can be seen that the existing federated fine-tuning method still has certain data security risks. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a machine learning model fine-tuning method, system, server and client, which can reduce the leakage of sensitive information during the model fine-tuning process, thereby reducing data security risks.

[0004] A first aspect of an embodiment of the present application provides a machine learning model fine-tuning method applied to a server, comprising:

[0005] Compress all network layers of the machine learning model to obtain compressed target network layers;

[0006] Send the target network layer to the client to instruct the client to optimize and update the model parameters of the target network layer based on the local training data set, and send the optimized and updated model parameters back to the server;

[0007] Fine-tune the machine learning model based on the received optimized updated model parameters.

[0008] The embodiment of the present application improves the existing federated fine-tuning method. First, the server compresses all network layers of the machine learning model to obtain the compressed target network layer, and sends the target network layer to the client; then, the client optimizes and updates the model parameters of the target network layer based on the local training data set, and sends the optimized and updated model parameters back to the server; finally, the server fine-tunes the machine learning model according to the received optimized and updated model parameters. In the above process, since only the model parameters of some network layers of the machine learning model are transmitted between the server and the client, rather than the model parameters of all network layers, the number of transmitted model parameters will decrease, which can reduce the leakage of sensitive information to a certain extent, thereby reducing data security risks.

[0009] In one implementation of the embodiment of the present application, all network layers of the machine learning model are compressed to obtain a compressed target network layer, including:

[0010] According to the similarity of the output results of each network layer in all network layers, all network layers are divided into various network layer clusters based on the clustering method;

[0011] Each network layer cluster is compressed, and each compressed network layer cluster is determined as a target network layer.

[0012] In one implementation of the embodiment of the present application, compressing each network layer cluster includes:

[0013] Get the client's computing resource parameters;

[0014] For each network layer cluster, each network layer included in the network layer cluster is compressed according to the computing resource parameters and the similarity of the output results of each network layer included in the network layer cluster to obtain the compressed network layer cluster.

[0015] In an implementation manner of an embodiment of the present application, the number of clients is multiple, and the computing resource parameter includes the maximum number of reservable network layers; for each network layer cluster, according to the computing resource parameter and the similarity of the output results of each network layer included in the network layer cluster, each network layer included in the network layer cluster is compressed, including:

[0016] According to the defined proportion error and layer error, an objective function is constructed; the proportion error is used to represent the deviation between the actual proportion of network layers retained by multiple clients in each network layer cluster and the expected proportion of network layers retained, and the layer error is used to constrain the total number of network layers retained in each network layer cluster after compression of each client to approach but not exceed the maximum number of network layers that can be retained by the corresponding client;

[0017] The discarded network layers in each network layer cluster are used as control parameters, and the simulated annealing algorithm is used to iteratively optimize the objective function until the optimal solution of the control parameters is obtained when the objective function is minimized.

[0018] According to the optimal solution, some network layers included in each network layer cluster are discarded.

[0019] In one implementation of the embodiment of the present application, fine-tuning the machine learning model according to the received optimized and updated model parameters includes:

[0020] The model parameters of other network layers except the target network layer in the machine learning model are kept unchanged, and the original model parameters of the target network layer in the machine learning model are adjusted to the optimized and updated model parameters.

[0021] A second aspect of an embodiment of the present application provides a machine learning model fine-tuning method applied to a client, comprising:

[0022] Receiving a target network layer sent by a server; wherein the target network layer is a partial network layer obtained after the server compresses all network layers of the machine learning model;

[0023] Optimize and update the model parameters of the target network layer based on the local training data set;

[0024] The optimized and updated model parameters are sent back to the server to instruct the server to fine-tune the machine learning model according to the optimized and updated model parameters.

[0025] In one implementation of the embodiment of the present application, the target network layer is a compressed network layer cluster; optimizing and updating the model parameters of the target network layer based on the local training data set includes:

[0026] For each network layer cluster, if the network layer cluster does not discard the network layer during the compression process, the model parameters of each network layer included in the network layer cluster are optimized and updated based on the training data set.

[0027] A third aspect of an embodiment of the present application provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the machine learning model fine-tuning method provided in the first aspect of the embodiment of the present application is implemented.

[0028] The fourth aspect of an embodiment of the present application provides a client, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the machine learning model fine-tuning method provided in the second aspect of the embodiment of the present application.

[0029] The fifth aspect of the embodiments of the present application provides a machine learning model fine-tuning system, which includes a server as provided in the third aspect of the embodiments of the present application and at least one client as provided in the fourth aspect of the embodiments of the present application.

[0030] The sixth aspect of the embodiments of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the machine learning model fine-tuning method provided in the first aspect of the embodiments of the present application, or implements the machine learning model fine-tuning method provided in the second aspect of the embodiments of the present application.

[0031] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 is a schematic diagram of a machine learning model fine-tuning system provided in an embodiment of the present application;

[0034] Figure 2 is a flow chart of a machine learning model fine-tuning method provided in an embodiment of the present application;

[0035] Figure 3 This is a schematic diagram of determining the optimal number of clusters by using the silhouette coefficient method provided in an embodiment of the present application;

[0036] Figure 4 It is a heat map provided in an embodiment of the present application for clustering network layers of a machine learning model according to similarity;

[0037] Figure 5 It is a schematic diagram of an operation of compressing each network layer included in a network layer cluster according to similarity provided by an embodiment of the present application;

[0038] Figure 6 This is a schematic diagram of an overall operation flow of the machine learning model fine-tuning method provided in an embodiment of the present application;

[0039] Figure 7 It is a structural schematic diagram of a machine learning model fine-tuning device applied to a server provided in an embodiment of the present application;

[0040] Figure 8 It is a structural diagram of a machine learning model fine-tuning device applied to a client provided in an embodiment of the present application;

[0041] Fig. 9 It is a schematic diagram of a server / client provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures, technologies, etc. are proposed, so as to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, the detailed description of well-known systems, devices, circuits and methods is omitted to prevent unnecessary details from hindering the description of the present application. In addition, in the description of the present application specification and the attached claims, the terms "first", "second", "third" etc. are only used to distinguish the description, and cannot be interpreted as indicating or suggesting relative importance.

[0043] Machine learning models such as Transformer perform well in downstream tasks, and in order to improve the processing capabilities of the models in specific tasks, the models often need to be fine-tuned. The existing federated fine-tuning method can ensure that the training data set is only stored locally on the client. The parameters of the machine learning model can be optimized and updated by simply transmitting the model parameters between the server and the client, that is, the machine learning model is fine-tuned. However, the model parameters transmitted in the federated fine-tuning method may also lead to the leakage of sensitive information. Some attackers can infer sensitive information about the training data set from the transmitted model parameters. In addition, due to the different computing resources of different clients, clients with low computing resources may encounter problems such as excessive computing and excessive burden when receiving the model for training, while clients with high computing resources may not fully utilize their computing power when receiving the model for training, resulting in resource waste.

[0044] In response to the above technical problems, the embodiments of the present application provide a machine learning model fine-tuning method, system, server and client. By clustering and compressing the network layer of the machine learning model, the number of transmitted model parameters can be reduced to reduce the possibility of leakage of sensitive information. In addition, an adaptive network layer compression strategy can be formulated according to the client's computing resources, so that the network layer cluster received by the client matches its computing capability, thereby avoiding problems such as excessive computing burden or waste of computing resources on the client.

[0045] Figure 1 Schematic diagram of a machine learning model fine-tuning system provided in an embodiment of the present application. Figure 1The system shown includes a server and at least one client (n≥1), and each client can establish a communication connection with the server through a network or other means. When fine-tuning the machine learning model, the server first divides all network layers of the machine learning model into various network layer clusters by clustering, and then formulates different compression strategies according to the computing resources of each client, compresses each network layer cluster according to the corresponding compression strategy, and sends it to the corresponding client. After receiving its own compressed network layer cluster, each client optimizes and updates the model parameters of the network layer cluster using the local training data set, and sends the optimized and updated model parameters back to the server. After receiving the model parameters sent by each client, the server integrates these model parameters back into the machine learning model on the server side, thereby completing the model fine-tuning. About Figure 1 For a more specific description of the working principle of the system shown, please refer to the method embodiment described below.

[0046] It should be understood that Figure 1 The server and client in the system shown can be various types of electronic devices, such as mobile phones, tablet computers, wearable devices, learning machines, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), large-screen TVs, etc. The embodiments of the present application do not impose any restrictions on the specific types of servers and clients.

[0047] See also Figure 2 , showing a machine learning model fine-tuning method provided by an embodiment of the present application, comprising:

[0048] 201. The server compresses all network layers of the machine learning model to obtain a compressed target network layer;

[0049] The embodiment of the present application improves the existing federated fine-tuning method. The server will not directly distribute all the network layers of the machine learning model to the client, but will first compress the network layers of the machine learning model to obtain the compressed part of the network layers, which are recorded as the target network layers. The server distributes the target network layers to the client. For example, assuming that the machine learning model has 24 network layers, the server will first compress these 24 network layers to obtain 18 compressed network layers, and these 18 network layers will be distributed to each client as the target network layer. In addition, the server can also perform different degrees of network layer compression according to the computing resources of different clients, so that the number of network layers received by different clients may be different. For example, the number of network layers received by clients with more computing resources is more, while the number of network layers received by clients with less computing resources is less, so as to match the number of network layer processing with the computing resource capacity of the client. When the server compresses the network layer, it can give priority to discarding the part of the network layer that has a higher similarity and higher substitutability with other network layers, so as to minimize the impact of the reduction of network layers on the overall function of the model.

[0050] In order to improve the accuracy of network layer compression, the server can first group the network layers of the machine learning model. Network layers with high similarity can be divided into the same group, and then compress each network layer group separately, so that it is easy to find the network layers with high replaceability and discard them. When grouping network layers, clustering methods can be used. Please refer to the following for the specific clustering operation process.

[0051] In one implementation of the embodiment of the present application, the server compresses all network layers of the machine learning model to obtain a compressed target network layer, including:

[0052] (1) The server divides all network layers into different network layer clusters based on the similarity of the output results of each network layer in all network layers using the clustering method;

[0053] (2) The server compresses each network layer cluster and determines each compressed network layer cluster as the target network layer.

[0054] When the server clusters all the network layers of the machine learning model, it can first calculate the similarity of the output results of each network layer. For example, assuming that a machine learning model has 24 network layers, the 24 network layers have their own output results for the same test data set. The cosine similarity between these output results can be calculated to evaluate the similarity between each network layer. After calculating the similarity of the output results of different network layers, the clustering method can be used to divide the network layers with higher similarity into the same network layer cluster. In addition, considering that the functions of the network layers at adjacent levels are similar, they usually have higher similarity, so the network layers at adjacent levels can be divided into the same network layer cluster. After all the network layers of the machine learning model are divided into each network layer cluster, each network layer cluster is compressed, so that the network layers with higher substitutability can be easily found and discarded. The compressed network layer clusters are used as target network layers and distributed to each client.

[0055] The above clustering process can use various clustering algorithms. The following uses the K-means clustering algorithm as an example. Assume that the number of network layers of the machine learning model is L M , all its network layers can be expressed as {l k |k=1,2,…,L M}, clustering these network layers into m clusters, denoted as {c j ||j=1,2,…,m}, where each cluster c j By L j It can be seen that each cluster c j is a subset of all network layers and can be expressed as When using the K-means clustering algorithm for clustering, the silhouette coefficient method can be used to determine the optimal number of clusters m. The silhouette coefficient is an evaluation index used to evaluate the effectiveness of clustering. The silhouette coefficient method can be used to calculate the silhouette score corresponding to each number of clusters. The number of clusters corresponding to the highest silhouette score can be regarded as the optimal number of clusters. Figure 3 FIG. 1 is a schematic diagram of determining the optimal number of clusters by using the silhouette coefficient method provided in an embodiment of the present application. Figure 3 It can be seen that when the number of clusters is 6, the corresponding silhouette score is higher. Although the corresponding silhouette score is higher when the number of clusters is 2 or 3, too few clusters will reduce the ability to distinguish groups, which may have a negative impact on model compression. Therefore, the optimal number of clusters is determined to be 6, that is, 6 network layer clusters are divided. After that, by calculating the cosine similarity between the output results of different network layers, the corresponding cluster heat map can be constructed. The cluster heat map can clearly know the similarity between each network layer, so that the network layers with higher similarity can be conveniently divided into the same network layer cluster. Figure 4 As shown, a heat map of clustering network layers of a machine learning model according to similarity is provided in an embodiment of the present application. Figure 4 In the machine learning model, there are 24 network layers. The cosine similarity between the output results of any two network layers is calculated separately, and then all the similarities are integrated together to construct the following Figure 4 The cluster heatmap shown, for example, Figure 4 The elements in the second row and third column represent the cosine similarity between the output results of the second network layer and the third network layer, and so on. Based on the execution results of the clustering algorithm, each network layer cluster can be selected in the clustering heat map. For example, Figure 4 Six rectangular boxes are selected in the middle box, each of which represents a network layer cluster. The similarity between adjacent network layers contained in each network layer cluster is relatively high, where network layer cluster 1 contains network layer 0, network layer cluster 2 contains network layer 1-network layer 8, network layer cluster 3 contains network layer 9-network layer 14... and so on.

[0056] After obtaining each network layer cluster through the above clustering method, the server can then perform compression on each network layer cluster separately.

[0057] In an implementation of the embodiment of the present application, the server compresses each network layer cluster, including:

[0058] (1) The server obtains the computing resource parameters of the client;

[0059] (2) For each network layer cluster, the server compresses each network layer included in the network layer cluster according to the computing resource parameters and the similarity of the output results of each network layer included in the network layer cluster to obtain the compressed network layer cluster.

[0060] In a system consisting of a server and n clients, the server can formulate different compression strategies for each client, and the specific compression strategy can be reasonably formulated according to the computing resource parameters of each client. For example, assuming that all network layers contained in a machine learning model are clustered into 6 network layer clusters, the server formulates compression strategy A according to the computing resources of client A, compresses the 6 network layer clusters using compression strategy A to obtain 6 network layer clusters A, and then sends the 6 network layer clusters A to client A; the server formulates compression strategy B according to the computing resources of client B, compresses the 6 network layer clusters using compression strategy B to obtain 6 network layer clusters B, and then sends the 6 network layer clusters B to client B, and so on. Generally speaking, if a client has more computing resources, it can be allocated more network layers, and accordingly, the number of network layers that need to be discarded in the compression process of its network layer cluster is also less; if a client has fewer computing resources, it can be allocated fewer network layers, and accordingly, the number of network layers that need to be discarded in the compression process of its network layer cluster is more. Specifically, after obtaining the computing resource parameters of the client, the server can determine the number x of network layers that need to be discarded based on the computing resource parameters. However, based on the similarity of the output results of each network layer contained in each network layer cluster, the x network layers with higher similarity are discarded first, thereby completing the compression of each network layer cluster and obtaining each compressed network layer cluster.

[0061] As an example, Figure 5 It is a schematic diagram of an operation of compressing each network layer included in a network layer cluster according to similarity provided by an embodiment of the present application. Figure 5 Six network layer clusters (including 24 network layers in total) are shown, namely network layer cluster 1 (including network layer 0), network layer cluster 2 (including network layer 1-network layer 8), network layer cluster 3 (including network layer 9-network layer 14), network layer cluster 4 (including network layer 15-network layer 18), network layer cluster 5 (including network layer 19-network layer 21) and network layer cluster 6 (including network layer 22 and network layer 23). The color of each network layer represents the similarity between it and other network layers in the same network layer cluster. The darker the color, the lower the similarity, that is, white represents the highest similarity. When compressing the six network layer clusters, network layers with higher similarity in the same network layer cluster are discarded first. These network layers are more replaceable and therefore relatively less important. Discarding network layers with lower importance can minimize the impact on the overall function of the model. Figure 4In the example, network layer 4, network layer 11, network layer 12, network layer 16, network layer 17 and network layer 20, which have high similarity with other network layers, are given priority. As for how many network layers are discarded, they can be reasonably set according to the computing resources of each client. For example, if client A has more computing resources, only two network layers can be discarded, so that the six network layer clusters distributed by the server to client A contain a total of 22 network layers; if client B has fewer computing resources, six network layers can be discarded, so that the six network layer clusters distributed by the server to client B contain a total of 18 network layers, and so on.

[0062] In an implementation manner of an embodiment of the present application, the number of clients is multiple, and the computing resource parameter includes the maximum number of reservable network layers; the server compresses each network layer contained in the network layer cluster according to the computing resource parameter and the similarity of the output results of each network layer contained in the network layer cluster, including:

[0063] (1) The server constructs an objective function based on the defined ratio error and layer error. The ratio error is used to represent the deviation between the actual ratio of network layers retained by multiple clients in each network layer cluster and the expected ratio of network layers retained. The layer error is used to constrain the total number of network layers retained in each network layer cluster after compression of each client to approach but not exceed the maximum number of network layers that can be retained by the corresponding client.

[0064] (2) The server uses the discarded network layers in each network layer cluster as control parameters and uses the simulated annealing algorithm to iteratively optimize the objective function until the optimal solution of the control parameters is obtained when the objective function is minimized;

[0065] (3) The server discards part of the network layers contained in each network layer cluster according to the optimal solution.

[0066] Assumption C i is the set of network layer clusters assigned to a client i. For C i Each network layer cluster c in j , the network layer discarded by the server is denoted as d ij , the retained network layer is denoted as r ij , specific d ij and r ij It can be determined based on the computing resources of client i, d ij and r ij It can be a continuous network layer or a discontinuous network layer. Specifically, the computing resource parameter may include the maximum number of network layers that can be retained by client i, denoted as cap i , cap iReflects the computing resource size of client i, which indicates the maximum number of network layers that client i can retain in all network layer clusters, that is, the total number of network layers that client i needs to retain in all network layer clusters The goal of the server to compress the network layer clusters is to optimize the number of network layers discarded and retained by each client in each network layer cluster when determining the network layer clusters assigned to each client, so as to maximize the computing resource performance of the server and each client while meeting the requirements of resource constraints.

[0067] To achieve the above goals, the server can first define the target ratio tp j , which represents the expected proportion of network layers retained by the jth network layer cluster among all clients. It is necessary to make the actual proportion of network layers retained by the jth network layer cluster equal to the target proportion tp j As close as possible. For this purpose, a proportional error PE can be defined, which represents the deviation between the actual proportion of network layers retained by all clients in each network layer cluster and the expected proportion of network layers retained, which can capture the compression of different network layer clusters. After research, it was found that the network layers of the top modules and the bottom modules in the machine learning model are more important than the network layers of modules in other positions. Therefore, the network layers of the top modules and the bottom modules should remain intact, or the number of discarded network layers should be small, while the modules in the middle position can discard more network layers. In addition, for modules with ≤2 layers, no network layers should be discarded to maintain their functions. For example, in the example of the six network layer clusters mentioned above, network layer clusters 1 and network layer cluster 6 at the front and back ends should not discard any network layers, while network layer clusters 2, 3 and 4 in the middle can discard some network layers with higher similarity, which can minimize the impact on the overall function of the model. By reasonably setting the target ratio tp j , the objective function is constructed using the defined proportional error PE. In the subsequent process, the optimal solution for discarding and retaining network layers in each network layer cluster can be obtained by continuously optimizing the objective function.

[0068] In addition, considering the different computing resource sizes of different clients, the server also defines a layer error LE, which is used to constrain the total number of network layers retained in each network layer cluster after compression of each client to approach but not exceed the maximum number of network layers that can be retained by the corresponding client. The layer error LE can capture resource constraints, which limits the number of network layers retained by any client to r ij The sum of the number of network layers cannot exceed the maximum number of network layers that can be retained, cap i , and any client retains the network layer r ij The sum of the number of network layers is as close as possible to the maximum number of network layers that can be retained, cap i For the reserved network layer r ij The quantity exceeds the capi In the case of , a higher penalty can be imposed to prevent excessive use of client resources, while for the reserved network layer r ij The number of cap i In the following cases, a lower penalty can be imposed to encourage near-optimal client resource utilization.

[0069] The objective function is constructed based on the proportional error PE and the layer error LE, which can be expressed by the following formula:

[0070] Min TE=α·LE+γ·PE

[0071] Among them, Min TE represents the objective function, which aims to minimize the total error TE including the proportional error PE and the layer error LE, and α and γ are weight factors used to balance the importance of the layer error and the proportional error. For example, if the layer error is considered more important, α>y can be set; if the two errors are considered equally important, α=γ can be set; if the proportional error is considered more important, α<γ can be set. After constructing the objective function, the server uses the network layer cluster c j The network layer d is discarded ij As the control parameter, the simulated annealing algorithm is used to iteratively optimize the objective function. The algorithm starts from a broad solution, explores a large number of possible solutions, and gradually optimizes the solution over time. In each iterative calculation, the random perturbation d ij The elements in the simulated annealing algorithm are used to evaluate the new solution obtained after the perturbation using the objective function. If the new solution reduces the loss, the new solution is accepted. Otherwise, the new solution is accepted according to a probability that decreases with the increase of iterations. After multiple iterations, the simulated annealing algorithm can obtain the optimal solution of the control parameters corresponding to the minimization of the objective function. The optimal solution is to select the network layer cluster c j The network layer d is discarded ij The optimal solution is then used to discard the network layer cluster c. j The corresponding network layer in .

[0072] In the above process, the proportional error is for the entire client, and is used to measure whether the network layer retention ratio of all clients in each network layer cluster is close to the set target value, that is, the expected network layer retention ratio. For example, assuming there are a total of 3 network layer clusters, and their expected network layer retention ratios are (0.9, 0.4, 0.9), it means that the expected network layer retention ratio of the first network layer cluster is 90%, the expected network layer retention ratio of the second network layer cluster is 40%, and the expected network layer retention ratio of the third network layer cluster is 90%. Each network layer cluster has a corresponding proportional error. For a certain network layer cluster c j For example, its proportional error is equal to the error of all clients in the network layer cluster c jThe number of network layers retained on the network, divided by the network layer cluster c j The product of the original number of network layers and the number of clients, and then subtract the network layer cluster c j The expected network layer retention ratio. The layer error constrains individual clients to ensure that the number of network layers retained by each client does not exceed the maximum value allowed to match the client's computing power. Through this dual error setting, different clients are allowed to have differentiated performances, and the model can achieve a balance of compression both globally and locally, ensuring the overall effect of all clients while ensuring that a single client does not over-compress or retain too many network layers, achieving better training and inference results.

[0073] By constructing the above objective function and adopting the simulated annealing algorithm, the server can obtain the optimal network layer cluster compression method for each client, so that the network layer cluster received by each client matches its computing power, avoiding problems such as excessive computing burden or waste of computing resources on the client.

[0074] 202. The server sends the target network layer to the client;

[0075] After obtaining the compressed target network layer, the server can send the target network layer to each client through the network or other means.

[0076] As an example, the target network layers sent by the server to each client may be the same. For example, after compressing 24 network layers of the machine learning model into 18 network layers, the server may send the 18 network layers to each client respectively.

[0077] As another example, the target network layer sent by the server to each client may also be different, which is determined according to the computing resources of each client. For example, assuming that the target network layer is the six network layer clusters described above, the server can formulate different compression strategies according to the computing resources of each client, compress the six network layer clusters according to different compression strategies, and then distribute the compression results of different network layer clusters to the corresponding clients.

[0078] 203. The client optimizes and updates the model parameters of the target network layer based on the local training data set;

[0079] After receiving the target network layer sent by the server, each client can train based on the local training data set to optimize and update the model parameters of the target network layer. This process uses the client-specific training data to fine-tune the model parameters, which can adapt the machine learning model to specific application scenarios while protecting the user's data privacy. Training the model on the client's local training data set is mainly based on two goals. The first goal is to make each client's model more adaptable to local data, and the second goal is to make the model more adaptable to downstream tasks.

[0080] Considering the large number of clients, which may reach hundreds or more, and for large machine learning models, the number of target network layers and model parameters that need to be transmitted is also huge, which will bring huge traffic pressure to the server's data transmission. In order to alleviate the traffic pressure of the server, the server can randomly select a part of the clients to participate in the training in each round of training, for example, only select 30% of the clients.

[0081] In one implementation of the embodiment of the present application, the target network layer is a compressed network layer cluster; the client optimizes and updates the model parameters of the target network layer based on the local training data set, including:

[0082] For each network layer cluster, if the network layer cluster does not discard the network layer during the compression process, the client optimizes and updates the model parameters of each network layer included in the network layer cluster based on the training data set.

[0083] Assuming that the target network layers obtained by a client are the compressed network layer clusters, some of which are complete clusters (network layer clusters that have not discarded network layers during the compression process) and the other network layer clusters are compressed clusters (network layer clusters that have discarded network layers during the compression process), the client only trains the complete clusters, that is, based on the training data set, the model parameters of each network layer contained in the complete cluster are optimized and updated. As for the compressed cluster, the client can freeze its model parameters, that is, keep the model parameters of each network layer contained in the compressed cluster unchanged. For example, assuming that the network layer clusters received by a client are network layer cluster 1 (network layer 0 and network layer 1), network layer cluster 2 (network layer 3, network layer 5, network layer 7 and network layer 9) and network layer cluster 3 (network layer 10 and network layer 11), where network layer cluster 1 and network layer cluster 3 are complete clusters and network layer cluster 2 is a compressed cluster, the client only trains network layer cluster 1 and network layer cluster 3, and updates the model parameters of network layer 0, network layer 1, network layer 10 and network layer 11. As for network layer cluster 2, since it is compressed (network layers 4, 6, and 8 are discarded), the client does not train network layer cluster 2, and keeps the model parameters of network layers 3, 5, 7, and 9 unchanged. The reason for this is that if a network layer cluster discards a part of the network layers, the model parameters obtained by training the network layer cluster are inaccurate, which is not conducive to the performance optimization of the model. However, only training the network layer clusters that do not discard the network layers can obtain more accurate model parameters, which is conducive to the performance optimization of the model.

[0084] 204. The client sends the optimized and updated model parameters back to the server;

[0085] After each client obtains the optimized and updated model parameters through training, it sends these model parameters back to the server. For example, in the above example, the client sends the optimized and updated model parameters of network layer 0, network layer 1, network layer 10, and network layer 11 back to the server.

[0086] 205. The server fine-tunes the machine learning model according to the received optimized and updated model parameters.

[0087] After receiving the optimized and updated model parameters sent by each client, the server will update the original model parameters of each corresponding network layer in the machine learning model according to these model parameters, that is, embed the optimized and updated model parameters into the machine learning model on the server side, thereby completing the fine-tuning of the machine learning model.

[0088] In one implementation of the embodiment of the present application, the server fine-tunes the machine learning model according to the received optimized and updated model parameters, including:

[0089] The server keeps the model parameters of other network layers in the machine learning model except the target network layer unchanged, and adjusts the original model parameters of the target network layer in the machine learning model to the optimized and updated model parameters.

[0090] Since the server compresses the network layer distributed to the client, the optimized and updated model parameters received by the server are only the model parameters of some network layers. Assuming that the server receives the optimized and updated model parameters of the target network layer, when fine-tuning the machine learning model, the model parameters of other network layers except the target network layer remain unchanged, and the original model parameters of the target network layer in the machine learning model are adjusted to the optimized and updated model parameters. For example, assuming that the server receives the optimized and updated model parameters of network layer 0, network layer 1, network layer 10 and network layer 11, these model parameters are used to replace the original model parameters of network layer 0, network layer 1, network layer 10 and network layer 11 in the machine learning model respectively, and the model parameters of other network layers remain unchanged. In addition, if the server receives different model parameters for the same network layer returned by multiple different clients, the average value of these model parameters can be calculated, and the original model parameters of the corresponding network layer can be replaced by the average value; or, the weighted average value of these model parameters (the weight can be set according to the importance of the corresponding client) is calculated, and the original model parameters of the corresponding network layer are replaced by the weighted average value.

[0091] After the server completes a model parameter update of the machine learning model, it is said to have completed a round of fine-tuning. Next, the server can continue to compress and distribute the various network layers of the machine learning model after the model parameters are updated to each client in the same way, entering the next round of fine-tuning, and so on, continuously training and optimizing the machine learning model until the performance of the machine learning model meets the requirements or reaches the set fine-tuning round.

[0092] The embodiment of the present application improves the existing federated fine-tuning method. First, the server compresses all network layers of the machine learning model to obtain the compressed target network layer, and sends the target network layer to the client; then, the client optimizes and updates the model parameters of the target network layer based on the local training data set, and sends the optimized and updated model parameters back to the server; finally, the server fine-tunes the machine learning model according to the received optimized and updated model parameters. In the above process, since only the model parameters of some network layers of the machine learning model are transmitted between the server and the client, rather than the model parameters of all network layers, the number of transmitted model parameters will decrease, which can reduce the leakage of sensitive information to a certain extent, thereby reducing data security risks.

[0093] As an example, Figure 6This is a schematic diagram of the overall operation flow of the machine learning model fine-tuning method provided in the embodiment of the present application. Figure 6 In the process, the server first enters the clustering stage, clustering all the network layers of the machine learning model, and divides the adjacent network layers in the model into several network layer clusters according to the similarity of the network layer output results; then enters the compression process, the server formulates different compression strategies according to the computing resources of each client, and compresses each network layer cluster obtained by clustering using different compression strategies; then enters the allocation stage, the server distributes each compressed network layer cluster to the corresponding client, where each network layer cluster obtained by compression strategy A of client A will be allocated to client A, and each network layer cluster obtained by compression strategy C of client C will be allocated to Client C, and so on. In addition, the server can select some clients to participate in the training, for example, select client A and client C to participate in the training, while client B does not participate; then enter the training phase, client A and client C train the received network layer cluster based on the local training data set to obtain the optimized and updated model parameters of the corresponding network layer; finally enter the embedding phase, each client sends the optimized and updated model parameters back to the server, which is integrated into the server-side machine learning model, thereby completing a round of model fine-tuning process, and then enter the next round of model fine-tuning process, and repeat the training continuously until a machine learning model that meets the performance requirements is obtained.

[0094] From the above content, it can be seen that the embodiment of the present application is equivalent to proposing an innovative federated fine-tuning method, which uses integrated hierarchical clustering and model compression technology to reduce the number of model parameters transmitted, thereby reducing the risk of sensitive information leakage. Moreover, the method can also reasonably formulate an adaptive compression strategy based on the computing resources of the client to ensure that different clients with different computing resources can efficiently participate in the fine-tuning process of the model. In addition, experimental results show that the use of this method can enhance data privacy protection while also achieving a model optimization effect comparable to that of the traditional federated fine-tuning method.

[0095] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0096] The above mainly describes the machine learning model fine-tuning method, and the machine learning model fine-tuning device will be described below.

[0097] See also Figure 7 , showing a machine learning model fine-tuning device applied to a server provided by an embodiment of the present application, comprising:

[0098] The network layer compression module 701 is used to compress all network layers of the machine learning model to obtain a compressed target network layer;

[0099] The network layer distribution module 702 is used to send the target network layer to the client to instruct the client to optimize and update the model parameters of the target network layer based on the local training data set, and send the optimized and updated model parameters back to the server;

[0100] The model fine-tuning module 703 is used to fine-tune the machine learning model according to the received optimized and updated model parameters.

[0101] In one implementation of the embodiment of the present application, the network layer compression module includes:

[0102] A network layer clustering unit, used for dividing all network layers into respective network layer clusters based on a clustering method according to the similarity of the output results of each network layer in all network layers;

[0103] The network layer compression unit is used to compress each network layer cluster and determine each compressed network layer cluster as a target network layer.

[0104] In one implementation of the embodiment of the present application, the network layer compression unit includes:

[0105] A computing resource acquisition subunit, used to obtain computing resource parameters of the client;

[0106] The network layer compression subunit is used to compress each network layer contained in the network layer cluster according to the computing resource parameters and the similarity of the output results of each network layer contained in the network layer cluster to obtain the compressed network layer cluster.

[0107] In an implementation manner of the embodiment of the present application, the number of clients is multiple, and the computing resource parameter includes the maximum number of reservable network layers; the network layer compression subunit includes:

[0108] The objective function construction subunit is used to construct an objective function according to the defined proportion error and layer error; wherein the proportion error is used to represent the deviation between the proportion of network layers actually retained by multiple clients in each network layer cluster and the expected proportion of network layers retained, and the layer error is used to constrain the total number of network layers retained in each network layer cluster after compression of each client to approach but not exceed the maximum number of network layers that can be retained by the corresponding client;

[0109] The optimal solution acquisition subunit is used to use the discarded network layers in each network layer cluster as control parameters, and adopt the simulated annealing algorithm to iteratively optimize the objective function until the optimal solution of the control parameters is obtained when the objective function is minimized;

[0110] The network layer discarding subunit is used to discard part of the network layers contained in each network layer cluster according to the optimal solution.

[0111] In one implementation of the embodiment of the present application, the model fine-tuning module includes:

[0112] The parameter adjustment unit is used to keep the model parameters of other network layers in the machine learning model except the target network layer unchanged, and adjust the original model parameters of the target network layer in the machine learning model to the optimized and updated model parameters.

[0113] See also Figure 8 , showing a machine learning model fine-tuning device applied to a client provided by an embodiment of the present application, comprising:

[0114] The network layer receiving module 801 is used to receive the target network layer sent by the server; wherein the target network layer is a partial network layer obtained after the server compresses all network layers of the machine learning model;

[0115] A training module 802 is used to optimize and update the model parameters of the target network layer based on a local training data set;

[0116] The model parameter sending module 803 is used to send the optimized and updated model parameters back to the server to instruct the server to fine-tune the machine learning model according to the optimized and updated model parameters.

[0117] In one implementation of the embodiment of the present application, the target network layer is each compressed network layer cluster; the training module includes:

[0118] The network layer cluster training unit is used to optimize and update the model parameters of each network layer contained in each network layer cluster based on the training data set if the network layer cluster does not discard the network layer during the compression process.

[0119] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned machine learning model fine-tuning method performed by the server, or implements the above-mentioned machine learning model fine-tuning method performed by the client.

[0120] Fig. 9 Schematic diagram of a server / client provided in an embodiment of the present application. Fig. 9As shown, the server / client 9 of this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90. When the processor 90 executes the computer program 92, the steps in the above-mentioned embodiments of the machine learning model fine-tuning method are implemented, such as the steps of the machine learning model fine-tuning method executed by the server, or the steps of the machine learning model fine-tuning method executed by the client. Alternatively, when the processor 90 executes the computer program 92, the functions of the modules / units in the above-mentioned device embodiments are implemented, such as the steps of Figure 7 The functions of modules 701 to 703 of the device shown in the figure, or the implementation Figure 8 Functions of modules 801 - 803 of the illustrated apparatus.

[0121] The computer program 92 may be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 92 in the server / client 9.

[0122] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0123] The memory 91 may be an internal storage unit of the server / client 9, such as a hard disk or memory of the server / client 9. The memory 91 may also be an external storage device of the server / client 9, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server / client 9. Further, the memory 91 may also include both an internal storage unit of the server / client 9 and an external storage device. The memory 91 is used to store the computer program and other programs and data required by the server / client. The memory 91 may also be used to temporarily store data that has been output or is to be output.

[0124] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0126] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0128] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0130] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0132] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A machine learning model fine-tuning method, applied to a server, characterized in that: The method comprises: Compress all network layers of the machine learning model to obtain compressed target network layers; Sending the target network layer to the client to instruct the client to optimize and update the model parameters of the target network layer based on the local training data set, and sending the optimized and updated model parameters back to the server; The machine learning model is fine-tuned according to the received optimized and updated model parameters.

2. The method according to claim 1, characterized in that The method of compressing all network layers of the machine learning model to obtain a compressed target network layer includes: According to the similarity of the output results of each network layer in all the network layers, dividing all the network layers into respective network layer clusters based on a clustering method; Each of the network layer clusters is compressed, and each of the compressed network layer clusters is determined as the target network layer.

3. The method according to claim 2, characterized in that The compressing each of the network layer clusters includes: Obtaining computing resource parameters of the client; For each of the network layer clusters, the network layers included in the network layer cluster are compressed according to the computing resource parameters and the similarity of the output results of the network layers included in the network layer cluster to obtain the compressed network layer cluster.

4. The method according to claim 3, characterized in that The number of the clients is multiple, and the computing resource parameter includes a maximum number of network layers that can be retained; for each of the network layer clusters, compressing each network layer included in the network layer cluster according to the computing resource parameter and the similarity of the output results of each network layer included in the network layer cluster, including: According to the defined proportion error and layer error, an objective function is constructed; wherein the proportion error is used to represent the deviation between the proportion of network layers actually retained by the plurality of clients in each of the network layer clusters and the proportion of network layers expected to be retained, and the layer error is used to constrain the total number of network layers retained in each of the network layer clusters after compression of each of the clients to approach but not exceed the maximum number of network layers that can be retained by the corresponding client; The network layers discarded in each of the network layer clusters are used as control parameters, and the objective function is iteratively optimized using a simulated annealing algorithm until an optimal solution of the control parameters is obtained when the objective function is minimized; Part of the network layers included in each of the network layer clusters is discarded according to the optimal solution.

5. The method according to any one of claims 1 to 4, characterized in that: The step of fine-tuning the machine learning model according to the received optimized and updated model parameters comprises: The model parameters of other network layers in the machine learning model except the target network layer are kept unchanged, and the original model parameters of the target network layer in the machine learning model are adjusted to the optimized and updated model parameters.

6. A machine learning model fine-tuning method, applied to a client, characterized in that: The method comprises: Receiving a target network layer sent by a server; wherein the target network layer is a partial network layer obtained after the server compresses all network layers of the machine learning model; Optimizing and updating the model parameters of the target network layer based on the local training data set; The optimized and updated model parameters are sent back to the server to instruct the server to fine-tune the machine learning model according to the optimized and updated model parameters.

7. The method according to claim 6, characterized in that The target network layer is each compressed network layer cluster; the optimizing and updating of the model parameters of the target network layer based on the local training data set includes: For each of the network layer clusters, if the network layer cluster does not discard the network layer during the compression process, the model parameters of each network layer included in the network layer cluster are optimized and updated based on the training data set.

8. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the machine learning model fine-tuning method as described in any one of claims 1 to 5 is implemented.

9. A client comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the machine learning model fine-tuning method as described in claim 6 or 7 is implemented.

10. A machine learning model fine-tuning system, characterized in that: The method comprises a server as claimed in claim 8 and at least one client as claimed in claim 9.

Images