A semi-asynchronous federated learning method based on blockchain

Abstract

The present invention relates to the field of federated learning technology, and discloses a semi-asynchronous federated learning method based on blockchain, comprising the following steps: creating a global model A and initializing it; the client extracts the global model A and trains it on local data to generate updated model parameters; then generates a new global model B and packages it into a new block; uses the NDP-Kmeans clustering algorithm to perform clustering grouping; after the grouping is completed, distributes the global model B to each client group; the client in the group uses the global model B to perform model update on local data; packages the latest global model into a block, uploads it to the blockchain network and reaches consensus, and executes it cyclically until the global model converges and reaches the expected performance index, or the training process reaches the preset maximum round. While maintaining the quality of model update, the method of the present invention greatly improves the overall efficiency of federated learning in a device heterogeneous environment.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a semi-asynchronous federated learning method based on blockchain. Background Art

[0002] As an emerging distributed machine learning technology, federated learning aims to learn a shared model through the cooperation of multiple participants without exchanging their respective data, making it widely used in various fields. In the field of federated learning, there are many technical implementation methods, among which synchronous and asynchronous federated learning are the two main strategies. Traditional synchronous federated learning requires all participating devices to synchronously submit their own model updates in each round of updates. Although this method can ensure the consistency of the model, the differences in the computing power and network communication capabilities of the devices lead to inconsistent processing speeds, thus affecting the overall training efficiency.

[0003] Asynchronous federated learning provides a solution to the above problems by allowing devices to train at their own pace and send updates to a central server when they are ready. This approach can better handle heterogeneity between devices and the non-IID nature of data because it does not require all devices to complete training at the same time to update the model. However, asynchronous federated learning also faces its own challenges, such as the impact of model obsolescence. When the server aggregates the model, it may receive "outdated" gradients or parameter updates. These "outdated" updates may interfere with the direction of the global model and affect the stability and effectiveness of training.

[0004] In addition, security is also an important consideration in existing technologies. In an open network environment, devices participating in training may be subject to malicious attacks. Malicious devices may deliberately send erroneous or biased updates to disrupt the learning process of the global model. Existing federated learning mechanisms have limited defense capabilities against such attacks, and more effective security mechanisms are needed to ensure the reliability of the training process and the overall quality of the model. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a semi-asynchronous federated learning method based on blockchain, so as to achieve the purpose of greatly improving the overall efficiency of federated learning in a heterogeneous device environment while maintaining the quality of model update.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] A semi-asynchronous federated learning method based on blockchain includes the following steps:

[0008] Step 1: The task publisher creates a global model A and publishes it to the blockchain network, and initializes it according to the preset initial parameters;

[0009] Step 2: The client extracts the global model A from the blockchain network and trains it on local data to generate updated model parameters. The client packages these model parameters and uploads them to the blockchain network. After receiving them, the smart contract automatically aggregates the model parameters of each client to generate a new global model B and packages it into a new block. At the same time, the clients also save their basic information into the same clustering round file to provide basic data for subsequent clustering grouping.

[0010] Step 3: Use the NDP-Kmeans clustering algorithm to perform clustering and grouping based on the basic information of the clients stored in the clustering round file; after the grouping is completed, distribute the global model B to each client group;

[0011] Step 4: The clients in the group use the global model B to perform model updates on local data. During training, the basic information of the clients is recorded in different clustering round files. For groups whose model deviation does not exceed the threshold, the local models of the clients in this group are aggregated and updated with the current global model using an asynchronous update method, and redistributed to the clients in the current group. If the local model deviation of a client in a group exceeds the threshold, the training of all the clients in the group is stopped, and the model is not aggregated. The local models of all the clients in the group are directly synchronized and updated to the current global model, and all the clients are clustered again according to the latest clustering round file, and a new clustering round file is created.

[0012] Step 5: After each round of model update is completed, the latest global model is packaged into a block and uploaded to the blockchain network. After the latest block is uploaded to the blockchain network, all participating nodes verify the validity of the latest block through the consensus mechanism and reach an agreement on it. After reaching a consensus, the block will be officially added to the blockchain network and become the only recognized model version in the system. This latest global model will be used by the client group in the next round of training.

[0013] Step 6: Repeat steps 4 and 5 until the global model converges and reaches the expected performance index, or the training process reaches the preset maximum number of rounds.

[0014] In the above scheme, in step 2, the basic information of the client includes computing power time overhead, communication power time overhead and model deviation degree;

[0015] Client After receiving the global model, it is recorded as time , the model training time starts at , model training end time , the time to package and upload the model to the blockchain network is , For Clients The total time spent, including computing power time overhead and communication power time overhead; For Clients The time it takes to train the model, i.e. the computing power time overhead;

[0016] The degree of model deviation is calculated by comparing the global model with the client's local model. The norm is used to express, The norm is a measure of the difference between model parameter vectors and is calculated as:

[0017] ;

[0018] in, Represents the relationship between the client local model and the global model norm, that is, the degree of difference between the two models, x =[ x 1 , x 2 , x 3 , … , x n ] represents the parameter vector of the client model, y =[ y 1 , y 2 , y 3 , … , y n ] Represents the parameter vector of the global model, and n is the dimension of the parameter vector.

[0019] In the above scheme, the specific method of step 3 is as follows:

[0020] Step 1: Use the basic information saved during client training as the clustering data set. The clustering data set consists of the client's computing power time overhead, communication power time overhead, and model deviation degree.

[0021] Step 2: Define the density of client objects and use natural neighbors to calculate the density. Natural neighbors are obtained by adaptively searching for nearest neighbors and recording reverse neighbor information.

[0022] Step 3: Use neighbor-based distance to calculate the graph distance between natural density peaks. The graph distance combines the performance data of the client, making the graph distance between clients with high computing power, high communication capacity and low model offset smaller;

[0023] Step 4: Select the natural density peak with the largest density as the first center, and then select the natural density peak with the largest graph distance from the selected centers as the new center until the required number of centers is reached; during the center selection process, clients with high computing power and high communication capabilities are given priority;

[0024] Step 5: Calculate the member allocation of each group based on the graph distance, so that the distance between clients in the group is smaller.

[0025] In the further technical solution, the specific process of Step 2 is as follows:

[0026] (1) Calculate the client object density: For client objects , is the set of all client objects, whose density The calculation is as follows:

[0027] ;

[0028] in, Set to or The maximum value of It is the natural eigenvalue that can adaptively determine the number of nearest neighbors. Represents a client object The number of reverse neighbors, Is the client object of A set of nearest neighbor objects, Is the client object and The Euclidean distance between

[0029] (2) Determine natural neighbors: For each client object, select a set of nearest neighbor objects as its natural neighbors. These neighbors are similar to the client object in computing and communication capabilities. , its natural neighbor set It is The nearest neighbor object set ;

[0030] (3) Selecting a natural representative: The natural representative of each client object is the object with the highest density among itself or its natural neighbors. Natural representative of yes the object with the highest density among its natural neighbors;

[0031] (4) Update the natural representation chain: By updating the natural representation, the natural representation chain of the client object can continue; if the client object Natural representative of Is the client object ,and Natural representative of Is the client object , then the client object Natural representative of Update to client object ;

[0032] (5) Identifying natural density peaks: After updating the natural representatives, those client objects that select themselves as natural representatives are marked as natural density peaks. These natural density peak objects will serve as core nodes in the subsequent clustering process.

[0033] In the further technical solution, the specific process of Step 3 is as follows:

[0034] (1) Determine the member set of the natural density peak: After the natural representative is updated, each natural density peak is composed of clients that select the natural density peak as the natural representative and whose density is higher than the set density threshold. The density threshold is used to filter out high-density objects and exclude low-density and noise objects.

[0035] (2) Calculate the neighbor set of the natural density peak: The neighbor set of each natural density peak p A collection of its members The natural neighbors of objects with density greater than the density threshold are composed;

[0036] (3) Calculate the common neighbor set between the natural density peaks: two natural density peaks p and The neighbor distance between is calculated as follows:

[0037] ;

[0038] in, represents the natural density peak p and The neighbor distance between represents p and The Euclidean distance between Represents the value p and The set of common neighbors between express The size of , i.e. the number of common neighbors; express The sum of the natural densities of all objects in ; Indicates all The maximum Euclidean distance of represents the product;

[0039] (4) Calculate graph distance through minimum spanning tree: A minimum spanning tree based on neighbor distance is used to represent the connection relationship between natural density peaks. In the minimum spanning tree, the nodes represent natural density peaks and the weight of the edges is the neighbor distance. Assume that the natural density peak p on a minimum spanning tree is The path is , is the starting point p, It is the end ; then the natural density peak p and The distance between the graphs It is expressed as:

[0040] ;

[0041] in, is the natural density peak p to All natural density peak nodes on the shortest path, , is the distance between adjacent natural density peaks.

[0042] In the further technical solution, the specific process of Step 5 is as follows:

[0043] The new center calculation formula is:

[0044] ;

[0045] Among them, c i It is clustering The new center of is the graph distance between the natural density peaks p and x;

[0046] Each point n is assigned a distance from its graph The smallest cluster center c i The member allocation formula is:

[0047] ;

[0048] in, Represents the cluster index to which point n belongs. Indicates the number of clusters required.

[0049] In the above scheme, in step 4, the threshold interval of the model deviation degree is expressed as , μ is The mean of the norm, σ is The standard deviation of the norm.

[0050] In the above scheme, in step 4, for the group whose model deviation does not exceed the threshold, the local model of the client in this group will be aggregated with the current global model using an asynchronous update method. The aggregation formula is as follows:

[0051] ;

[0052] in, is the aggregated global model parameter, is the number of clients in this client group, It is The weight of the client, It is Model parameters for each client;

[0053] After the aggregation is completed, all client models of the group are updated to the latest global model. The update formula is as follows:

[0054] ;

[0055] in, It is The model parameters of each client, is the current latest global model parameter.

[0056] In the above scheme, in step 4, when the local models of all group clients are updated synchronously, the weight of the normal client is calculated in the following way:

[0057] ;

[0058] in, Is a positive scaling parameter used to adjust the weight for The sensitivity of the norm value, Is the client The degree of deviation from the global model, It is The weight of the client.

[0059] In the above scheme, during the model training process, the steps to identify malicious clients are as follows:

[0060] Step 1: Based on the parameter differences between the client local model and the global model Norm, which measures the maximum difference between two model parameter vectors, is given by:

[0061] ;

[0062] in, Represents the relationship between the client local model and the global model The norm difference, i.e. the maximum parameter difference, x =[ x 1 , x 2 , x 3 , … , x n ] represents the parameter vector of the client model, y =[ y 1 , y 2 , y 3 , … , y n ] represents the parameter vector of the global model, is the dimension of the parameter vector;

[0063] Step 2: Determine the client's local model and the global model Whether the norm exceeds the threshold, if it exceeds the threshold, it is a normal client, otherwise it is a malicious client;

[0064] Step 3: When the malicious client aggregates the model, the new weight is calculated according to the weight decay formula, and the new global model is aggregated with other client models in the group and the current global model; the weight decay formula is as follows:

[0065] ;

[0066] in, is the adjusted client weight, is the client weight before adjustment, α is the adjustment coefficient, which is used to control the attenuation strength. Is the client The model update of the global model is different from the parameter difference of Norm.

[0067] Through the above technical solution, the semi-asynchronous federated learning method based on blockchain provided by the present invention has the following beneficial effects:

[0068] (1) Improve model training efficiency: The present invention uses a clustering algorithm to group devices according to computing power, communication capabilities, and model deviation, assigns devices with similar capabilities to the same group, and adopts a synchronous update method within the group. This not only ensures the coordination and data consistency of devices within the group, but also reduces the waiting time between devices. Compared with the traditional global synchronous update method, the present invention significantly improves the training efficiency of the model.

[0069] (2) Enhance the robustness and security of the system: The present invention adopts Norm identifies malicious clients and can effectively detect and remove malicious devices that upload abnormal model updates, reduce interference with the global model, and ensure the accuracy and robustness of the model. This will help improve the security of the entire federated learning system and prevent malicious attacks from affecting model training results.

[0070] (3) Solving the “outdated” gradient problem in asynchronous federated learning: This invention introduces As a distinguishing indicator between synchronous and asynchronous updates, the norm can timely judge the rationality of the model update between groups. When the norm exceeds the set range, the system stops the current model training and synchronously uses the previous global model to ensure the directionality and convergence of the global model, thereby avoiding the negative impact of "outdated" gradients on model performance.

[0071] (4) Improve resource utilization efficiency: The present invention uses asynchronous federated learning between groups, allowing different devices to perform asynchronous model updates and uploads based on their own computing speeds and network conditions, reducing resource waste. This update mechanism fully considers the differences in device heterogeneity and greatly improves resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0073] Figure 1 A flowchart of a semi-asynchronous federated learning method based on blockchain disclosed in an embodiment of the present invention;

[0074] Figure 2 This is the NDP-Kmeans clustering flow chart. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0076] The present invention provides a semi-asynchronous federated learning method based on blockchain, comprising the following steps:

[0077] Step 1: Release the task and distribute the model:

[0078] The task publisher creates a global model A (such as a neural network model or other machine learning model) and publishes it to the blockchain network, and initializes it according to the preset initial parameters. Subsequently, the global model A is distributed to all participating clients. After each client receives the global model, it uses it as the starting point for local training.

[0079] Step 2: Cluster the devices based on the initial centralized training:

[0080] The client extracts the global model A from the blockchain network and trains it on local data to generate updated model parameters; the client packages these parameters into transactions and uploads the transactions to the blockchain network. After receiving these transactions, the smart contract automatically aggregates the model parameters of each client to generate a new global model B and packages it into a new block; at the same time, the client also saves their basic information, such as computing power time overhead, communication power time overhead, and model deviation degree, into the same clustering round file to provide basic data for subsequent clustering grouping.

[0081] Specifically, the computing power and communication capabilities of the client cannot be directly reflected, so time overhead is used to replace computing and communication capabilities. After receiving the global model, it is recorded as time , the model training time starts at , model training end time , the time to package and upload the model to the blockchain network is , For Clients The total time spent, including computing power time overhead and communication power time overhead; For Clients The time spent training the model, i.e. the computing power time overhead.

[0082] The degree of model deviation is calculated by comparing the global model with the client's local model. norm, The norm is a measure of the difference between model parameter vectors and is calculated as:

[0083] ;

[0084] in, Represents the relationship between the client local model and the global model norm, that is, the degree of difference between the two models, x =[ x 1 , x 2 , x 3 , … , x n ] represents the parameter vector of the client model, y =[ y 1 , y 2 , y 3 , … , y n ] Represents the parameter vector of the global model, and n is the dimension of the parameter vector.

[0085] Counting the deviation degree of all clients and using it as a feature of the data during clustering is conducive to the stability of model convergence.

[0086] Step 3, clustering grouping:

[0087] The NDP-Kmeans clustering algorithm is used to perform clustering and grouping according to the basic information of the clients stored in the clustering round file; after the grouping is completed, the global model B is distributed to each client group.

[0088] The specific method is as follows:

[0089] Step 1: Use the basic information saved during client training as the clustering data set. The clustering data set consists of the client's computing power time overhead, communication power time overhead, and model deviation degree. These data reflect the client's computing power, communication power, and model deviation degree, respectively.

[0090] Step 2: Define the density of client objects and use natural neighbors to calculate the density. Natural neighbors are obtained by adaptively searching for nearest neighbors and recording reverse neighbor information.

[0091] (1) Calculate the client object density: Based on the nearest neighbor set The density of each client object is calculated based on the distance. The shorter the distance, the higher the density. Client objects with high density are more concentrated in the performance space. , is the set of all client objects, whose density The calculation is as follows:

[0092] ;

[0093] in, Set to or The maximum value of It is the natural eigenvalue that can adaptively determine the number of nearest neighbors. Represents a client object The number of reverse neighbors, Is the client object of A set of nearest neighbor objects, Is the client object and The Euclidean distance between .

[0094] Here, the computing power time overhead and communication power time overhead will affect , because these overheads reflect the differences in the time required for training and communication between clients. If the computational power and communication capabilities of two client objects are very close, then the distance between them in the performance space will be smaller, resulting in Smaller, thus increasing its density .

[0095] (2) Determine natural neighbors: Natural neighbors are defined based on the performance similarity of client objects. For each client object, a set of nearest neighbor objects are selected as its natural neighbors, which are similar to the client object in computing and communication capabilities. , its natural neighbor set It is The nearest neighbor object set .

[0096] (3) Selecting a natural representative: The natural representative of each client object is the object with the highest density among itself or its natural neighbors. The natural representative usually has high computing and communication capabilities and is a local density peak. Natural representative of yes and the object with the highest density among its natural neighbors. Since density is affected by performance indicators, clients with high computing power and high communication capabilities are more likely to become natural representatives of other clients, which means they are more likely to appear as central nodes in the clustering process.

[0097] (4) Update the natural representative chain: By updating the natural representative, the natural representative chain of the client object can be continued, and those client objects with strong computing and communication capabilities are given priority as natural representatives. Natural representative of Is the client object and Natural representative of Is the client object , then the client object Natural representative of Update to client object .

[0098] (5) Identify Natural Density Peaks (NDP): After updating the natural representatives, those client objects that select themselves as natural representatives are marked as natural density peaks. These NDP objects will serve as core nodes in the subsequent clustering process.

[0099] Step 3: Use neighbor-based distance to calculate the graph distance between natural density peaks. The graph distance combines the performance data of the client, making the graph distance between clients with high computing power, high communication capacity and low model offset smaller;

[0100] (1) Determine the member set of the natural density peak: After updating the natural representative, each natural density peak consists of clients that choose the natural density peak as the natural representative and whose density is higher than the set threshold. The density threshold is used to filter out high-density objects and exclude low-density and noise objects. High-density objects are more likely to be located in areas with similar performance, thereby reducing inconsistency within the group and ensuring that the computing and communication capabilities of the clients within the group are more balanced.

[0101] (2) Calculate the neighbor set of the natural density peak: The neighbor set of each natural density peak p A collection of its members The natural neighbors of objects with density greater than the density threshold. Objects in the neighbor set need to meet certain density thresholds, which are directly affected by the client's computing power and communication capabilities. Therefore, clients with similar performance will gather in the neighbor set.

[0102] (3) Calculate the common neighbor set between natural density peaks: natural density peak p and The neighbor distance between two natural density peaks is not only based on the Euclidean distance between them, but also takes into account the number of their common neighbors and the density of these neighbors. When two natural density peaks have more common neighbors and higher density, the neighbor distance between them will be shorter, and vice versa. This distance calculation method can more accurately reflect the similarity of clients in performance.

[0103] The two natural density peaks p and The neighbor distance between is calculated as follows:

[0104] ;

[0105] in, represents the natural density peak p and The neighbor distance between represents p and The Euclidean distance between Represents the value p and The set of common neighbors between express the size of (i.e. the number of common neighbors), express The sum of the natural densities of all objects in . Indicates all The maximum Euclidean distance, represents the product. The neighbor distance here is not only based on physical or spatial distance ,The density of clients and the number of common neighbors are also considered.,Clients with similar performance are more likely to be clustered together,due to their smaller distance and higher density, and thus, their neighbors,are closer.

[0106] (4) Calculate graph distance through minimum spanning tree: A minimum spanning tree based on neighbor distance is used to represent the connection relationship between natural density peaks. In the minimum spanning tree, the nodes represent natural density peaks and the weight of the edges is the neighbor distance. In order to calculate the graph distance between two natural density peaks, it is necessary to find their shortest path on the minimum spanning tree and add the neighbor distances between all adjacent nodes on the path. Assume that a natural density peak p on the minimum spanning tree is The path is ( is the starting point p, It is the end ) then the natural density peak p and The distance between the graphs It can be expressed as:

[0107] ;

[0108] in, is the natural density peak p to All natural density peak nodes on the shortest path, , is the distance between adjacent natural density peaks. This graph distance reflects the performance similarity between clients, making the distance between clients with similar performance shorter and easier to be clustered together.

[0109] Step 4: Select the natural density peak with the largest density as the first center, and then select the natural density peak with the largest graph distance from the selected centers as the new center until the required number of centers is reached; when selecting new centers, give priority to clients with strong computing power, high communication capabilities, and low model deviation, because these clients are more likely to play a key role in the subsequent federated learning process.

[0110] Step 5: Calculate the member allocation of each group based on the graph distance, so that the distance between clients in the group is smaller.

[0111] The new center calculation formula is:

[0112] ;

[0113] Among them, c i It is clustering Here, the selection of new centers not only considers the graph distance, but also combines the performance indicators of the clients, making high-performance clients more likely to become cluster centers.

[0114] Each point n is assigned a distance from its graph The smallest cluster center c i The member allocation formula is:

[0115] ;

[0116] in, Represents the cluster index to which point n belongs. The final allocation of members is based on the overall performance of the client, making the collaboration between clients in the group more efficient in computing and communication.

[0117] Step 4: The local model of the client in the group is updated asynchronously or synchronously with the global model in the latest block:

[0118] The clients in the group use the global model B to update the model on the local data. During the training, the basic information of the clients is recorded in different clustering round files. For the groups whose model deviation does not exceed the threshold, the local models of the clients in this group are aggregated and updated with the current global model using an asynchronous update method and redistributed to the clients in the current group. If the deviation of the local model of a group's clients exceeds the threshold, the training of all the clients in the group is stopped, and the model is not aggregated. The local models of all the clients in the group are directly synchronized and updated to the current global model, and all the clients are clustered again according to the latest clustering round file, and then a new clustering round file is created.

[0119] Specifically, we first need to set the normal range, where we use the normal device The mean (μ) and standard deviation (σ) of the norm are used to represent the threshold interval of the model deviation degree. .

[0120] For groups whose model deviation does not exceed the threshold, the local models of the group's clients will be aggregated with the current global model using an asynchronous update method. The aggregation formula is as follows:

[0121] ;

[0122] in, is the aggregated global model parameter, is the number of clients in this client group, It is The weight of the client, It is Model parameters for each client;

[0123] After the aggregation is completed, all client models of the group are updated to the latest global model. The update formula is as follows:

[0124] ;

[0125] in, It is The model parameters of each client, is the current latest global model parameter.

[0126] When the local models of all group clients are updated synchronously, the weight of normal clients is calculated as follows:

[0127] ;

[0128] in, Is a positive scaling parameter used to adjust the weight for The sensitivity of the norm value, Is the client The degree of deviation from the global model, It is The weight of the client.

[0129] Step 5, consensus on the latest block model:

[0130] After each round of model update is completed, the latest global model is packaged into a block and uploaded to the blockchain network; after the latest block is uploaded to the blockchain network, all participating nodes verify the validity of the block through the consensus mechanism and reach a consensus on it; after reaching a consensus, the block will be officially added to the blockchain network and become the only recognized model version in the system; this latest global model will be used by the client group in the next round of training to ensure training and updating based on the same model.

[0131] In addition to the updated global model parameters, the block also records relevant information about this round of training, such as the allocation of client groups, the model update method (synchronous or asynchronous), and other metadata. This information helps maintain the transparency and traceability of the training process, and provides data support for the subsequent consensus mechanism.

[0132] Step 6, model convergence:

[0133] Repeat steps 4 and 5 and regularly check the performance indicators of the model and the progress of the training process until the global model converges and reaches the expected performance indicators (showing sufficient prediction accuracy, robustness, etc.), or the training process reaches the preset maximum number of rounds.

[0134] During the model training process, it is inevitable to encounter the problem of malicious clients uploading malicious parameters. The steps to identify malicious clients are as follows:

[0135] Step 1: Based on the parameter differences between the client local model and the global model Norm, which measures the maximum difference between two model parameter vectors, is given by:

[0136] ;

[0137] in, Represents the relationship between the client local model and the global model The norm difference, i.e. the maximum parameter difference, x =[ x 1 , x 2 , x 3 , … , x n ] represents the parameter vector of the client model, y =[ y 1 , y 2 , y 3 , … , y n ] represents the parameter vector of the global model, is the dimension of the parameter vector;

[0138] Step 2: Determine the client's local model and the global model Whether the norm exceeds the set threshold, if it exceeds the threshold, it is a normal client, otherwise it is a malicious client;

[0139] Step 3: When the malicious client aggregates the model, the new weight is calculated according to the weight decay formula, and the new global model is aggregated with other client models in the group and the current global model; the weight decay formula is as follows:

[0140] ;

[0141] in, is the adjusted client weight, is the client weight before adjustment, α is the adjustment coefficient, which is used to control the attenuation strength. Is the client The model update of the global model is different from the parameter difference of Norm.

[0142] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A semi-asynchronous federated learning method based on blockchain, characterized in that: The steps include: Step 1: The task publisher creates a global model A and publishes it to the blockchain network, and initializes it according to the preset initial parameters; Step 2: The client extracts the global model A from the blockchain network and trains it on local data to generate updated model parameters. The client packages these model parameters and uploads them to the blockchain network. After receiving them, the smart contract automatically aggregates the model parameters of each client to generate a new global model B and packages it into a new block. At the same time, the clients also save their basic information into the same clustering round file to provide basic data for subsequent clustering grouping. Step 3: Use the NDP-Kmeans clustering algorithm to perform clustering grouping according to the basic information of the clients stored in the clustering round file; After the grouping is completed, the global model B is distributed to each client group; Step 4: The clients in the group use the global model B to perform model updates on local data. During training, the basic information of the clients is recorded in different clustering round files. For groups whose model deviation does not exceed the threshold, the local models of the clients in this group are aggregated and updated with the current global model using an asynchronous update method, and redistributed to the clients in the current group. If the local model deviation of a client in a group exceeds the threshold, the training of all the clients in the group is stopped, and the model is not aggregated. The local models of all the clients in the group are directly synchronized and updated to the current global model, and all the clients are clustered again according to the latest clustering round file, and a new clustering round file is created. Step 5: After each round of model update is completed, the latest global model is packaged into a block and uploaded to the blockchain network. After the latest block is uploaded to the blockchain network, all participating nodes verify the validity of the latest block through the consensus mechanism and reach an agreement on it. After reaching a consensus, the block will be officially added to the blockchain network and become the only recognized model version in the system. This latest global model will be used by the client group in the next round of training. Step 6: Repeat steps 4 and 5 until the global model converges and reaches the expected performance index, or the training process reaches the preset maximum number of rounds; In step 2, the basic information of the client includes computing power time overhead, communication power time overhead, and model deviation degree; Client After receiving the global model, it is recorded as time , the model training time starts at , model training end time , the time to package and upload the model to the blockchain network is , For Clients The total time spent, including computing power time overhead and communication power time overhead; For Clients The time it takes to train the model, i.e. the computing power time overhead; The degree of model deviation is calculated by comparing the global model with the client's local model. The norm is used to express, The norm is a measure of the difference between model parameter vectors and is calculated as: ； in, Represents the relationship between the client local model and the global model norm, that is, the degree of difference between the two models, represents the parameter vector of the client model, represents the parameter vector of the global model, n is the dimension of the parameter vector; The specific method of step 3 is as follows: Step 1: Use the basic information saved during client training as the clustering data set. The clustering data set consists of the client's computing power time overhead, communication power time overhead, and model deviation degree. Step 2: Define the density of client objects and use natural neighbors to calculate the density. Natural neighbors are obtained by adaptively searching for nearest neighbors and recording reverse neighbor information. Step 3: Use neighbor-based distance to calculate the graph distance between natural density peaks. The graph distance combines the performance data of the client, making the graph distance between clients with high computing power, high communication capacity and low model offset smaller; Step 4: Select the natural density peak with the largest density as the first center, and then select the natural density peak with the largest graph distance from the selected centers as the new center until the required number of centers is reached; during the center selection process, clients with high computing power and high communication capabilities are given priority; Step 5: Calculate the member allocation of each group based on the graph distance, so that the distance between clients in the group is smaller.

2. According to claim 1, a semi-asynchronous federated learning method based on blockchain is characterized in that: The specific process of Step 2 is as follows: (1) Calculate the client object density: For client objects , is the set of all client objects, whose density The calculation is as follows: ； in, Set to or The maximum value of It is the natural eigenvalue that can adaptively determine the number of nearest neighbors. Represents a client object The number of reverse neighbors, Is the client object of A set of nearest neighbor objects, Is the client object and The Euclidean distance between (2) Determine natural neighbors: For each client object, select a set of nearest neighbor objects as its natural neighbors. These neighbors are similar to the client object in computing and communication capabilities. , its natural neighbor set It is The nearest neighbor object set ; (3) Selecting a natural representative: The natural representative of each client object is the object with the highest density among itself or its natural neighbors. Natural representative of yes the object with the highest density among its natural neighbors; (4) Update the natural representation chain: By updating the natural representation, the natural representation chain of the client object can continue; if the client object Natural representative of Is the client object ,and Natural representative of Is the client object , then the client object Natural representative of Update to client object ; (5) Identifying natural density peaks: After updating the natural representatives, those client objects that select themselves as natural representatives are marked as natural density peaks. These natural density peak objects will serve as core nodes in the subsequent clustering process.

3. According to a blockchain-based semi-asynchronous federated learning method according to claim 1, it is characterized in that: The specific process of Step 3 is as follows: (1) Determine the member set of the natural density peak: After the natural representative is updated, each natural density peak is composed of clients that select the natural density peak as the natural representative and whose density is higher than the set density threshold. The density threshold is used to filter out high-density objects and exclude low-density and noise objects. (2) Calculate the neighbor set of the natural density peak: The neighbor set of each natural density peak p A collection of its members The natural neighbors of objects with density greater than the density threshold are composed; (3) Calculate the common neighbor set between the natural density peaks: two natural density peaks p and The neighbor distance between is calculated as follows: ； in, represents the natural density peak p and The neighbor distance between represents p and The Euclidean distance between Represents the value p and The set of common neighbors between express The size of , i.e. the number of common neighbors; express The sum of the natural densities of all objects in ; Indicates all The maximum Euclidean distance of represents the product; (4) Calculate graph distance through minimum spanning tree: A minimum spanning tree based on neighbor distance is used to represent the connection relationship between natural density peaks. In the minimum spanning tree, the nodes represent natural density peaks and the weight of the edges is the neighbor distance. Assume that the natural density peak p on a minimum spanning tree is The path is , is the starting point p, It is the end ; then the natural density peak p and The distance between the graphs It is expressed as: ； in, is the natural density peak p to All natural density peak nodes on the shortest path, , is the distance between adjacent natural density peaks.

4. According to a blockchain-based semi-asynchronous federated learning method according to claim 1, it is characterized in that: The specific process of Step 5 is as follows: The new center calculation formula is: ； Among them, c i It is clustering The new center of is the graph distance between the natural density peaks p and x; Each point n is assigned a distance from its graph The smallest cluster center c i The member allocation formula is: ； in, Represents the cluster index to which point n belongs. Indicates the number of clusters required.

5. According to claim 2, a semi-asynchronous federated learning method based on blockchain is characterized in that: In step 4, the threshold interval of the model deviation degree is expressed as , μ is The mean of the norm, σ is The standard deviation of the norm.

6. A blockchain-based semi-asynchronous federated learning method according to claim 1, characterized in that: In step 4, for the group whose model deviation does not exceed the threshold, the local model of the client in this group will be aggregated with the current global model using an asynchronous update method. The aggregation formula is as follows: ； in, is the aggregated global model parameter, is the number of clients in this client group, It is The weight of the client, It is Model parameters for each client; After the aggregation is completed, all client models of the group are updated to the latest global model. The update formula is as follows: ； in, It is The model parameters of each client, is the current latest global model parameter.

7. A blockchain-based semi-asynchronous federated learning method according to claim 1, characterized in that: In step 4, when the local models of all group clients are updated synchronously, the weight of normal clients is calculated as follows: ； in, Is a positive scaling parameter used to adjust the weight for The sensitivity of the norm value, Is the client The degree of deviation from the global model, It is The weight of the client.

8. According to a blockchain-based semi-asynchronous federated learning method according to claim 1, it is characterized in that: During the model training process, the steps to identify malicious clients are as follows: Step 1: Based on the parameter differences between the client local model and the global model Norm, which measures the maximum difference between two model parameter vectors, is given by: ； in, Represents the relationship between the client local model and the global model norm difference, that is, maximizing the parameter difference, represents the parameter vector of the client model, represents the parameter vector of the global model, is the dimension of the parameter vector; Step 2: Determine the client's local model and the global model Whether the norm exceeds the threshold, if it exceeds the threshold, it is a normal client, otherwise it is a malicious client; Step 3: When the malicious client aggregates the model, the new weight is calculated according to the weight decay formula, and the new global model is aggregated with other client models in the group and the current global model; the weight decay formula is as follows: ； in, is the adjusted client weight, is the client weight before adjustment, α is the adjustment coefficient, which is used to control the attenuation strength. Is the client The model update of the global model is different from the parameter difference of Norm.

Images