Blockchain-based Federated Learning Model Training Method and System

Abstract

The present application provides a blockchain-based federated learning model training method and system, the method comprising a client performing multiple rounds of iterative training on a local model, and performing the following operations for each round of iteration: the client determines multiple remote sensing images and obtains global parameters; the client trains the local model; the client generates and sends first transaction information to other clients; the client receives second transaction information; determines a first target client; the first target client determines a second target client; each second target client calculates a corresponding new global parameter; determines a third target client; the third target client generates a new block, and stores the new global parameters corresponding to the third target client in the new block until the local model meets the convergence conditions, thereby solving the technical problem in the prior art that the introduction of blockchain in the training process of the federated learning model leads to an increase in training delay, and improving the training efficiency of the federated learning model.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular, to a method and system for training a federated learning model based on a blockchain. Background Art

[0002] In the traditional federated learning process, each client does not transmit local data, but only transmits the trained local parameters to the server. The server determines the global parameters by receiving the local parameters sent by each client, and then feeds back the global parameters to each client for the next round of training. Traditional federated learning mainly relies on the server to determine the global parameters. Once the server is attacked, a large amount of parameter information will be leaked, and the federated learning model may also be tampered with, making the subsequent application of the trained federated learning model inaccurate and causing losses to the application party.

[0003] In this case, the blockchain becomes an effective solution to the attack problem, coordinating the federated learning process through decentralization. Although the blockchain improves the security of federated learning, the joint deployment of federated learning and the blockchain will bring additional computing and communication overheads. The distributed nature of the blockchain requires each node to store integrity data and participate in the verification and consensus process. Therefore, the combination of federated learning and the blockchain will additionally increase a very large amount of computing overhead, further damaging the training efficiency of federated learning and resulting in an increase in the training delay of federated learning. Summary of the Invention

[0004] In view of this, the purpose of the present application is to propose a method and system for training a federated learning model based on a blockchain to overcome all or part of the deficiencies in the prior art.

[0005] For the above purposes, the present application provides a method for training a federated learning model based on a blockchain, which is applied to a federated learning system based on a blockchain. The blockchain federated learning system includes multiple clients and at least one block. Each client deploys a local model, and the block is used to store global parameters associated with it. The method includes: The client performs multiple rounds of iterative training on the local model associated with the client. For each round of the multiple rounds of iterative training, the following operations are performed: The client determines multiple remote sensing images and obtains the global parameters in the latest generated block; Based on the multiple remote sensing images and the global parameters, the client trains the local model to obtain local parameters; The client generates first transaction information associated with the local parameters and sends the first transaction information to other randomly selected clients among the multiple clients; The client receives second transaction information sent by other clients among the multiple clients; Based on all the first transaction information and all the second transaction information, a first target client is determined from the multiple clients; The first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information; Based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates the corresponding new global parameters; The client that first calculates the new global parameters among the multiple second target clients is used as the third target client; The third target client generates a new block and stores the new global parameters corresponding to the third target client in the new block until the local model meets the convergence condition.

[0006] Optionally, the determining of the first target client from the multiple clients based on all the first transaction information and all the second transaction information includes: The client determines the credibility of the first transaction information and the credibility of the second transaction information associated with it; Based on the multiple credibilities associated with each client, the client selects the client whose total number of determined credibilities is first greater than the preset number from the multiple clients as the first target client.

[0007] Optionally, the first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information, including: Based on the multiple credibilities associated with the first target client, the first target client sorts the multiple credibilities in descending order, and determines the clients corresponding to the top N credibilities after sorting as the second target clients that meet the second preset condition.

[0008] Optionally, the client determines multiple credibility degrees, including determining the credibility degree through the following formula: where is the credibility degree, e is the preset base number, s is the total number of current blocks in the blockchain federated learning system, is the total number of times that client i is the second target client, v x represents whether client i participated in the generation of blocks in the previous iteration. If it participated, the value is 1; if it did not participate, the value is 0. Mal x represents whether the credibility degree of client i in the previous iteration is lower than the preset threshold. If it is lower, the value is 0; if it is greater than or equal to, the value is 1. η is the adjustment coefficient, and x represents the iteration round of the previous round.

[0009] Optionally, after the third target client stores the new global parameter corresponding to the third target client in the new block, the method includes: each second target client increments the associated identification value by one, where the identification value is used to represent the number of times the client is determined as the second target client; the third target client generates and broadcasts a credibility degree update instruction; in response to receiving the credibility degree update instruction, based on the first transaction information, the second transaction information, and the credibility degree update instruction, each second target client other than the third target client updates the multiple credibility degrees associated with it, and sends the credibility degree corresponding to the second transaction information in the updated multiple credibility degrees to the associated client.

[0010] Optionally, each transaction information further includes the local parameter of the client associated with it; based on the first transaction information, the second transaction information, and the global parameter, each second target client calculates the corresponding new global parameter, including determining the new global parameter through the following formula: where W (t) is the new global parameter, β is the weighting coefficient of the local model associated with the second target client, γ i is the correlation measurement value of the local model of the second target client i determined based on the first transaction information and the second transaction information, w i is the local parameter of the current iteration round of the second target client i determined based on the first transaction information and the second transaction information, E is the total number of second target clients participating in the t-th iteration training, W (t-1) is the global parameter, and t represents the current iteration round.

[0011] Optionally, it further includes: determining the weighting coefficient of each local model through the following formula: Among them, β is the weighting coefficient of the local model, k is the adjustment coefficient between the local parameters of the second target client in the t-th round and the local parameters in the (t - 1)-th round, Δt is the aggregation time difference between the aggregation of the local model of the second target client in the t-th round and the aggregation of the local model in the (t - 1)-th round, d is the Euclidean distance between the local parameters obtained by the second target client i in the t-th round and the local parameters obtained in the (t - 1)-th round, and e is the preset base number.

[0012] Optionally, the client determines a plurality of remote sensing images, including: the client receives remote sensing data, performs imaging processing on the remote sensing data by using two-dimensional matched filtering to obtain a plurality of initial remote sensing images, and normalizes the pixel values of each initial remote sensing image to obtain the plurality of remote sensing images.

[0013] Optionally, before the client trains the local model, the method includes: in response to determining that the number of the plurality of remote sensing images is less than or equal to a preset number threshold, based on the plurality of remote sensing images, the client sequentially inputs the plurality of remote sensing images into a trained image generation model, and the image generation model sequentially outputs new remote sensing images until the sum of the number of the plurality of remote sensing images and the new remote sensing images is greater than the preset number threshold, wherein the noise information of the remote sensing image and its associated new remote sensing image is different.

[0014] Based on the same inventive concept, the present application also provides a blockchain-based federated learning system, which includes multiple clients and at least one block. Each client deploys a local model, and the block is used to store the global parameters associated with it. The client is used to perform multiple rounds of iterative training on the local model associated with the client. For each round of the multiple rounds of iterative training, the following operations are performed: determine multiple remote sensing images, and obtain the global parameters in the latest generated block; based on the multiple remote sensing images and the global parameters, train the local model to obtain local parameters; generate first transaction information associated with the local parameters, and send the first transaction information to other randomly selected clients among the multiple clients; receive second transaction information sent by other clients among the multiple clients; based on all the first transaction information and all the second transaction information, determine a first target client from the multiple clients; the first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information; based on the first transaction information, the second transaction information and the global parameters, each second target client calculates and obtains a corresponding new global parameter; use the client among the multiple second target clients that first calculates the new global parameter as the third target client; the third target client generates a new block and stores the new global parameter corresponding to the third target client in the new block until the local model meets the convergence condition.

[0015] As can be seen from the above, for the blockchain-based federated learning model training method and system provided in this application, the method includes the client performing multiple rounds of iterative training on the local model associated with the client, and for each round of the multiple rounds of iterative training, the following operations are performed: the client determines multiple remote sensing images and obtains the global parameters in the latest generated block; based on the multiple remote sensing images and the global parameters, the client trains the local model to obtain local parameters; the client generates first transaction information associated with the local parameters and sends the first transaction information to other random clients among the multiple clients; the client receives second transaction information sent by other clients among the multiple clients, and exchanges transaction information among the clients, so as to subsequently screen out clients for determining global parameters based on each transaction information. Based on all the first transaction information and all the second transaction information, a first target client is determined from the multiple clients; the first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information. By screening the clients, the latency required to train the completed federated learning model is further reduced. Based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates the corresponding new global parameters; the client among the multiple second target clients that first calculates the new global parameters is used as the third target client, which improves the efficiency of determining the new global parameters and thus reduces the training latency of the local model. The third target client generates a new block and stores the new global parameters corresponding to the third target client in the new block, so that a new federated learning process can be started based on the global parameters of any block, which improves the training efficiency of the local model. Until the local model meets the convergence condition, the trained local model can be subsequently applied to the classification of remote sensing images to improve the accuracy of remote sensing image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions in this application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only the embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a flowchart of the blockchain-based federated learning model training method according to an embodiment of this application;

[0018] Figure 2 It is a structural diagram of the blockchain-based federated learning system according to an embodiment of this application. Detailed implementation manners

[0019] To make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to specific embodiments and the accompanying drawings.

[0020] It should be noted that unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the ordinary meanings understood by those of ordinary skill in the art to which the present application belongs. The "first", "second" and similar terms used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0021] As described in the background art section, with the further development of artificial intelligence, the application of integrated machine learning has promoted the joint training among multiple clients, solving the problems of less data and low diversity of individual clients. In the research on the joint training of each client, most are designed based on the traditional centralized computer architecture, and the server is responsible for coordinating the communication and computing between different clients. In the above architecture, the client needs to upload local data to the server, and the server centrally processes all the data and returns the updated model parameters. However, there are some problems in the above architecture. First, uploading local data to the server may involve privacy issues, and the client may not be willing to transmit sensitive information to an untrusted server. Second, the server needs to process a large amount of data, which may lead to processing delays and data security problems. Finally, the server may become the bottleneck of the entire system, limiting the scalability of the above architecture.

[0022] With the increasing emphasis on local data privacy by clients and their unwillingness to share local data in integrated machine learning, federated learning has been proposed. In the traditional federated learning process, each client does not transmit local data, but only transmits the trained local model parameters to the server. The server determines the global parameters by receiving the local parameters sent by each client, and then feeds back the global parameters to each client for the next round of training. Traditional federated learning mainly relies on the server to determine the global parameters. Once the server is attacked, a large amount of parameter information will be leaked, and the federated learning model may also be tampered with, making the subsequent application of the trained federated learning model inaccurate and causing losses to the application party.

[0023] In this case, the blockchain becomes an effective solution to the attack problem, coordinating the federated learning process through decentralization. Although the blockchain improves the security of federated learning, the joint deployment of federated learning and the blockchain will bring additional computational and communication overheads. The distributed nature of the blockchain requires each node to store integrity data and participate in the process of verification and consensus. Therefore, the combination of federated learning and the blockchain will add a very large additional computational overhead, further impairing the training efficiency of federated learning and increasing the training latency of federated learning.

[0024] In view of this, the embodiments of the present application propose a method for training a federated learning model based on the blockchain, referring to Figure 1 , which is applied to a federated learning system based on the blockchain. The blockchain federated learning system includes multiple clients and at least one block. Each client deploys a local model, and the block is used to store the global parameters associated with it; the method includes the following steps:

[0025] Step 101, the client performs multiple rounds of iterative training on the local model associated with the client. For each round of the multiple rounds of iterative training, the following operations are performed: the client determines multiple remote sensing images and obtains the global parameters in the latest generated block.

[0026] In this step, remote sensing images have the characteristics of rich image information, complex scene composition, and details of ground object elements. It is very important to accurately extract the features of remote sensing images using the trained local model. Therefore, it is necessary to perform multiple rounds of iterative training on the local model to be trained based on remote sensing images. In addition, since the areas where clients obtain remote sensing data are relatively dispersed, and the remote sensing data collected by clients may have confidentiality requirements and it is inconvenient to share data with other clients. Exemplarily, when the remote sensing data is remote sensing data in the field of military activities, each client can be carried on different types of remote sensing data collection vehicles, and at this time, the collected remote sensing data needs to be kept confidential. Therefore, each client can train the local model through the federated learning process. In order to eliminate the serious impact caused by the attack on the server in the traditional federated learning process, the present application introduces the blockchain in the federated learning process to achieve decentralization, making the federated learning process have a certain degree of security. The client performs multiple rounds of iterative training on the local model associated with it, and the trained local model can be applied to the classification of remote sensing images to ensure accurate classification results. The global parameters are stored in the block. Before the client trains the local model, it is necessary to obtain the global parameters to provide the same training direction for the local models of different clients in the future.

[0027] It should be noted that the trained local model can be applied to the classification of remote sensing images. The classification of remote sensing images is a process of identifying and classifying the surface attributes of all pixels on the remote sensing images. The purpose of classification is to obtain information such as the area and spatial distribution of various land cover types within the region based on attribute recognition. Therefore, the classification results of remote sensing images play a very important role in fields such as military activities, geological exploration, agricultural production, and urban planning. Remote sensing images have the characteristics of rich image information, complex scene composition, and details of ground object elements. Due to the above characteristics of remote sensing images, the classification results often suffer from problems such as missed classification and misclassification, resulting in low classification accuracy. Therefore, each client can achieve the purpose of classifying remote sensing images through the federated learning process.

[0028] Step 102, based on the multiple remote sensing images and the global parameters, the client trains the local model to obtain local parameters; the client generates first transaction information associated with the local parameters and sends the first transaction information to other randomly selected clients among the multiple clients; the client receives second transaction information sent by other clients among the multiple clients.

[0029] In this step, when the client obtains the global parameters, it replaces the local parameters in the local model pre-deployed on the client local. Based on the remote sensing images determined by the client, the client trains the local model after parameter replacement, that is, updates the local model by minimizing the loss function to obtain local parameters. Before each client trains its associated local model, the same global parameters are used for parameter replacement, so that each client has the same training direction without data exchange. Each client also includes an information exchange module. The information exchange module can receive the first transaction information generated by other modules built into the client and randomly send the first transaction information to other clients except itself. The client also needs to store the first transaction information and the second transaction information associated with it. The first transaction information is generated based on the local parameters associated with it. Among them, the first transaction information includes information such as the node number for generating the local parameters, the start time and duration of local model aggregation, and the number of the information exchange module. The clients exchange transaction information with each other to screen out the clients used to determine the global parameters based on each transaction information later.

[0030] It should be noted that when the client receives the second transaction information, it needs to verify it using the hash algorithm and the asymmetric encryption algorithm, which further ensures the security of training the local model. The whole process of verifying the second transaction information is as follows: The information exchange module in the client that sends the second transaction information first calculates the hash value of the second transaction information, encrypts the hash value with its own private key to generate the signature information, and then broadcasts the signature information and the second transaction information together. When the client receives the second transaction information, it decrypts the above signature information using the public key in the broadcast, then verifies whether the transmitted hash value is correct, and performs a hash operation on the transmitted second transaction information to compare whether the received hash value is equal to the transmitted hash value. If they are equal, the transmitted second transaction information is correct; otherwise, the second transaction information is discarded.

[0031] Step 103: Based on all the first transaction information and all the second transaction information, determine a first target client from the multiple clients; the first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information.

[0032] In this step, there are multiple clients in this application, and the number of initial clients can be set according to historical experience. Exemplarily, when applying the trained local model to the military activity field, the number of initial clients can be 50. In the case of having a relatively large number of clients, if each client participates in the federated learning process of determining the global parameters, it will lead to low efficiency in determining the global parameters, and further lead to a long time to obtain the trained local model. To solve the above problems, it is necessary to screen out the first target client from all the clients. However, the accuracy of the global parameters determined by only one first target client is relatively low. Therefore, the first target client also needs to determine at least one second target client that meets the preset condition from the clients corresponding to the first transaction information and the second transaction information associated with it. It should be noted that the number of second target clients is related to the number of initial clients. When the number of initial clients is relatively large, the number of second target clients is relatively large; when the number of initial clients is relatively small, the number of second target clients is relatively small. By screening the clients, the time delay required to train the completed local model is further reduced.

[0033] Step 104: Based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates its corresponding new global parameter; the client among the multiple second target clients that calculates the new global parameter first is taken as the third target client; the third target client generates a new block and stores the new global parameter corresponding to the third target client in the new block until the local model meets the convergence condition.

[0034] In this step, among the multiple second target clients that have been screened out, the client that calculates the new global parameter first is screened out and taken as the third target client. The global parameter of the third target client is the global parameter finally obtained in this round of iterative training. The third target client has a relatively fast operation speed, which improves the efficiency of determining the new global parameter, and thus reduces the training latency of the local model. The third target client is used to generate a new block and store the new global parameter so that the training result of this round of iterative training can be obtained in the next round of iteration. In addition, the reason for generating a new block is also to ensure that the global parameters obtained in each round of iterative training can be saved, so that a new federated learning process can be started based on the global parameters of any block, improving the training efficiency of the local model. At the same time, using the block to store information has the characteristics of correctness, immutability, and traceability, ensuring the security of the stored information. In response to determining that the local model meets the convergence condition, the training of the local model can be stopped, and the trained local model can be applied to the classification of remote sensing images later to improve the accuracy of remote sensing image classification.

[0035] It should be noted that after the client determines that it is the third target client, the third target client also needs to package and store the first transaction information and the second transaction information associated with it in the new block, and calculate the hash value of the above transaction information. Through the hash algorithm and the asymmetric encryption algorithm, when other second target clients except the third target client verify the above transaction information and pass, it indicates that the new block is a legal extension of the blockchain. Due to network limitations, if the communication between other second target clients except the third target client and the block may be delayed or interrupted, and these clients do not verify the above transaction information within the preset time, these clients will also generate a new block. Two new blocks are generated in this round of iterative training. To ensure the uniqueness of the new block, it is necessary to re-determine the second target client.

[0036] Through the above solution, the client performs multiple rounds of iterative training on the local model associated with the client. For each round of the multiple rounds of iterative training, the following operations are performed: The client determines a plurality of remote sensing images and obtains the global parameters in the latest generated block; based on the plurality of remote sensing images and the global parameters, the client trains the local model to obtain local parameters; the client generates first transaction information associated with the local parameters and sends the first transaction information to other randomly selected clients among the plurality of clients; the client receives second transaction information sent by other clients among the plurality of clients, and exchanges transaction information among the clients, so as to subsequently screen out the clients used to determine the global parameters based on each transaction information. Based on all the first transaction information and all the second transaction information, a first target client is determined from the plurality of clients; the first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information. By screening the clients, the latency required to train the local model to completion is further reduced. Based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates the corresponding new global parameters; the client among the plurality of second target clients that first calculates the new global parameters is used as the third target client, which improves the efficiency of determining the new global parameters, and thus reduces the training latency of the local model. The third target client generates a new block and stores the new global parameters corresponding to the third target client in the new block, so that a new federated learning process can be started based on the global parameters of any block, which improves the training efficiency of the local model. Until the local model meets the convergence condition, the trained local model can be subsequently applied to the classification of remote sensing images to improve the accuracy of remote sensing image classification.

[0037] In some embodiments, the determining, from the plurality of clients, a first target client based on all the first transaction information and all the second transaction information includes: the client determines the credibility of the first transaction information and the credibility of the second transaction information associated with it; based on the plurality of credibilities associated with each client, the client selects the client whose total number of determined credibilities is first greater than the preset number as the first target client.

[0038] In this embodiment, the client needs to calculate the credibility of the first transaction information associated with it and the credibility of the second transaction information. Here, the credibility refers to the metric value that each client in the blockchain has in real time for evaluating historical performance. The higher the credibility, the better the historical performance of the client corresponding to the credibility, and the credibility is immutable. In response to determining that the total number of credibilities determined by the client is greater than a preset number, it indicates that the computing performance of the client is good, so that the speed of calculating the credibility is fast. Selecting the client with a fast-determined credibility realizes a preliminary screening of all clients, further reducing the latency of training the local model. Exemplarily, the preset number can be 10. It should be noted that there may be a situation where the total number of credibilities determined by the client is less than or equal to the preset number, but the duration for which the client determines the determined credibility is greater than the preset determination duration. Therefore, the determination condition of the first target client can also be: selecting the client whose duration for determining the credibility is the first to be greater than the preset determination duration from multiple clients as the first target client, ensuring the rationality of the latency required for training the local model. Exemplarily, the preset determination duration can be 60 seconds.

[0039] In some embodiments, the first target client determines at least one second target client that meets a second preset condition from the clients corresponding to the first transaction information and the second transaction information, including: based on a plurality of credibilities associated with the first target client, the first target client sorts the plurality of credibilities in descending order, and determines the clients corresponding to the top N credibilities after sorting as the second target clients that meet the second preset condition.

[0040] In this embodiment, the credibility is immutable, and the credibility can reflect the historical performance of the client. The accuracy of the global parameters determined by using the client with relatively good historical performance is relatively high. Therefore, the first target client sorts the plurality of credibilities in descending order to select the top N clients with high credibilities. Here, N is a positive integer, and the determination of N is related to the number of second target clients. When the number of second target clients is relatively large, N is also relatively large; when the number of second target clients is relatively small, N is also relatively small. Exemplarily, when the number of second target clients is 10, N can be 5; when the number of second target clients is 5, N can be 3. The client with high credibility indicates that it has relatively good performance in the federated learning process of training the local model. Subsequently, using the client with relatively good performance to determine the global parameters can make the determined global parameters more accurate.

[0041] In some embodiments, the client determines a plurality of credibilities, including: determining the credibility through the following formula: Where, For the credibility, e is the preset base number, s is the total number of current blocks in the blockchain federated learning system, is the total number of times that client i is the second target client, v x indicates whether client i participated in the generation of blocks in the previous iteration. The value for participation is 1, and the value for non - participation is 0, mal x indicates whether the credibility of client i in the previous iteration is lower than the preset threshold. The value for lower than is 0, and the value for greater than or equal to is 1. η is the adjustment coefficient, and x represents the iteration round of the previous round.

[0042] In this embodiment, the credibility of the client is determined in combination with the historical performance of the client. The historical performance can be reflected by the parameters included in the first transaction information associated with it. Among them, the above - mentioned parameters can be the total number of times the client is the second target client, whether the client participated in the generation of blocks in the previous iteration, and the credibility of the client in the previous iteration, etc. The adjustment coefficient represents the penalty coefficient for clients with relatively low credibility. The larger the adjustment coefficient, the greater the penalty received by the client, and thus the credibility of the client with relatively low credibility is also relatively reduced in the current iteration training. The credibility is calculated through a specific formula, and the historical performance of the client is measured by specific values, making the determination of the credibility more accurate.

[0043] In some embodiments, after the third target client stores the new global parameters corresponding to the third target client in the new block, the method includes: each second target client increments the associated identification value by one, where the identification value is used to represent the number of times the client is determined to be the second target client; the third target client generates and broadcasts a credibility update instruction; in response to receiving the credibility update instruction, based on the first transaction information, the second transaction information, and the credibility update instruction, each second target client except the third target client updates the multiple credibilities associated with it, and sends the credibility corresponding to the second transaction information in the updated multiple credibilities to the associated client.

[0044] In this embodiment, each client increments the identification value of itself as the second client by 1 to update the information that can reflect the historical performance of the client, so as to ensure the accuracy of determining the credibility of the client in the future. The clients participating in determining the global parameters of this round can obtain different rewards according to their respective credibility scores in this round, so that the penalty coefficient is relatively reduced. In addition, since each second target client except the third target client is also selected to participate in determining the global parameters in this round of iterative training, although the finally determined global parameters are not used as the new global parameters, it is also necessary to determine multiple credibility degrees associated with the above clients, and send the credibility degree corresponding to the second transaction information in the updated multiple credibility degrees to the associated clients to update the historical performance of the above associated clients. It should be noted that the above credibility degree carries encrypted verification information, and when the client receives the credibility degree associated with it, it is also necessary to verify the credibility degree to ensure the security of the received credibility degree.

[0045] In some embodiments, each transaction information further includes the local parameters of the client associated with it; based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates the corresponding new global parameter, including: determining the new global parameter through the following formula: Where, W (t) is the new global parameter, β is the weighting coefficient of the local model associated with the second target client, γ i is the correlation metric value of the local model of the second target client i determined based on the first transaction information and the second transaction information, w i is the local parameter of the second target client i at the current iteration round determined based on the first transaction information and the second transaction information, E is the total number of second target clients participating in the t-th round of iterative training, W (t-1) is the global parameter, and t represents the current iteration round.

[0046] In this embodiment, the correlation metric value is determined through the following formula:

[0047]

[0048] Where, r i is the correlation metric value, is the mean of all local parameters in the first transaction information of the second client i, is the mean of all local parameters of the client j corresponding to the second transaction information in the second client i, Q is the number of parameters of the local model in the client, ω ip is the p-th parameter of the local model of the client i, ω jp is the p-th parameter of the local model of the client j.

[0049] On the one hand, new global parameters can be determined through the correlation metric value. On the other hand, to prevent attacks by malicious nodes and improve the communication reliability of the blockchain federated learning system, the correlation between the second target client and the client corresponding to the second transaction information it receives can be determined through the correlation metric value. Once the correlation is relatively low, it indicates that the client is malicious and is not conducive to the determination of global parameters, and should be excluded when aggregating global parameters. Exemplarily, a correlation less than 0.8 is considered relatively low. The above-mentioned correlation can be determined by the product of the correlation metric value and the quality evaluation coefficient of the local model of the client. Among them, the quality evaluation coefficient of the local model of the client is determined by the following formula:

[0050]

[0051] Among them, γ i represents the local model quality evaluation coefficient, which is used to determine the weight of the i-th node model in the current round of model aggregation, and y i represents the local model accuracy rate.

[0052] The weighted coefficient of the local model can balance the proportion of the local parameters of the previous round and the local parameters of the current round according to actual needs, making the determination of global parameters relatively flexible. Determining the new global parameters through a relatively rigorous formula ensures the accuracy of the determination of the new global parameters.

[0053] In some embodiments, it further includes: determining the weighted coefficient of each local model through the following formula: Among them, β is the weighted coefficient of the local model, k is the adjustment coefficient between the local parameters of the t-th round and the local parameters of the t - 1-th round of the second target client, Δt is the aggregation time difference between the local model aggregation of the t-th round and the local model aggregation of the t - 1-th round of the second target client, d is the Euclidean distance between the local parameters obtained in the t-th round and the local parameters obtained in the t - 1-th round of the second target client i, and e is the preset base number.

[0054] In this embodiment, during the process of aggregating and training global parameters, as the number of local iterations increases, the difference between the local parameters of the current round and the local parameters of the previous round becomes larger and larger. By calculating the Euclidean distance between the two, the training degree of the local model can be intuitively reflected. The Euclidean distance is determined by the following formula:

[0055]

[0056] Among them, d is the Euclidean distance, and x 1i represents the i-th parameter in the local parameters obtained in the t-th round, and x 2i represents the i-th parameter in the local parameters obtained in the t - 1-th round.

[0057] The above adjustment coefficient is a constant coefficient representing the relationship between the local parameters of the current round and those of the previous round. The smaller the above adjustment coefficient, the smaller the proportion of the local parameters of the current round; the larger the above adjustment coefficient, the larger the proportion of the local parameters of the current round. When determining the weighting coefficients of each local model, relatively comprehensive parameters are considered, and the weighting coefficients of each local model are determined relatively accurately, thereby making the newly determined global parameters more accurate subsequently.

[0058] In some embodiments, the client determines multiple remote sensing images, including: the client receives remote sensing data, performs imaging processing on the remote sensing data using two-dimensional matched filtering to obtain multiple initial remote sensing images, and normalizes the pixel values of each initial remote sensing image to obtain the multiple remote sensing images.

[0059] In this embodiment, synthetic aperture radar (SAR) can be used to collect remote sensing data within a preset sensing area. For example, synthetic aperture radar is used to collect radar waveform data of vehicles such as buses and trucks in a specific area, and the radar waveform data is determined as remote sensing data. When a vehicle is stationary at a certain location, the collected image data is centered on that location point. The collected remote sensing data is subjected to two-dimensional matched filtering imaging processing using a matched filter to achieve high-resolution imaging of the radar waveform data and obtain multiple initial remote sensing images. For a dual-functional radar and communication (DFRC) system with synthetic aperture radar, its remote sensing data can obtain high-resolution images through two-dimensional matched filtering, and since the generated images are not affected by illumination and climate conditions and can present targets blocked by vegetation, it has good target recognition performance. Therefore, the detection and classification tasks of radar targets can be converted into image classification and recognition tasks. The pixel values of each initial remote sensing image are normalized to make the pixel values of the initial remote sensing images uniform. Exemplarily, multiple initial remote sensing images are uniformly processed into a size of 64×64 pixel numbers, and the processed image dataset is divided into a training set and a test set according to a preset ratio, where the preset ratio can be 4:1, which facilitates subsequent image recognition and improves image training efficiency.

[0060] In some embodiments, before the client trains the local model, the method includes: in response to determining that the number of the multiple remote sensing images is less than or equal to a preset number threshold, based on the multiple remote sensing images, the client sequentially inputs the multiple remote sensing images into a trained image generation model, and the image generation model sequentially outputs new remote sensing images until the sum of the number of the multiple remote sensing images and the new remote sensing images is greater than the preset number threshold, wherein the noise information of the remote sensing image and its associated new remote sensing image is different.

[0061] In this embodiment, when the determined number of remote sensing images is relatively small, training the local model with a relatively small number of remote sensing images results in low training efficiency of the local model. Therefore, when the determined number of remote sensing images is less than the preset number threshold, a trained image generation model can be used to output new remote sensing images. The image generation model is provided with a generator, a discriminator, and a classifier, and the feature extractor and the discriminator form a generative adversarial network. The performance of remote sensing dataset classification can be improved by reducing the pressure of data collection and expanding the samples through the image generation model.

[0062] The training process of the image generation model is as follows: Initially, a first preset number of clients are set. For example, the first preset number can be 20. In each iteration of the training process, a second preset number of clients are randomly selected to participate in the aggregation of the global parameters of the image generation model. For example, the second preset number can be 7. Among them, each type of client has a third preset number of remote sensing images participating in the aggregation of the global parameters in the current round. For example, the third preset number is 12. The image generation model in each client is deployed with a generator, a discriminator, and a classifier. The image generation model uses an adversarial network based on a convolutional network as its core. The above adversarial network can improve the sample quality and the model convergence speed. The minibatch is set to 16, and each generator can have 6 convolutional layers. The image generation model first fuses the noise information of the input remote sensing image with label information through the convolutional layer of the first layer, and then performs the directional generation of new remote sensing images through a preset number of convolutional networks. Each convolutional network includes a transposed convolutional layer of ConvTranspose2d, a non-linear activation layer of ReLU, and a batch normalization layer of BatchNorm. The last convolutional network includes a transposed convolution of ConvTranspose2d and a non-linear activation output of Tanh. The discriminator can include 6 convolutional layers and 1 linear layer. First, the generated new remote sensing image is fused with information through the first convolutional layer, and then the new remote sensing image is discriminated through 4 convolutional networks. Each layer of the network includes a convolutional layer of Conv2d, a non-linear activation layer of LeakyReLU, and an instance normalization layer of InstanceNorm2d. Among them, DropOut layers for preventing overfitting are added to the last two layers of the network, and the discriminant result is linearly output in the last layer. In order to classify the remote sensing data set into 7 categories, a classifier based on CNN is designed, which consists of 3 convolutional layers and 3 linear layers. The convolutional layers composed of 2 5*5 convolutional kernels and 1 4*4 convolutional kernel respectively pass through max pooling and ReLU activation, and Adam is used as the optimizer to linearly output reliable classification results. It should be noted that the values associated with the number of layers of the image generation model above can be replaced according to actual needs.

[0063] It should be noted that the method of the embodiment of the present application can be executed by a single device, such as a computer or a server, etc. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In this case of the distributed scenario, one of the multiple devices can only execute one or more steps of the method of the embodiment of the present application, and these multiple devices will interact with each other to complete the described method.

[0064] It should be noted that some embodiments of the present application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the above embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0065] Based on the same inventive concept, corresponding to any of the above-described method embodiments, the present application further provides a blockchain-based federated learning system.

[0066] Referring to Figure 2 , the blockchain-based federated learning system, the system includes a plurality of clients and at least one block, each client deploys a local model, and the block is used to store the global parameters associated with it, including:

[0067] The client is used to perform multiple rounds of iterative training on the local model associated with the client. For each round of the multiple rounds of iterative training, the following operations are performed: determining a plurality of remote sensing images, and obtaining the global parameters in the latest generated block; training the local model based on the plurality of remote sensing images and the global parameters to obtain local parameters; generating first transaction information associated with the local parameters, and sending the first transaction information to other randomly selected clients among the plurality of clients; receiving second transaction information sent by other clients among the plurality of clients; determining a first target client from the plurality of clients based on all the first transaction information and all the second transaction information; the first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information; based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates a corresponding new global parameter; taking the client that first calculates the new global parameter among the plurality of second target clients as the third target client; the third target client generates a new block and stores the new global parameter corresponding to the third target client in the new block until the local model meets the convergence condition.

[0068] In some embodiments, determining a first target client from the plurality of clients based on all the first transaction information and all the second transaction information includes:

[0069] The client determines the credibility of the first transaction information and the credibility of the second transaction information associated with it;

[0070] Select, as the first target client, the client for which the total number of determined credibility levels is first greater than a preset number from among the multiple clients based on the multiple credibility levels associated with each client.

[0071] In some embodiments, the first target client determines at least one second target client that meets a second preset condition from among the clients corresponding to the first transaction information and the second transaction information, including:

[0072] Based on the multiple credibility levels associated with the first target client, the first target client sorts the multiple credibility levels in descending order and determines the clients corresponding to the top N credibility levels after sorting as the second target clients that meet the second preset condition.

[0073] In some embodiments, the client determines multiple credibility levels, including:

[0074] Determine the credibility level through the following formula:

[0075]

[0076] Where is the credibility level, e is the preset base, s is the total number of current blocks in the blockchain federated learning system, is the total number of times client i is a second target client, v x indicates whether client i participated in the generation of a block in the previous iteration, with a value of 1 for participation and 0 for non - participation, mal x indicates whether the credibility level of client i in the previous iteration is lower than a preset threshold, with a value of 0 for lower than and 1 for greater than or equal to, η is an adjustment coefficient, and x represents the iteration round of the previous round.

[0077] In some embodiments, after the third target client stores the new global parameters corresponding to the third target client in the new block, the method includes:

[0078] Each second target client increments the associated identification value, where the identification value is used to represent the number of times the client is determined as the second target client;

[0079] The third target client generates and broadcasts a credibility update instruction;

[0080] In response to receiving the credibility update instruction, based on the first transaction information, the second transaction information, and the credibility update instruction, each second target client except the third target client updates its associated multiple credibility levels and sends the credibility level corresponding to the second transaction information among the updated multiple credibility levels to the associated client.

[0081] In some embodiments, each transaction information further includes local parameters of the client associated therewith;

[0082] Based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates a corresponding new global parameter, including:

[0083] Determine the new global parameter through the following formula:

[0084]

[0085] where, W (t) is the new global parameter, β is the weighting coefficient of the local model associated with the second target client, γ i is the correlation metric value of the local model of the second target client i determined based on the first transaction information and the second transaction information, w i is the local parameter of the current iteration round of the second target client i determined based on the first transaction information and the second transaction information, E is the total number of second target clients participating in the t-th round of iterative training, W (t-1) is the global parameter, and t represents the current iteration round.

[0086] In some embodiments, it further includes:

[0087] Determine the weighting coefficient of each local model through the following formula:

[0088]

[0089] where, β is the weighting coefficient of the local model, k is the adjustment coefficient between the local parameter of the t-th round and the local parameter of the (t - 1)-th round of the second target client, Δt is the aggregation time difference between the aggregation of the local model of the t-th round and the aggregation of the local model of the (t - 1)-th round of the second target client, d is the Euclidean distance between the local parameter obtained in the t-th round and the local parameter obtained in the (t - 1)-th round of the second target client i, and e is the preset base number.

[0090] In some embodiments, the client determines a plurality of remote sensing images, including:

[0091] The client receives remote sensing data, performs imaging processing on the remote sensing data using two-dimensional matched filtering to obtain a plurality of initial remote sensing images, and normalizes the pixel values of each initial remote sensing image to obtain the plurality of remote sensing images.

[0092] In some embodiments, before the client trains the local model, the method includes:

[0093] In response to determining that the number of the multiple remote sensing images is less than or equal to a preset number threshold, based on the multiple remote sensing images, the client inputs the multiple remote sensing images into a trained image generation model in sequence, and the trained image generation model outputs new remote sensing images in sequence until the sum of the number of the multiple remote sensing images and the new remote sensing images is greater than the preset number threshold, wherein the noise information of the remote sensing image and its associated new remote sensing image is different.

[0094] For the convenience of description, when describing the above device, various modules are described separately according to their functions. Of course, when implementing the present application, the functions of each module can be implemented in one or more software and / or hardware.

[0095] The device of the above embodiment is used to implement the corresponding blockchain-based federated learning model training method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated herein.

[0096] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary, and is not intended to imply that the scope of the present application (including the claims) is limited to these examples; under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of brevity.

[0097] In addition, for the sake of simplicity of explanation and discussion, and in order not to make the embodiments of the present application difficult to understand, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. In addition, the device may be shown in block diagram form in order to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (that is, these details should be fully within the understanding of those skilled in the art). In the case where specific details (such as circuits) are set forth to describe the exemplary embodiments of the present application, it will be apparent to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with variations of these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0098] Although the present application has been described in connection with specific embodiments of the present application, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other memory architectures (such as dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0099] Embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included within the protection scope of the present application.

Claims

1. A method for training a federated learning model based on a blockchain, characterized in that, it is applied to a federated learning system based on a blockchain, wherein the blockchain federated learning system includes multiple clients and at least one block, each client deploys a local model, and the block is used to store global parameters associated with it; the method includes: The client performs multiple rounds of iterative training on the local model associated with the client, and for each round of the multiple rounds of iterative training, the following operations are performed: The client determines multiple remote sensing images and obtains the global parameters in the latest generated block; Based on the multiple remote sensing images and the global parameters, the client trains the local model to obtain local parameters; the client generates first transaction information associated with the local parameters and sends the first transaction information to other randomly selected clients among the multiple clients; the client receives second transaction information sent by other clients among the multiple clients; Based on all the first transaction information and all the second transaction information, a first target client is determined from the multiple clients; the first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information; Based on the first transaction information, the second transaction information and the global parameters, each second target client calculates a corresponding new global parameter; the client that first calculates the new global parameter among the multiple second target clients is used as the third target client; the third target client generates a new block and stores the new global parameter corresponding to the third target client in the new block until the local model meets the convergence condition.

2. The method according to claim 1, characterized in that, the determining of the first target client from the multiple clients based on all the first transaction information and all the second transaction information includes: The client determines the credibility of the first transaction information and the credibility of the second transaction information associated with it; Based on the multiple credibilities associated with each client, the client that is first selected from the multiple clients and whose total number of determined credibilities is greater than the preset number is used as the first target client.

3. The method according to claim 2, characterized in that, the first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information, including: Based on the multiple credibilities associated with the first target client, the first target client sorts the multiple credibilities in descending order, and determines the clients corresponding to the first N credibilities after sorting as the second target clients that meet the second preset condition.

4. The method according to claim 2, characterized in that, the client determines multiple credibilities, including: The credibility is determined by the following formula: Among them, is the said credibility, e is the preset base number, s is the total number of current blocks in the blockchain federated learning system, is the total number of times that client i is the second target client, v x indicates whether client i participated in the generation of blocks in the previous iteration. If participated, the value is 1; if not participated, the value is 0. mal x indicates whether the credibility of client i in the previous iteration is lower than the preset threshold. If lower, the value is 0; if greater than or equal to, the value is 1. η is the adjustment coefficient, and x represents the iteration round of the previous round.

5. The method according to claim 2, characterized in that, After the third target client stores the new global parameters corresponding to the third target client into the new block, the method includes: Each second target client increments the associated identification value, where the identification value is used to represent the number of times the client is determined to be the second target client; The third target client generates and broadcasts a reputation update instruction; In response to receiving the reputation update instruction, based on the first transaction information, the second transaction information, and the reputation update instruction, each second target client except the third target client updates multiple reputations associated with it, and sends the reputation corresponding to the second transaction information among the updated multiple reputations to the associated client.

6. The method according to claim 1, wherein, Each transaction information further includes local parameters of the client associated with it; Based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates the corresponding new global parameters, including: Determining the new global parameters through the following formula: Among them, W (t) is the new global parameter, β is the weighting coefficient of the local model associated with the second target client, and γ i is the correlation metric value of the local model of the second target client i determined based on the first transaction information and the second transaction information, and w i is the local parameter of the current iteration round of the second target client i determined based on the first transaction information and the second transaction information, E is the total number of second target clients participating in the t-th round of iterative training, and W (t-1) is the global parameter, and t represents the current iteration round.

7. The method according to claim 6, wherein, It further includes: Determining the weighted coefficient of each local model through the following formula: where β is the weighted coefficient of the local model, k is the adjustment coefficient between the t-th round local parameters and the (t - 1)-th round local parameters of the second target client, Δt is the aggregation time difference between the t-th round local model aggregation and the (t - 1)-th round local model aggregation of the second target client, d is the Euclidean distance between the local parameters obtained in the t-th round and the local parameters obtained in the (t - 1)-th round of the second target client i, and e is the preset base.

8. The method according to claim 1, wherein, The client determines multiple remote sensing images, including: The client receives remote sensing data, performs imaging processing on the remote sensing data using two-dimensional matched filtering to obtain multiple initial remote sensing images, and normalizes the pixel values of each initial remote sensing image to obtain the multiple remote sensing images.

9. The method according to claim 1, wherein, Before the client trains the local model, the method includes: In response to determining that the number of the multiple remote sensing images is less than or equal to a preset number threshold, based on the multiple remote sensing images, the client sequentially inputs the multiple remote sensing images into a trained image generation model, and the image generation model sequentially outputs new remote sensing images until the sum of the number of the multiple remote sensing images and the new remote sensing images is greater than the preset number threshold, where the noise information of the remote sensing image is different from that of the associated new remote sensing image.

10. A blockchain-based federated learning system, wherein, The system includes multiple clients and at least one block. Each client deploys a local model, and the block is used to store the global parameters associated with it; The client is used to perform multiple rounds of iterative training on the local model associated with the client, and for each round of the multiple rounds of iterative training, perform the following operations: Determine multiple remote sensing images and obtain the global parameters in the newly generated block; Based on the multiple remote sensing images and the global parameters, train the local model to obtain local parameters; generate first transaction information associated with the local parameters, and send the first transaction information to other randomly selected clients among the multiple clients; Receive second transaction information sent by other clients among the multiple clients; Based on all the first transaction information and all the second transaction information, determine a first target client from the multiple clients; The first target client determines at least one second target client that meets the second preset condition from the clients corresponding to the first transaction information and the second transaction information; Based on the first transaction information, the second transaction information, and the global parameters, each second target client calculates the corresponding new global parameter; the client among the multiple second target clients that first calculates the new global parameter is used as the third target client; the third target client generates a new block and stores the new global parameter corresponding to the third target client in the new block until the local model meets the convergence condition.

Images