Method, device, equipment and medium for constructing random model sharing pool for federated learning

Abstract

The present invention belongs to the field of federated learning technology and discloses a method, apparatus, device, and medium for constructing a random model sharing pool for federated learning. The method comprises: randomly selecting at least one base model from the model sharing pool; randomly assigning the at least one base model to multiple participants; obtaining the model parameters of the local model recorded by each participant after training, aggregating the model parameters of all local models corresponding to the same base model after training, and obtaining a global model after the aggregation of each base model; then randomly selecting at least one global model and re-randomly assigning it to multiple participants to update the local model of each participant, and each participant then performs the next iterative training with the updated local model until the global models corresponding to each base model converge. The present invention can provide better guarantees for privacy protection in federated learning and provide a better privacy protection solution for digital governance.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and specifically relates to a method, apparatus, device and medium for constructing a random model sharing pool for federated learning. Background Art

[0002] During the training and use of machine learning models, data is constantly at risk of leakage, posing significant challenges to data privacy. Incidents such as the Facebook and Yahoo data breaches have garnered significant attention from both industry and academia. The Google team pioneered the concept of federated learning, a distributed machine learning framework for multi-user scenarios designed to address data silos and privacy issues encountered in artificial intelligence. This technology enables the training of a global model by coordinating multiple participants through a server, without requiring them to upload their original data. This approach makes data available but invisible.

[0003] The training data in federated learning is distributed across the devices of the data owner. The data owner trains locally and only shares the model parameters with the service provider. The service provider aggregates the updates of the data owner through an algorithm (such as FedAvg, FedProx, etc.) to train the global model.

[0004] However, the privacy protection mechanism of federated learning based on stochastic gradient descent still causes the leakage of local data due to its gradient information. To solve this problem, the following solutions are used in existing technologies:

[0005] 1. A federated learning solution based on homomorphic encryption encrypts the gradient parameters of each participant before uploading them to the server to prevent malicious or curious servers from obtaining the participants' original data. However, since this solution uses traditional homomorphic encryption, all participants have the same key, which cannot prevent malicious or curious other participants.

[0006] 2. Privacy protection is achieved at the training data level through a hybrid protocol secure multi-party computation framework; however, this approach incurs significant computational and communication overhead.

[0007] 3. Combining differential privacy technology with homomorphic encryption technology can prevent collusion between the server and participants. However, differential privacy technology adds noise to the data to be encrypted. Too little noise will reduce the encryption effect, while too much noise will affect the calculation results.

[0008] 4. Combining secret sharing and homomorphic encryption technology, we propose a federated learning privacy protection scheme. Unlike previous methods, this scheme incorporates a third party to coordinate the training process to address data privacy and data centralization issues. However, the selection and stability of the third party in this scheme will affect the stability of the model.

[0009] Therefore, some progress has been made in protecting the privacy of federated learning, using techniques such as differential privacy, homomorphic encryption, secret sharing, and secure multi-party computation. However, most existing federated learning privacy protection mechanisms assume the independent and identically distributed nature of data. However, data in real-world applications often does not meet this independent and identically distributed nature. Furthermore, issues such as participant selection and evaluation mechanisms, and model uniformity significantly hinder the implementation of federated learning. Summary of the Invention

[0010] The purpose of the present invention is to provide a method, device, equipment and medium for constructing a random model sharing pool for federated learning, so as to solve the problems that most existing federated learning privacy protection mechanisms assume the independent and identically distributed nature of data to a certain extent, while the data in real application scenarios often do not meet the independent and identically distributed nature, and the participant selection and participant evaluation mechanism, model uniqueness issues, etc. greatly affect the implementation of federated learning.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] In a first aspect, the present invention provides a method for constructing a random model sharing pool for federated learning, the method comprising:

[0013] Randomly select at least one base model from a shared pool of pre-built models;

[0014] At least one basic model is randomly assigned to multiple participants, and the assigned basic model is used as the local model of each participant. Each participant is required to train the local model based on local sample data, and each participant records the basic model corresponding to the local model participating in this training and the model parameters of the local model after training;

[0015] Obtain the trained model parameters of the local model recorded by each participant, aggregate the trained model parameters of all local models corresponding to the same base model, and obtain the global model after the aggregation of each base model;

[0016] At least one global model is randomly selected from the global model after all basic models are aggregated and redistributed randomly to multiple participants to update the local model of each participant. Each participant then performs the next iterative training with the updated local model until the global model after the updated local models corresponding to each basic model are aggregated with the trained model parameters converges. The global model after the converged basic models are aggregated is used as the trained model in the model sharing pool.

[0017] Preferably, the model sharing pool has multiple basic models, all of which have the same or similar characteristic parameters, and the number of basic models selected in the model sharing pool is equal to or less than the total number of basic models in the model sharing pool.

[0018] Preferably, the method further comprises:

[0019] Select one participant from multiple participants as a trusted participant based on voting rules;

[0020] Obtain the trained model parameters of the trusted participant and the trained model parameters of the remaining participants;

[0021] Compare the trained model parameters of the remaining participants with the trained model parameters of the trusted participant;

[0022] When the difference between the trained model parameters of any remaining participant and the trained model parameters of the trusted participant reaches a preset difference, the participant is marked as an untrusted participant.

[0023] Preferably, the method further comprises:

[0024] When aggregating all trained model parameters of the same base model, the trained model parameters of untrusted participants are excluded;

[0025] When randomly selecting at least one global model from the global model obtained by aggregating all basic models and redistributing it randomly to multiple participants, non-trusted participants from the multiple participants are eliminated, and the at least one global model is redistributed to the remaining participants.

[0026] Preferably, at least one global model is randomly selected from the global model obtained by aggregating all base models and then randomly assigned to multiple participants to update the local models of each participant. Each participant then performs the next iterative training with the updated local model until the global model obtained by aggregating the trained model parameters of the updated local models corresponding to each base model converges, including:

[0027] S401: Testing the global model after all basic models are aggregated based on the test samples to obtain the test accuracy error of the global model after all basic models are aggregated;

[0028] S402: Determine whether the test accuracy errors of the global model after the aggregation of the basic models meet the preset accuracy error range at the same time. If so, it indicates that the global model after the aggregation of the basic models has converged. If not, execute step S403;

[0029] S403: Randomly select at least one global model from the global model aggregated from all basic models and randomly redistribute it to multiple participants. Use the global model to update the local models of the participants, and train each participant on the updated local model based on local sample data. Each participant records the global model corresponding to the updated local model participating in this training and the model parameters of the local model after training.

[0030] S404: Obtain the trained model parameters of the updated local model recorded by each participant, aggregate the trained model parameters of all updated local models corresponding to the same global model, and update the global model after aggregation of each basic model;

[0031] S405: Testing the global model aggregated from all updated basic models based on the test samples to obtain the test accuracy error of the global model aggregated from all updated basic models;

[0032] S406: Determine again whether the test accuracy errors of the global model after the updated basic models are aggregated meet the preset accuracy error range at the same time. If not, repeat steps S403 to S406 until the test accuracy errors of the global model after the updated basic models are aggregated meet the preset accuracy error range at the same time, so as to indicate that the global model after the updated basic models are aggregated has converged.

[0033] Preferably, after the updated local models corresponding to the respective basic models converge to a global model after aggregating the trained model parameters, the method further comprises:

[0034] Get the test accuracy error of each global model;

[0035] Determine whether the test accuracy error of each global model satisfies the confidence region, and add the global model that satisfies the confidence region to the acceptable model set;

[0036] Based on the preset elimination rules, the global model in the acceptable model set is eliminated to obtain the eliminated acceptable model set, and the global model in the eliminated acceptable model set is used as the shared model in the model sharing pool.

[0037] Preferably, the preset elimination rules include:

[0038] a10: gradually reduce the confidence region by the preset value until the confidence region reaches the preset range;

[0039] a20: After the confidence region is narrowed for the i-th time, all global models in the acceptable model set are randomly redistributed to multiple participants. Each participant updates the local model again with the global model in the acceptable model set. Each participant trains the updated local model based on local sample data. Each participant records the global model corresponding to the updated local model participating in this training and the model parameters of the updated local model after training.

[0040] a30: Obtain the trained model parameters of the updated local model recorded by each participant, aggregate the trained model parameters of all updated local models corresponding to the same global model in the acceptable model set, and obtain a new model for each global model in the acceptable model set;

[0041] a40: Test the new models of all global models in the acceptable model set based on the test samples, and obtain the test accuracy error of the new model of each global model;

[0042] a50: Determine whether the test accuracy error of the new model of each global model satisfies the confidence region after the i-th reduction, and remove the new model of the global model that does not satisfy the confidence region after the i-th reduction from the set of acceptable models;

[0043] a60: After the confidence region is reduced for the i+1th time, all new models of the global models in the acceptable model set after the i-th elimination are randomly redistributed to multiple participants, and steps a10 to a50 are repeated. After the confidence region reaches the preset range, the acceptable model set after elimination is obtained.

[0044] In a second aspect, the present invention provides a device for constructing a random model sharing pool for federated learning, the device being used to implement the above-mentioned method for constructing a random model sharing pool for federated learning, the device comprising:

[0045] A model selection module, configured to randomly select at least one base model from a pre-built model sharing pool;

[0046] The model training module is used to randomly assign at least one basic model to multiple participants, use the assigned basic model as the local model of each participant, and enable each participant to train the local model based on local sample data. Each participant records the basic model corresponding to the local model participating in this training and the model parameters of the local model after training;

[0047] The model aggregation module is used to obtain the model parameters of the local model recorded by each participant after training, aggregate the model parameters of all local models corresponding to the same basic model after training, and obtain the global model after the aggregation of each basic model;

[0048] The model iteration module is used to randomly select at least one global model from the global model after aggregating all basic models and redistribute it randomly to multiple participants to update the local model of each participant. Each participant then performs the next iterative training with the updated local model until the updated local models corresponding to each basic model converge to the global model after aggregating the trained model parameters. The converged global model after aggregating the converged basic models is used as the trained model in the model sharing pool.

[0049] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for constructing a random model sharing pool for federated learning when executing the computer program.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for constructing a random model sharing pool for federated learning.

[0051] Beneficial effects:

[0052] 1. This invention builds a model sharing pool with multiple basic models, so that different participants in typical federated learning can use different basic models for training, ensuring the practicality of the training results;

[0053] 2. During each training, the present invention randomly selects a certain number of basic models from the model sharing pool, and then randomly distributes at least one selected basic model to all participants. This can provide better guarantees for privacy protection in federated learning and provide a better privacy protection solution for digital governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0055] Figure 1 This is a flowchart of a method for constructing a random model sharing pool for federated learning provided by one embodiment of the present invention;

[0056] Figure 2 Schematic diagram of a confidence region provided by one embodiment of the present invention;

[0057] Figure 3 It is a block diagram of a random model sharing pool construction device for federated learning provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0059] Example 1

[0060] Figure 1 This is a flowchart of a method for constructing a random model sharing pool for federated learning provided by an embodiment of the present invention. The application scenarios of the method for constructing a random model sharing pool for federated learning in this embodiment include: a server and multiple clients, all of which are communicated with the server, and each client is a participant.

[0061] Based on the above application scenario, this embodiment provides a method for constructing a random model sharing pool for federated learning, which is applied to the server side of this scenario. Figure 1 As shown, the method includes:

[0062] Step S10: The server randomly selects at least one basic model from a pre-built model sharing pool.

[0063] In this embodiment, the model sharing pool contains multiple basic models, all of which have the same or similar characteristic parameters. The characteristic parameters of the basic models mean that they have the same or similar performance trends on the same data. This performance trend can also be understood as, under the same other resources (limiting training parameters, controlling parameters that are not required here under the same conditions), these basic models have the same or similar rising and falling trends when training the same data. Basic models with the same or similar characteristic parameters are added to the model sharing pool. If these basic models in the model sharing pool have different dimensions or the parameter differences are too large, it will affect the subsequent model aggregation. Therefore, it is necessary to revise some of the parameters (such as dimension, number of channels) before adding these models to the model sharing pool. After adjusting the parameters, test these parameters and model aggregation functions until they are compatible with each other without errors.

[0064] The basic model of this embodiment can be VGG16 (Visual Geometry Group 16-layer network), VGG19 (Visual Geometry Group 19-layer network), ResNet34 (Residual Network 34 layers), ResNet50 (Residual Network 50 layers), etc.

[0065] The number of base models selected in the model sharing pool is equal to or less than the total number of base models in the model sharing pool.

[0066] In this embodiment, the present invention constructs a model sharing pool on the server and adds multiple basic models to the model sharing pool, so that different participants in typical federated learning can use different basic models for training, thereby ensuring the practicality of the training results.

[0067] Step S20: The server randomly distributes at least one basic model to multiple participants, and uses the distributed basic model as the local model of each participant, and enables each participant to train the local model based on local sample data. Each participant records the basic model corresponding to the local model participating in this training and the model parameters of the local model after training, such as the gradient of the model and other parameters.

[0068] Step S30: The server obtains the model parameters of the local model after training recorded by each participant, aggregates the model parameters of all local models corresponding to the same basic model after training, and obtains a global model after the aggregation of each basic model.

[0069] In this embodiment, since the parameters provided by the user still contain sensitive information of local data, they may be attacked and cause privacy leakage. Therefore, it is necessary to screen out trustworthy participants from multiple participants.

[0070] Specifically, the method further includes:

[0071] Step S301: selecting a participant from multiple participants as a trusted participant based on voting rules;

[0072] Step S302: Obtain the trained model parameters of the trusted participant and the trained model parameters of the remaining participants;

[0073] Step S303: Compare the trained model parameters of the remaining participants with the trained model parameters of the trusted participant;

[0074] Step S304: When the difference between the trained model parameters of any remaining participant and the trained model parameters of the trusted participant reaches a preset difference, the participant is marked as an untrusted participant.

[0075] In this embodiment, untrusted participants are eliminated, and their model parameters do not participate in parameter aggregation, nor do they participate in subsequent iterative training. After eliminating untrusted participants, the remaining participants are all trustworthy, that is, they are all trusted participants.

[0076] Therefore, the method further includes: when aggregating all trained model parameters of the same basic model, excluding trained model parameters of untrusted participants.

[0077] Step S40: The server randomly selects at least one global model from the global model after all basic models are aggregated and redistributes it randomly to multiple participants to update the local model of each participant. Each participant then performs the next iterative training with the updated local model until the updated local models corresponding to each basic model converge to the global model after the trained model parameters are aggregated. The converged global model after the basic models are aggregated is used as the trained model in the model sharing pool.

[0078] When randomly selecting at least one global model from the global model after aggregating all basic models and re-randomly assigning it to multiple participants, non-trusted participants among the multiple participants are eliminated, and at least one global model is re-assigned to the remaining participants; that is, the server re-randomly assigns the global model to the trusted participants.

[0079] Specifically, at least one global model is randomly selected from the global model obtained by aggregating all basic models and randomly redistributed to multiple participants to update the local models of each participant. Each participant then performs the next iterative training with the updated local model until the global model obtained by aggregating the trained model parameters of the updated local models corresponding to each basic model converges, including:

[0080] Step S401: testing the global model after all basic models are aggregated based on the test samples, and obtaining the test accuracy error of the global model after all basic models are aggregated;

[0081] Step S402: Determine whether the test accuracy errors of the global model after the aggregation of the basic models meet the preset accuracy error range at the same time. If so, it indicates that the global model after the aggregation of the basic models has converged. If not, execute step S403;

[0082] Step S403: Randomly select at least one global model from the global model aggregated from all basic models and randomly redistribute it to multiple participants. Use the global model to update the local models of the participants, and train each participant on the updated local model based on local sample data. Each participant records the global model corresponding to the updated local model participating in this training and the model parameters of the local model after training.

[0083] Step S404: Obtain the trained model parameters of the updated local model recorded by each participant, aggregate the trained model parameters of all updated local models corresponding to the same global model, and update the global model after aggregation of each basic model;

[0084] Step S405: testing the global model aggregated from all updated basic models based on the test samples, and obtaining the test accuracy error of the global model aggregated from all updated basic models;

[0085] Step S406: Determine again whether the test accuracy errors of the global model after the aggregation of the updated basic models meet the preset accuracy error range at the same time. If not, repeat steps S403 to S406 until the test accuracy errors of the global model after the aggregation of the updated basic models meet the preset accuracy error range at the same time, so as to indicate that the global model after the aggregation of the updated basic models has converged.

[0086] In this embodiment, during the training process, a certain number of basic models are randomly selected from the model sharing pool (the number and type of basic models selected for each training are different), and then at least one selected basic model is randomly distributed to all participants (the type of basic model allocated to the participants is also random and different for each training). This can provide better guarantees for privacy protection in federated learning and provide a better privacy protection solution for digital governance.

[0087] As a further optimization of this embodiment, after the updated local models corresponding to the respective basic models converge to the global model after aggregating the trained model parameters, the method further includes:

[0088] Step S50: Obtain the test accuracy error of each global model.

[0089] Step S60: Determine whether the test accuracy error of each global model satisfies the confidence region, and add the global model that satisfies the confidence region to the acceptable model set; wherein, the confidence region is: introduce a concentric circle confidence region for testing the errors of different models, the radius of the small circle of the concentric circle is R1, the radius of the large circle is R2, and R2-R1 is the confidence region (here R1 is the minimum acceptable accuracy, R2 is the maximum acceptable accuracy), such as Figure 2 As shown in the figure, the errors within the confidence region are acceptable.

[0090] Step S70: Based on the preset elimination rules, the global model in the acceptable model set is eliminated to obtain the acceptable model set after elimination, and the global model in the acceptable model set after elimination is used as the shared model in the model sharing pool; wherein, the shared model is used as a model application to participate in the recognition tasks of each participant, etc.

[0091] In this embodiment, the models outside the confidence region are discarded as anomalies, that is, the models whose test accuracy is smaller than R1 or larger than R2 are discarded. If it is smaller than R1, it means that the model is unable to learn the characteristics of the training data this time, resulting in underfitting. If it is larger than R2, it may be that the model performs too well on the training data this time, which may be an overfitting phenomenon, which is not conducive to promotion and use.

[0092] In this embodiment, the preset elimination rules include:

[0093] Step a10: gradually reducing the confidence region by a preset value until the confidence region reaches a preset range;

[0094] Step a20: After the confidence region is narrowed for the i-th time, all global models in the acceptable model set are randomly redistributed to multiple participants again. Each participant updates the local model again with the global model in the acceptable model set. Each participant trains the updated local model based on local sample data. Each participant records the global model corresponding to the updated local model participating in this training and the model parameters of the updated local model after training.

[0095] Step a30: Obtain the trained model parameters of the updated local model recorded by each participant, aggregate the trained model parameters of all updated local models corresponding to the same global model in the acceptable model set, and obtain a new model for each global model in the acceptable model set;

[0096] Step a40: testing the new models of all global models in the acceptable model set based on the test samples, and obtaining the test accuracy error of the new model of each global model;

[0097] Step a50: Determine whether the test accuracy error of the new model of each global model meets the confidence region after the i-th reduction, and remove the new models of the global models that do not meet the confidence region after the i-th reduction from the set of acceptable models;

[0098] Step a60: After the (i + 1)-th reduction of the confidence region, re-randomly assign the new models of all the global models in the set of acceptable models after the i-th removal to multiple parties, and repeat Steps a10 to a50. After the confidence region reaches the preset range, obtain the set of acceptable models after removal.

[0099] In this embodiment, all models with test accuracy errors between R1 and R2 are continuously iteratively trained, and the value of R2 - R1 is sequentially reduced; until R2 - R1 < M (where M is a very small value, and the specific value is determined by the algorithm user according to the algorithm application scenario), select one model from the models that have not been removed for application.

[0100] The present invention constructs a model sharing pool, which has multiple basic models. Different parties in typical federated learning can use different basic models for training, ensuring the practicality of the training results; and in each training, randomly select a certain number of basic models from the model sharing pool, and then randomly assign at least one of the selected basic models to all parties, which can provide better guarantee for federated learning in terms of privacy protection and provide a better privacy protection solution for digital governance.

[0101] Embodiment 2

[0102] Figure 3 is a block diagram of a random model sharing pool construction device for federated learning provided by an embodiment of the present invention, as Figure 3 shown. This embodiment provides a random model sharing pool construction device for federated learning. The device is used to implement the random model sharing pool construction method for federated learning in Embodiment 1. The device includes:

[0103] A model selection module for randomly selecting at least one basic model from a pre-constructed model sharing pool;

[0104] A model training module for randomly assigning at least one basic model to multiple parties, using the assigned basic model as the local model of each party, and enabling each party to train the local model based on local sample data, and each party records the basic model corresponding to the local model participating in this training and the model parameters of the local model after training;

[0105] The model aggregation module is used to obtain the model parameters of the local model recorded by each participant after training, aggregate the model parameters of all local models corresponding to the same basic model after training, and obtain the global model after the aggregation of each basic model;

[0106] The model iteration module is used to randomly select at least one global model from the global model after aggregating all basic models and redistribute it randomly to multiple participants to update the local model of each participant. Each participant then performs the next iterative training with the updated local model until the updated local models corresponding to each basic model converge to the global model after aggregating the trained model parameters. The converged global model after aggregating the converged basic models is used as the trained model in the model sharing pool.

[0107] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for constructing a random model sharing pool for federated learning of the first embodiment is implemented.

[0108] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the method for constructing a random model sharing pool for federated learning in embodiment one.

[0109] The present invention constructs a model sharing pool with multiple basic models, so that different participants in typical federated learning can use different basic models for training, thereby ensuring the practicality of the training results; and in each training, a certain number of basic models are randomly selected from the model sharing pool, and then at least one selected basic model is randomly distributed to all participants, which can provide better guarantees for the privacy protection of federated learning and provide a better privacy protection solution for digital governance.

[0110] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for constructing a random model sharing pool for federated learning, characterized in that: The method comprises: Randomly select at least one base model from a shared pool of pre-built models; At least one basic model is randomly assigned to multiple participants, and the assigned basic model is used as the local model of each participant. Each participant is required to train the local model based on local sample data, and each participant records the basic model corresponding to the local model participating in this training and the model parameters of the local model after training; Obtain the trained model parameters of the local model recorded by each participant, aggregate the trained model parameters of all local models corresponding to the same base model, and obtain the global model after the aggregation of each base model; At least one global model is randomly selected from the global model after all base models are aggregated and redistributed randomly to multiple participants to update the local models of each participant. Each participant then performs the next iterative training with the updated local model until the global model after the updated local models corresponding to each base model are aggregated with the trained model parameters converges. The global model after the converged base models are aggregated is used as the trained model in the model sharing pool, including: S401: Testing the global model after all basic models are aggregated based on the test samples to obtain the test accuracy error of the global model after all basic models are aggregated; S402: Determine whether the test accuracy errors of the global model after the aggregation of the basic models meet the preset accuracy error range at the same time. If so, it indicates that the global model after the aggregation of the basic models has converged. If not, execute step S403; S403: Randomly select at least one global model from the global model aggregated from all basic models and randomly redistribute it to multiple participants. Use the global model to update the local models of the participants, and train each participant on the updated local model based on local sample data. Each participant records the global model corresponding to the updated local model participating in this training and the model parameters of the local model after training. S404: Obtain the trained model parameters of the updated local model recorded by each participant, aggregate the trained model parameters of all updated local models corresponding to the same global model, and update the global model after aggregation of each basic model; S405: Testing the global model aggregated from all updated basic models based on the test samples to obtain the test accuracy error of the global model aggregated from all updated basic models; S406: Determine again whether the test accuracy errors of the global model after the update of the aggregated base models simultaneously meet the preset accuracy error range. If not, repeat steps S403 to S406 until the test accuracy errors of the global model after the update of the aggregated base models simultaneously meet the preset accuracy error range, indicating that the global model after the update of the aggregated base models has converged. Among them, after the updated local models corresponding to the basic models converge to the global model after aggregating the trained model parameters, it also includes: Get the test accuracy error of each global model; Determine whether the test accuracy error of each global model satisfies the confidence region, and add the global model that satisfies the confidence region to the acceptable model set; Based on the preset elimination rules, the global model in the acceptable model set is eliminated to obtain the eliminated acceptable model set, and the global model in the eliminated acceptable model set is used as the shared model in the model sharing pool; The method further includes: selecting a participant as a trusted participant from multiple participants based on voting rules; Obtain the trained model parameters of the trusted participant and the trained model parameters of the remaining participants; Compare the trained model parameters of the remaining participants with the trained model parameters of the trusted participant; When the difference between the trained model parameters of any remaining participant and the trained model parameters of the trusted participant reaches a preset difference, the participant is marked as an untrusted participant.

2. The method for constructing a random model sharing pool for federated learning according to claim 1, characterized in that: The model sharing pool contains multiple basic models, and all basic models have the same or similar characteristic parameters; the number of basic models selected in the model sharing pool is equal to or less than the total number of basic models in the model sharing pool.

3. The method for constructing a random model sharing pool for federated learning according to claim 1, characterized in that: The method further comprises: When aggregating all trained model parameters of the same base model, the trained model parameters of untrusted participants are excluded; When randomly selecting at least one global model from the global model obtained by aggregating all basic models and redistributing it randomly to multiple participants, non-trusted participants from the multiple participants are eliminated, and the at least one global model is redistributed to the remaining participants.

4. The method for constructing a random model sharing pool for federated learning according to claim 1, characterized in that: The preset elimination rules include: a10: gradually reduce the confidence region by the preset value until the confidence region reaches the preset range; a20: After the confidence region is narrowed for the i-th time, all global models in the acceptable model set are randomly redistributed to multiple participants. Each participant updates the local model again with the global model in the acceptable model set. Each participant trains the updated local model based on local sample data. Each participant records the global model corresponding to the updated local model participating in this training and the model parameters of the updated local model after training. a30: Obtain the trained model parameters of the updated local model recorded by each participant, aggregate the trained model parameters of all updated local models corresponding to the same global model in the acceptable model set, and obtain a new model for each global model in the acceptable model set; a40: Test the new models of all global models in the acceptable model set based on the test samples, and obtain the test accuracy error of the new model of each global model; a50: Determine whether the test accuracy error of the new model of each global model satisfies the confidence region after the i-th reduction, and remove the new model of the global model that does not satisfy the confidence region after the i-th reduction from the set of acceptable models; a60: After the confidence region is reduced for the i+1th time, all new models of the global models in the acceptable model set after the i-th elimination are randomly redistributed to multiple participants, and steps a10 to a50 are repeated. After the confidence region reaches the preset range, the acceptable model set after elimination is obtained.

5. A device for constructing a random model sharing pool for federated learning, the device being used to implement the method for constructing a random model sharing pool for federated learning according to any one of claims 1 to 4, characterized in that: The device comprises: A model selection module, configured to randomly select at least one base model from a pre-built model sharing pool; The model training module is used to randomly assign at least one basic model to multiple participants, use the assigned basic model as the local model of each participant, and enable each participant to train the local model based on local sample data. Each participant records the basic model corresponding to the local model participating in this training and the model parameters of the local model after training; The model aggregation module is used to obtain the model parameters of the local model recorded by each participant after training, aggregate the model parameters of all local models corresponding to the same basic model after training, and obtain the global model after the aggregation of each basic model; The model iteration module is used to randomly select at least one global model from the global model after aggregating all basic models and redistribute it randomly to multiple participants to update the local model of each participant. Each participant then performs the next iterative training with the updated local model until the updated local models corresponding to each basic model converge to the global model after aggregating the trained model parameters. The converged global model after aggregating the converged basic models is used as the trained model in the model sharing pool.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the random model sharing pool construction method for federated learning according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for constructing a random model sharing pool for federated learning according to any one of claims 1 to 4 is implemented.

Images