A safe and efficient model co-construction method
By employing an asynchronous grouping robust aggregation algorithm and a security enhancement protocol, the Byzantine attack and privacy leakage issues in IoT federated learning are addressed, improving model robustness and training efficiency, reducing communication overhead, and achieving secure and efficient model co-construction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2023-04-25
- Publication Date
- 2026-07-21
AI Technical Summary
In IoT scenarios, federated learning faces risks of Byzantine attacks and privacy breaches. The gradient noise and high computational and communication overhead caused by asynchronous optimization algorithms are problems that existing defense methods cannot effectively solve on resource-constrained devices.
An asynchronous grouping robust aggregation algorithm is proposed, which combines delay-aware grouping, cosine anomaly filtering and inter-group delay aggregation. Paillier homomorphic encryption is used to reduce communication overhead, and a security-enhanced aggregation protocol is designed to achieve bidirectional defense in heterogeneous device environments.
While ensuring data privacy, it improves model robustness and training efficiency, reduces communication overhead, and enables secure and efficient model co-construction on IoT devices.
Smart Images
Figure CN116488906B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and relates to a safe and efficient method for model co-construction. Background Technology
[0002] In the era of the Internet of Things (IoT), where everything is interconnected, massive numbers of devices connect to the network, generating huge amounts of interactive data. The enormous demand for data processing and analysis is driving a deeper integration of IoT and artificial intelligence (AI), significantly enhancing IoT's analytical, decision-making, and collaborative processing capabilities. However, the exclusive ownership of this data by its owners hinders the progress of AI-assisted IoT applications. Federated learning, as a distributed machine learning method for collaborative model building, is well-suited to IoT due to its distributed training characteristics. By aggregating parameters from multiple sources, it updates and iterates model parameters, achieving data sharing and joint modeling while meeting privacy protection requirements.
[0003] Federated learning, as an emerging privacy-preserving machine learning technology, leverages powerful data processing and knowledge mining capabilities to help the Internet of Things (IoT) achieve ubiquitous connectivity and intelligence. However, the open model training environment, frequent sharing of model parameters, and vulnerable model aggregation algorithms expose federated learning to new risks. First, because IoT devices typically lack sufficient security measures, they are often susceptible to malicious control and attacks, increasing the risk of model training failure. A typical attack is the Byzantine attack, where a Byzantine attacker can send arbitrary model updates to the server, thereby impairing overall learning performance. Second, the privacy protection mechanism of federated learning is based on a very strong security assumption: that the intermediate parameters exposed in each iteration will not leak sensitive information. However, many studies have proven that this security assumption is invalid; inference attacks can be launched using intermediate parameters to achieve partial leakage of specific neural network layers or complete leakage of gradients.
[0004] Currently, there are some research works on defending against Byzantine attacks and privacy inference attacks. Xu C, Jia Y, and Zhu L, in "TDFL: Truth Discovery Based Byzantine Robust Federated Learning" [in IEEE Transactions on Parallel and Distributed Systems, vol. 33, no. 12, pp. 4835-4848, 2022], designed a robust truth discovery aggregation method to remove malicious updates by assigning weights based on client contributions. For the majority settings of Byzantine attackers, they used the maximum clique algorithm in graph theory to filter malicious updates and ensure the quality of the global model. Wu Z, Ling Q, and Chen T, in “Federated Variance-reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks” [in IEEE Transactions on Signal Processing, vol. 68, pp. 4583-4596, 2020.], combine the geometric median algorithm and variance reduction algorithm to reduce the impact of stochastic gradient noise and prove that the algorithm can linearly converge to the neighborhood of the optimal solution even in the presence of a few Byzantine attackers. Lu Y, Huang X, and Dai Y, in “Differentially Private Asynchronous Federated Learning for Mobile Edge Computing in Urban Informatics” [in IEEE Transactions on Industrial Informatics, vol. 16, no. 3, pp. 2134-2143, 2019.], utilize a Gaussian mechanism with differential privacy, comprehensively considering the iterative optimization effect and privacy protection cost, and adaptively allocate the privacy budget in each iteration. Fu A, Zhang X, and Xiong N, in “VFL: A Verifiable Federated Learning with Privacy-preserving for Big Data in Industrial IoT” [in IEEE Transactions on Industrial Informatics, vol. 18, no. 5, pp. 3316-3326, 2020.], achieve secret sharing through two sets of Lagrange interpolation points to ensure the correctness of the aggregation results and perform effective restoration.
[0005] While existing research on defending against Byzantine attacks and privacy inference attacks has shown excellent results, several challenges remain in IoT scenarios. First, the inherent resource differences among IoT devices lead to varying response times when devices participating in federated learning tasks send model updates to the server. Asynchronous optimization algorithms allow for immediate updates to the global model upon arrival of a single model update without waiting for updates from other devices. However, asynchronous optimization presents several challenges to the design of robust Byzantine optimization algorithms. On one hand, individual gradients lack comparison objects, making effective evaluation of gradient information difficult. On the other hand, it introduces additional noise into stochastic gradients, further complicating robust algorithm design. Second, unlike centralized training, federated learning requires continuous interaction between devices and servers. For complex tasks, model parameters can reach millions, leading to two problems. Encrypting high-dimensional model updates increases both computational and communication overhead. Furthermore, secure aggregation protocols with specific computational tasks require extensive communication for information exchange. Finally, existing approaches to defending against these two attacks are contradictory, necessitating research into secure and efficient model co-construction methods. Summary of the Invention
[0006] In view of this, the purpose of this invention is to provide a safe and efficient model co-construction method that effectively improves the robustness and security of the global model under conditions of heterogeneous client devices and limited resources.
[0007] To achieve the above objectives, the present invention provides the following technical solution:
[0008] A secure and efficient model co-construction method is proposed. First, an asynchronous grouping robust aggregation algorithm is devised. Based on delay-aware grouping, a cosine anomaly filter is designed to remove anomalous gradients. By combining intra-group truth estimation and inter-group delay aggregation, the impact of laggards and Byzantine attackers is effectively mitigated. Then, a communication-efficient and secure enhanced aggregation protocol is designed. Under the assumption that communication link security is not required, the protocol utilizes the additive homomorphic and homomorphic doubling properties of Paillier homomorphic encryption to support the implementation of the asynchronous grouping robust aggregation algorithm while protecting data privacy, and significantly reducing communication overhead between devices, between devices and servers, and between servers. This method can provide bidirectional defense against semi-honest servers and Byzantine attackers in heterogeneous device environments, achieving a good balance between model performance, training efficiency, and communication overhead. The method specifically includes the following steps:
[0009] S1: Region Division: Each edge server is divided into regions to manage based on its coverage capabilities, and manages users and physical entities within the regions;
[0010] S2: System initialization: This includes two parts: model initialization and key generation;
[0011] Model initialization includes: initializing the global model parameters w on the cloud server. t The initial global model is then sent to the edge server, which in turn sends it to the IoT devices within its jurisdiction.
[0012] Key generation includes: Edge server S e and auxiliary server S a Generate your own public and private keys separately, and use (pk e ,sk e (pk) a ,sk a ) to represent S e and S a The key pair.
[0013] S3: Local Model Training: After receiving the global model, the IoT device uses local data to calculate model updates. Generate random numbers locally Update the model perturbation as follows Then the perturbation data is encrypted as Return to the edge server, and then and Returned together to the auxiliary server; among which Indicates the use of edge server S e public key pk e Encryption is performed, where N represents the dimension of the gradient and i represents the device index.
[0014] S4: Robust aggregation for edge security: The auxiliary server and the edge server receive the perturbation gradients from the first M devices returned. and encrypted perturbation data Subsequently, a security-enhanced aggregation protocol for efficient collaborative communication is implemented to realize an asynchronous grouping robust aggregation algorithm based on perturbation gradients.
[0015] S5: Cloud Global Model Aggregation: After receiving the local model aggregation results returned by each edge server, the cloud server uses the classic federated averaging algorithm to quickly aggregate the global model; if the global model index reaches the training task stopping standard, local model training is stopped, otherwise steps S3 to S5 are repeated.
[0016] Furthermore, step S4 specifically includes the following steps:
[0017] S41: The edge server performs delay-aware grouping on all received gradient information, assuming... Let be a set of devices selected in round s to participate in training and return gradients in round t. Let s be the set of devices selected in the s-th round. This refers to devices that were selected in round s but failed to successfully return the results to the server, and the groups satisfy the following conditions:
[0018] S42: Auxiliary Server S a For all received perturbation gradients Encryption is performed to obtain And Send to edge server S e ;in Indicates the use of auxiliary server S a public key pk a Encrypt;
[0019] S43: Edge Server S e receive Then, through calculation get To prevent the auxiliary server from directly seeing the device's gradient information, a random number q is selected to perform blinding processing on the gradient information. Finally Send to auxiliary server S a .
[0020] S44: Auxiliary Server S a Upon receiving Afterwards, the auxiliary server decrypted and obtained... It also calculates the cosine distance between each gradient and the true gradient value within the group, and uses a thresholding mechanism to exclude abnormal cosine distances to avoid the continuous impact of Byzantine attackers on the global model. The equipment consists of devices with the same initial round. Represents the true value of the within-group gradient for the current iteration. For a device i, the true value of the intra-group gradient cannot be predicted in the current iteration. Assuming that model training typically converges stably, the difference between two consecutive model updates should be as small as possible. Therefore, the global update from the previous iteration is used to estimate the true value of the intra-group gradient in the current iteration. The cosine distance cs between device i and the true value of the intra-group gradient is given. i Represented as:
[0021]
[0022] in, This represents a set of devices selected in round s to participate in training and return gradients in round t.
[0023] Here, in the case where most nodes are honest, the cosine distance is sorted and the median is used as the threshold.
[0024] S45: After filtering out anomalies based on the cosine distance according to the threshold, a random number r is selected to perform a secondary perturbation on the filtered gradient, and the result is calculated. And Returned to the edge server; the edge server joins get And compare each dimension to obtain the median. And Return to the auxiliary server.
[0025] S46: The auxiliary server removes the secondary perturbation value r to obtain... As the initial gradient ground truth; within a group of devices with similar latency, the weights of the gradients uploaded by each device can be adjusted by measuring the distance between each dimension and the group's gradient ground truth, and the distance information can be returned to the edge server S. e .
[0026] Calculate distance-related information, represented as:
[0027]
[0028]
[0029] Among them, T total T represents the sum of distances between all gradients and their true values within a group. single This indicates information related to the distance between a single gradient within a group and the true gradient value.
[0030] S47: Edge Server S e Received T total and T single Then, use the private key to decrypt and obtain and By adding and This means that the sum of the distances between the gradients of all devices and their true values can be obtained. and the distance between the gradient of each device and the true gradient value. Next, the weight update value for each device can be calculated. Represented as:
[0031]
[0032] Where N represents the dimension of the gradient, and M represents... The number of devices included, i.e. dist(·) is a measurement and The function of the distance between them is calculated as follows:
[0033] S48: The edge server calculates based on the weight update value. And use the auxiliary server's public key to PK a Encryption is performed, then the sum of weighted gradients across all devices is calculated. And Sum of the updated weight values of all devices Return to the auxiliary server.
[0034] Calculate the sum of weighted gradients for all devices. Represented as:
[0035]
[0036] S49: Auxiliary Server S a get and Then, use the public key to PK. a Decryption Next, calculate the true gradient value for each group. And send it to the edge server S e .
[0037] Calculate the true gradient value for each group. Represented as:
[0038] S410: Edge servers evaluate the true value of the gradient of a group based on different degrees of antiquity. Delayed aggregation is represented as:
[0039]
[0040] Among them, w t+1 Let w represent the global model parameters in the (t+1)th round, η represent the learning rate, and Λ(τ) be an arbitrary decay function, set here as Λ(τ) = 1 / (1+τ), τ = ts, where s represents the round in which the device is selected to participate in training, and t represents the round in which the device returns the gradient; finally, the edge server will... t+1 Send the data to the cloud server for global model aggregation.
[0041] Furthermore, step S5 specifically includes the following steps:
[0042] S51: The cloud server receives the w returned by all edge servers. t+1 The global model is aggregated based on the federated average algorithm, as follows:
[0043]
[0044] in, These are the global model parameters in the (t+1)th round, n e n is the total amount of data contained in the edge region, and n is the total amount of data contained in all edge servers. E represents the number of edge servers;
[0045] S52: Next, the cloud server will... The data is distributed to all edge servers, which then distribute it to all IoT devices within their jurisdiction for a new round of training until a predefined accuracy or model convergence is achieved.
[0046] The beneficial effects of this invention are as follows: The secure and efficient model co-construction method proposed in this invention first proposes an asynchronous grouping robust aggregation method for the Internet of Things (IoT), which mitigates the impact of laggards and Byzantine attackers through delay-aware grouping, cosine anomaly filtering, intra-group truth estimation, and inter-group delayed aggregation. Furthermore, it constructs a communication-efficient and secure enhanced aggregation protocol that balances model robustness, training efficiency, and communication overhead while ensuring data privacy.
[0047] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0048] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:
[0049] Figure 1 This is a system block diagram of the safe and efficient model co-construction method of the present invention;
[0050] Figure 2 This is a schematic diagram of the asynchronous grouping robust aggregation method in this invention.
[0051] Figure 3 This is a flowchart of a security-enhanced aggregation protocol for efficient communication provided in an embodiment of the present invention. Detailed Implementation
[0052] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0053] Please see Figures 1-3 This invention provides a communication system component module:
[0054] Devices: Located in the sensing layer of the Internet of Things (IoT), they consist of physical entities and attached sensing devices. Each device has a fixed geographical location and is assigned to a unique edge server for management. Devices are responsible for data collection, model training, and gradient perturbation.
[0055] Edge servers: Located at base stations or access points, they possess strong communication, storage, and computing capabilities. They are responsible for receiving model updates uploaded by devices and implementing efficient and secure aggregation protocols, supporting the execution of asynchronous grouping robust aggregation algorithms for local models in encrypted environments.
[0056] Cloud server: A cloud server is a central device with very powerful communication, storage and computing capabilities. It is responsible for receiving local models returned by all edge servers and then performing global model aggregation based on the amount of data.
[0057] To address the aforementioned communication system, this invention provides a secure and efficient model co-construction method, specifically comprising the following steps:
[0058] Step 1: Region Division: Each edge server is assigned a region to manage based on its coverage capabilities, and manages the users and physical entities within that region.
[0059] Step 2: System Initialization: This includes two parts: model initialization and key generation. The cloud server initializes the global model parameters w. t The initial global model is then sent to the edge server, which in turn sends it to the IoT devices within its jurisdiction. Edge server S e and auxiliary server S a Generate your own public and private keys separately, and use (pk e ,sk e (pk) a ,sk a ) to represent S e and S a The key pair.
[0060] Step 3: Local Model Training: After receiving the global model, the IoT device uses local data to calculate model updates. Generate random numbers locally Update the model perturbation as follows Then the perturbation data is encrypted as Returned to the edge server, where Indicates the use of edge server S e public key pk e Encrypt, then and It is returned to the auxiliary server along with the data.
[0061] Step 4: Robust aggregation for edge security: The auxiliary server and the edge server receive the perturbation gradients from the first M devices returned. and encrypted perturbation data Subsequently, a security-enhanced aggregation protocol for efficient collaborative communication is implemented to realize an asynchronous grouping robust aggregation algorithm based on perturbation gradients.
[0062] Step 4 specifically includes the following steps:
[0063] Step 4.1: The edge server performs delay-aware grouping on all received gradient information, assuming... Let be a set of devices selected in round s to participate in training and return gradients in round t. Let s be the set of devices selected in the s-th round. This refers to devices that were selected in round s but failed to successfully return the results to the server, and the groups satisfy the following conditions:
[0064] Step 4.2: Auxiliary Server S a For all received perturbation gradients Encryption is performed to obtain Indicates the use of auxiliary server S a public key pk a Encrypt it. And then... Send to edge server S e .
[0065] Step 4.3: Edge Server S e receive Then, through calculation get To prevent the auxiliary server from directly seeing the device's gradient information, a random number q is selected to perform blinding processing on the gradient information. Finally Send to auxiliary server S a .
[0066] Step 4.4: Auxiliary Server S a Upon receiving Afterwards, the auxiliary server decrypted and obtained... It also calculates the cosine distance between each gradient and the true gradient value within the group, and uses a thresholding mechanism to exclude abnormal cosine distances to avoid the continuous impact of Byzantine attackers on the global model. The equipment consists of devices with the same initial round. Represents the true value of the within-group gradient for the current iteration. Given a device that cannot predict the true value of the intra-group gradient in the current iteration, and assuming that model training typically converges stably, the difference between two consecutive model updates should be as small as possible. Therefore, the true value of the intra-group gradient in the current iteration is estimated using the global update from the previous iteration, expressed as:
[0067]
[0068] Here, in the case that most nodes are honest, it is reliable to sort the calculated cosine distances and use the median as a threshold.
[0069] Step 4.5: After filtering out anomalies based on the cosine distance according to the threshold, select a random number r to perform a secondary perturbation on the filtered gradient, and calculate the result. And Returned to the edge server. The edge server joins... get And compare each dimension to obtain the median. And Return to the auxiliary server.
[0070] Step 4.6: The auxiliary server removes the secondary perturbation value r to obtain... This serves as the initial ground truth for the gradient. Within a group of devices with similar latency, we can consider measuring the distance between each dimension and the ground truth gradient within the group to adjust the weights of the gradients uploaded by each device. Therefore, distance-related information is calculated and represented as:
[0071]
[0072]
[0073] The above distance information T total and T single Return to edge server S e .
[0074] Step 4.7: Edge Server S e Received T total and T singleThen, use the private key to decrypt and obtain and By adding and This gives the sum of the distances between the gradients of all devices and their true values. and the distance between the gradient of each device and the true gradient value. Next, the weight update value for each device can be calculated. Represented as:
[0075]
[0076] Where N represents the dimension of the gradient, and M represents... The number of devices included, i.e. dist(·) is a measurement and The function of the distance between them is calculated as follows:
[0077] Step 4.8: The edge server calculates the updated value based on the weight. And use the auxiliary server's public key to PK a Encryption is performed, then the sum of weighted gradients across all devices is calculated. Represented as:
[0078]
[0079] Sum of the weighted gradients of all the above devices The sum of the updated weight values of all devices Return to the auxiliary server.
[0080] Step 4.9: Auxiliary Server S a get and Then, use the public key to PK. a Decryption Next, calculate the true gradient value for each group. Represented as:
[0081]
[0082] And send it to the edge server S e .
[0083] Step 4.10: The edge server evaluates the ground truth of the gradient of the group according to different staleness. Delayed aggregation is represented as:
[0084]
[0085] Where Λ(τ) is an arbitrary decay function, here set as Λ(τ) = 1 / (1+τ), τ = ts. Finally, the edge server will... t+1 Send the data to the cloud server for global model aggregation.
[0086] Step 5: Cloud Global Model Aggregation: After receiving the local model aggregation results from each edge server, the cloud server uses the classic federated averaging algorithm to quickly aggregate the global model. If the global model metrics reach the training task's stopping criteria, local model training is stopped; otherwise, steps 3-5 are repeated.
[0087] Step 5 specifically includes the following steps:
[0088] Step 5.1: The cloud server receives the w returned by all edge servers. t+1 The global model is aggregated based on the federated average algorithm, as follows:
[0089]
[0090] in, These are the global model parameters in the (t+1)th round, n e n is the total amount of data contained in the edge region, and n is the total amount of data contained in all edge servers.
[0091] Step 5.2: Next, the cloud server will... The data is distributed to all edge servers, which then distribute it to all IoT devices within their jurisdiction for a new round of training until a predefined accuracy or model convergence is achieved.
[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A safe and efficient model co-construction method, characterized in that, The method specifically includes the following steps: S1: Region Division: Each edge server is divided into regions to manage based on its coverage capabilities, and manages users and physical entities within the regions; S2: System initialization: This includes two parts: model initialization and key generation; The model initialization includes: initializing global model parameters on the cloud server. The initial global model is sent to the edge server, which then sends it to IoT devices within its jurisdiction; the key generation includes: the edge server. and auxiliary server Generate your own public and private keys separately, and use them... , to indicate and The key pair; S3: Local Model Training: After receiving the global model, the IoT device uses local data to calculate model updates. And generate random numbers locally. Update the model perturbation as follows Then encrypt the perturbation data into Return to the edge server, and then and Returned together to the auxiliary server; among which Indicates the use of edge servers public key Encryption, N The dimension representing the gradient. Indicates the index of the device; S4: Robust aggregation for edge security: The auxiliary server and the edge server receive the first response. The perturbation gradient of each device and encrypted perturbation data Subsequently, a security-enhanced aggregation protocol for efficient collaborative communication is implemented to realize an asynchronous grouping robust aggregation algorithm based on perturbation gradients; Step S4 specifically includes the following steps: S41: The edge server performs delay-aware grouping on all received gradient information; S42: Auxiliary Server For all received perturbation gradients Encryption is performed to obtain and will Send to edge server ;in Indicates the use of an auxiliary server public key Encrypt; S43: Edge Server receive Then, through calculation get To avoid the auxiliary server directly seeing the device's gradient information, random numbers were selected. Obtain by blinding the gradient information Finally Send to auxiliary server ; S44: Auxiliary Server Upon receiving Afterwards, the auxiliary server decrypted and obtained... The model calculates the cosine distance between each gradient and the true value of the gradient within the group, and uses a threshold mechanism to exclude abnormal cosine distances to avoid the continuous impact of Byzantine attackers on the global model. In the case that most nodes are honest, the cosine distances are sorted and the median is used as the threshold. S45: After filtering out anomalies based on the cosine distance according to the threshold, select a random number. The filtered gradient is perturbed twice, and the result is calculated. and will Returned to the edge server; the edge server joins get And compare each dimension to obtain the median. and will Return to the auxiliary server; S46: The auxiliary server removes secondary perturbation values. get This serves as the initial gradient ground truth. Within a group of devices with similar latency, the weights of the gradients uploaded by each device are adjusted by measuring the distance between each dimension and the group's internal gradient ground truth, and this distance information is returned to the edge server. ; S47: Edge Server receive and Then, use the private key to decrypt and obtain and By adding and This means that the sum of the distances between the gradients of all devices and their true values can be obtained. And the distance between the gradient of each device and the true gradient value. Next, the weight update value for each device is calculated. ; S48: The edge server calculates based on the weight update value. and using the auxiliary server's public key Encryption is performed, then the sum of weighted gradients across all devices is calculated. And sum it with the weight update values of all devices. Return to the auxiliary server; S49: Auxiliary Server get and Then, use the public key. Decryption Next, calculate the true gradient value for each group. and send it to the edge server. ; S410: Edge servers evaluate the true value of the gradient of a group based on different degrees of antiquity. Delayed aggregation is represented as: in, Indicates the first Global model parameters of the wheel, Indicates the learning rate. Let be any decay function, set here as , s This indicates the round in which the device was selected to participate in the training. t This indicates the round in which the device returns the gradient; finally, the edge server will... Send the data to the cloud server for global model aggregation; S5: Cloud Global Model Aggregation: After receiving the local model aggregation results returned by each edge server, the cloud server uses the classic federated averaging algorithm to achieve rapid aggregation of the global model; if the global model index reaches the training task stopping standard, local model training is stopped, otherwise steps S3~S5 are repeated.
2. The model co-construction method according to claim 1, characterized in that, Step S41 specifically includes: Assuming For the first Participate in training in the round of selection and in the first A set of devices that cycle through the gradient, then Represented as the first The set of devices selected in the round-robin process Indicates in Devices selected in a round-robin process but failing to return results to the server, and meeting the requirements between groups. .
3. The model co-construction method according to claim 2, characterized in that, In step S44, the global update from the previous iteration is used to estimate the true value of the intra-group gradient in the current iteration. Cosine distance from the true gradient value within the group Represented as: in, Indicates the first Participate in training in the round of selection and in the first A set of devices that cycle through gradients.
4. The model co-construction method according to claim 3, characterized in that, In step S46, distance-related information is calculated and represented as follows: in, This represents information related to the sum of distances between all gradients and their true values within a group. This indicates information related to the distance between a single gradient within a group and the true gradient value.
5. The model co-construction method according to claim 4, characterized in that, In step S47, the weight update value for each device is calculated. , is represented as: in, The dimension representing the gradient. Represented as The number of devices included, i.e. ; It is a measurement and The function of the distance between them is calculated as follows: .
6. The model co-construction method according to claim 5, characterized in that, In step S48, the sum of weighted gradients for all devices is calculated. , is represented as: 。 7. The model co-construction method according to claim 6, characterized in that, In step S49, the true gradient value of each group is calculated. , is represented as: .
8. The model co-construction method according to claim 7, characterized in that, Step S5 specifically includes the following steps: S51: The cloud server receives responses from all edge servers. The global model is aggregated based on the federated average algorithm, as follows: in, It is the global first Global model parameters of the wheel, It is the total amount of data contained in the edge region. It is the total amount of data contained in all edge servers. , E Indicates the number of edge servers; S52: Next, the cloud server will... The data is distributed to all edge servers, which then distribute it to all IoT devices within their jurisdiction for a new round of training until a predefined accuracy or model convergence is achieved.