Edge device distributed training method and device based on multiple levels

Through multi-level distributed training methods for edge devices, matching devices and gateways, optimizing model parameter exchange mode, and combining reinforcement learning to adjust training load, the problem of low model training efficiency caused by terminal equipment heterogeneity is solved, and the overall intelligent system is improved Training efficiency.

CN119067198APending Publication Date: 2024-12-03SUZHOU INST FOR ADVANCED STUDY USTC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410981903.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the fields of intelligent transportation, intelligent medical care and intelligent agriculture, the heterogeneity of terminal devices leads to a longer model training time and reduced efficiency. How to reduce the impact of device heterogeneity on model training and improve the efficiency of joint model training.

Method used

Using a multi-level distributed training method for edge devices, the correlation index is maximized by matching the model training equipment with the gateway, the gateway matches the device layer, the model parameters are exchanged using synchronous and asynchronous modes, and the training load is adjusted using reinforcement learning algorithm until the center The server meets the convergence standard.

Benefits of technology

It significantly reduces the effect of the heterogeneity of the model training equipment on the efficiency of model parameter exchange, reduces training delay, and improves the efficiency of distributed training of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067198A_ABST
    Figure CN119067198A_ABST
Patent Text Reader

Abstract

The invention discloses an edge device distributed training method and device based on multiple levels, and the method comprises the steps: enabling each model training device to be matched with a gateway, and enabling the sum of correlation indexes between all model training devices and the matched gateways to be maximum; dividing all the matching devices corresponding to each gateway into a plurality of gateway matching device layers, and enabling the modularity among all the gateway matching device layers to meet a preset modularity division condition; the exchange of model parameters between matching devices in each gateway matching device layer is a synchronous mode, the exchange of other model parameters is an asynchronous mode, and joint training is executed based on the parameter exchange. By optimizing matching of the model training devices and the gateways, layering of the devices in the gateways and adoption of a multi-level parameter exchange mode, the influence of the heterogeneity of the model training devices on the exchange efficiency of the model parameters between the model training devices and the central server is reduced, and the efficiency of model distributed training is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a multi-level edge device distributed training method and device. Background Art

[0002] In the fields of smart transportation, smart healthcare, smart agriculture, etc., it is necessary to collect data from multiple terminal devices for model training. For data security reasons, each terminal device uses its own data to train a copy of the model locally, and then only sends the model parameters to a central server. The central server aggregates these model parameters to improve the model, and then sends the improved model back to each terminal device. The main advantage of this approach is that it protects data privacy and security because sensitive data does not need to leave its original location.

[0003] However, differences in terminal devices (such as processing power, storage capacity, and communication bandwidth) may lead to inconsistencies in data processing speed and model training capabilities. During training, it is necessary to wait for terminal devices with poor performance to complete tasks, thereby extending the overall training time and reducing efficiency. How to reduce the impact of device heterogeneity on model training time and improve the efficiency of joint model training between terminal devices has become an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a multi-level edge device distributed training method and apparatus, which can reduce the impact of the heterogeneity of model training devices on the efficiency of exchange of model parameters between each model training device and a central server, thereby improving the efficiency of model distributed training.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a multi-level model parameter aggregation method, which is applied to a federated learning system. The federated learning system includes three levels, which sequentially include a device layer composed of multiple model training devices, a gateway layer composed of multiple gateways, and a master control layer composed of a central server. The method includes:

[0006] Matching each of the model training devices with one of the gateways so that the sum of the correlation indexes between each of the model training devices and the gateways matched thereto in the federated learning system is maximized, wherein the correlation index refers to the data similarity and data transmission delay between one of the model training devices and one of the gateways;

[0007] All matching devices corresponding to each of the gateways are divided into multiple gateway matching device layers, so that the modularity between all the gateway matching device layers meets the preset modularity division condition; all the matching devices corresponding to each of the gateways refer to all the model training devices matched therewith; the modularity division condition refers to that the difference in data interaction efficiency between all the matching devices in one gateway matching device layer is less than the preset maximum interaction efficiency threshold, and the difference in data interaction efficiency between all the matching devices in different gateway matching device layers is greater than the preset minimum interaction efficiency threshold;

[0008] The model parameters are exchanged among the three levels, wherein the exchange mode of the model parameters between the matching devices in each gateway matching device layer is a synchronous mode, the exchange mode of the model parameters between each gateway matching device layer and its corresponding gateway is an asynchronous mode, the exchange mode of the model parameters between each gateway is an asynchronous mode, and the exchange mode of the model parameters between each gateway and the central server is an asynchronous mode; based on the exchange of model parameters, joint model training is performed at the three levels until the model trained by the central server reaches a preset convergence standard.

[0009] As an optional embodiment, in the first aspect of the present invention, based on the exchange of model parameters, joint model training is performed at the three levels until the model trained by the central server reaches a preset convergence standard, including: based on the exchange of model parameters, joint model training is performed at the three levels, and in each round of model training, the training load of each gateway matching device layer is used as one arm in the multi-armed bandit algorithm based on reinforcement learning, and the upper confidence limit strategy is used to adjust the training load of the gateway matching device layer until the model trained by the central server reaches a preset convergence standard.

[0010] As an optional implementation, in the first aspect of the present invention, matching each of the model training devices with one of the gateways so that the sum of the association indexes between each of the model training devices and the gateway matched thereto in the federated learning system is maximized comprises:

[0011] In each round of device gateway matching iteration, each model training device that has not established a matching relationship with any of the gateways is used as a device to be matched, and the gateway with the highest association index with one of the devices to be matched is used as its target gateway. For each device to be matched, it is determined in turn whether the number of model training devices that have established a matching relationship with the corresponding target gateway in the current round of device gateway matching iteration exceeds the preset single-round maximum matching number;

[0012] If it exceeds, then find the model training device with the lowest association index among all the model training devices that have been matched with the target gateway as a low-matching device, and judge whether the association index between the low-matching device and the target gateway is less than the association index between the device to be matched and the target gateway; if the association index between the low-matching device and the target gateway is less than the association index between the device to be matched and the target gateway, then cancel the matching relationship between the low-matching device and the target gateway, and establish a matching relationship between the device to be matched and the target gateway; if the association index between the low-matching device and the target gateway is greater than or equal to the association index between the device to be matched and the target gateway, then do not establish a matching relationship between the device to be matched and the target gateway;

[0013] If, in the device gateway matching iteration of the current round, the number of the model training devices that have established a matching relationship with the target gateway does not exceed the maximum matching number of a single round, a matching relationship is established between the device to be matched and the target gateway;

[0014] The device-gateway matching iteration is repeatedly performed until all the model training devices establish a matching relationship with one of the gateways.

[0015] As an optional implementation manner, in the first aspect of the present invention, the method for obtaining the association index includes:

[0016] The training data generated by the model training device are divided into multiple batches, and multiple rounds of model training are performed on the model training device. The average difference in gradients of all batches in one round of model training and the difference in gradients between this round of model training and the previous round of model training are combined as the heterogeneity parameters of the model training device in this round of model training, and the combination of the heterogeneity parameters of all rounds of model training is used as the heterogeneity parameters of the model training device;

[0017] Calculate the average time taken by the model training device to perform one model training and the average time taken by the model training device to send the model parameters to one of the gateways, and use the sum of the two as a transmission delay parameter between the model training device and the gateway;

[0018] For each of the gateways, calculating the mean value of the heterogeneity parameters of all the model training devices matched therewith as the heterogeneity parameter of the gateway;

[0019] Using the data similarity between the heterogeneity parameter of one of the gateways and the heterogeneity parameter of one of the model training devices as the data association degree between the model training device and the gateway;

[0020] When a gateway is not matched with any of the model training devices, the association index between the gateway and any of the model training devices is inversely proportional to the transmission delay parameter between the two. When a gateway is matched with any of the model training devices, the association index between the gateway and the model training device is inversely proportional to the transmission delay parameter between the two and is directly proportional to the data association between the two.

[0021] As an optional implementation, in the first aspect of the present invention, the dividing all the matching devices corresponding to the gateway into a plurality of gateway matching device layers so that the modularity between all the gateway matching device layers meets a preset modularity division condition includes:

[0022] Each of the model training devices is regarded as a community, and each community is regarded as a node to construct a fully connected weight network, wherein the weight of the edge between two nodes represents the training utility between the two nodes;

[0023] Determine whether the training utility between the two nodes exceeds the global average utility, and if so, disconnect the connection between the two nodes, wherein the global average utility refers to the ratio of the sum of the training utilities of all the model training devices to the number of the model training devices;

[0024] Defining the modularity Q among communities m for:

[0025]

[0026] in, is the sum of the weights of all edges in the weighted network, E a and E b are the degrees of the a-th node and the b-th node respectively, Utils (a,b) represents the training utility between the a-th node and the b-th node, c a and c b represent the communities to which the a-th node and the b-th node belong respectively, represents the indicator function;

[0027] Repeat the iterative movement of each node step until moving any of the nodes does not make the modularity Q m Increase, and obtain an updated community; wherein, the iterative movement of each node refers to determining whether moving each node to its adjacent community can improve the modularity Q m ,Only when the judgment result is that it can be improved, the node is moved to the adjacent community;

[0028] Iterate the community merging operation until the community merging operation cannot make the modularity Q m The value of increases to obtain several final communities; wherein, the community merging operation refers to taking each of the updated communities as a new node to update the weight network, and for any two of the new nodes: new node A and new node B, the weight between the two is the sum of the weights of the edges between all nodes constituting the new node A and any node constituting the new node B before the current update of the weight network, and each of the new nodes is moved by iteratively moving each node in the weight network until the modularity Q is not affected by moving any of the new nodes. m Increase;

[0029] All the model training devices corresponding to each of the final communities form a gateway matching device layer.

[0030] As an optional implementation, in the first aspect of the present invention, the upper limit of the confidence interval in the upper confidence limit strategy is the sum of the combined reward of the gateway matching device layer and a predefined exploration factor;

[0031] And, the using the upper confidence limit strategy to adjust the training load of the gateway matching device layer includes:

[0032] Sort all the gateway matching device layers in ascending or descending order according to the upper limit of the confidence interval, calculate the difference between the upper limits of the confidence interval of every two adjacent gateway matching device layers, take the gateway matching device layer with the larger upper limit of the confidence interval among the two gateway matching device layers with the largest difference as the reference device layer, take the reference device layer and all gateway matching device layers whose upper limits of the confidence interval exceed the reference device layer as the upward-adjusted gateway matching device layers, and take the other gateway matching device layers as the downward-adjusted gateway matching device layers;

[0033] The training load of each gateway matching device layer in the upward gateway matching device layer is adjusted to K(1+π), and the training load of each gateway matching device layer in the downward gateway matching device layer is adjusted to K(1-π), wherein K represents the original training load of the gateway matching device layer, and π is a hyperparameter for adjusting the training load.

[0034] As an optional implementation, in the first aspect of the present invention, the method for determining the combined reward includes:

[0035] The ratio of the system reward of the gateway matching device layer to the data reward of the gateway matching device layer is used as the combined reward of the gateway matching device layer;

[0036] The data reward is determined in the following ways:

[0037] The ratio of the average training loss of one gateway matching device layer in the process of exchanging the model parameters with the gateway matched therewith to the average training loss of all the gateway matching device layers under the gateway is used as the data reward of the gateway matching device layer;

[0038] The method for determining the system reward includes:

[0039] The ratio of the average delay of one gateway matching device layer in the process of exchanging the model parameters with the gateway matched therewith to the average delay of all the gateway matching device layers under the gateway is used as the system reward of the gateway matching device layer.

[0040] The second aspect of the present invention discloses a multi-level distributed training device for edge devices, which is applied to a federated learning system. The federated learning system includes three levels, which sequentially include a device layer composed of multiple model training devices, a gateway layer composed of multiple gateways, and a master control layer composed of a central server. The device includes:

[0041] A matching module, used to match each of the model training devices with one of the gateways, so that the sum of the association indexes between each of the model training devices in the federated learning system and the gateways matched therewith is maximized, wherein the association index refers to the data similarity and data transmission delay between one of the model training devices and one of the gateways;

[0042] A layering module is used to divide all matching devices corresponding to each of the gateways into multiple gateway matching device layers, so that the modularity between all the gateway matching device layers meets the preset modularity division condition; all the matching devices corresponding to each of the gateways refer to all the model training devices matched therewith; the modularity division condition refers to that the difference in data interaction efficiency between all the matching devices in one gateway matching device layer is less than a preset maximum interaction efficiency threshold, and the difference in data interaction efficiency between all the matching devices in different gateway matching device layers is greater than a preset minimum interaction efficiency threshold;

[0043] A training module is provided for exchanging the model parameters among the three levels, wherein the exchange mode of the model parameters between the matching devices in each gateway matching device layer is a synchronous mode, the exchange mode of the model parameters between each gateway matching device layer and its corresponding gateway is an asynchronous mode, the exchange mode of the model parameters between each gateway is an asynchronous mode, and the exchange mode of the model parameters between each gateway is an asynchronous mode, and the exchange mode of the model parameters between each gateway and the central server is an asynchronous mode; based on the exchange of model parameters, joint model training is performed at the three levels until the model trained by the central server reaches a preset convergence standard.

[0044] The third aspect of the present invention discloses another multi-level edge device distributed training device, the device comprising:

[0045] A memory storing executable program code;

[0046] a processor coupled to the memory;

[0047] The processor calls the executable program code stored in the memory to execute the multi-level edge device distributed training method disclosed in the first aspect of the present invention.

[0048] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the multi-level edge device distributed training method disclosed in the first aspect of the present invention.

[0049] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0050] The present invention significantly reduces the impact of the heterogeneity of model training devices on the efficiency of exchange of model parameters between various model training devices and the central server by optimizing the matching of model training devices and gateways, device layering within the gateway, and adopting a multi-level parameter exchange mode, thereby reducing training delays and improving the efficiency of distributed model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0052] Figure 1 is a schematic diagram of the structure of the federated learning system disclosed in an embodiment of the present invention;

[0053] Figure 2It is a flowchart of a multi-level edge device distributed training method disclosed in an embodiment of the present invention;

[0054] Figure 3 This is an example of a round of device gateway matching iteration when the maximum number of matches in a single round disclosed in an embodiment of the present invention is set to 1;

[0055] Figure 4 It is a flowchart of another multi-level edge device distributed training method disclosed in an embodiment of the present invention;

[0056] Figure 5 It is a structural schematic diagram of a multi-level edge device distributed training device disclosed in an embodiment of the present invention;

[0057] Figure 6 It is a structural schematic diagram of another multi-level edge device distributed training device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0060] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0061] The present invention discloses a multi-level edge device distributed training method and device, which can be applied to a federated learning system. Figure 1The following is a schematic diagram of the structure of the federated learning system. The federated learning system consists of three layers, which are the device layer consisting of multiple model training devices, the gateway layer consisting of multiple gateways, and the master control layer consisting of a central server. The following are detailed descriptions of each layer.

[0062] Embodiment 1

[0063] See also Figure 2 , Figure 2 : is a flowchart of a multi-level edge device distributed training method disclosed in an embodiment of the present invention. Figure 2 The described multi-level edge device distributed training method can be applied to the fields of intelligent transportation, intelligent medical care, intelligent agriculture, etc., and the embodiments of the present invention are not limited thereto. Figure 1 As shown, the multi-level edge device distributed training method may include the following operations:

[0064] 101. Match each model training device with a gateway so that the sum of the correlation index between each model training device and its matching gateway in the federated learning system is maximized. The correlation index refers to the data similarity and data transmission delay between a model training device and a gateway.

[0065] 102. Divide all matching devices corresponding to each gateway into multiple gateway matching device layers, so that the modularity between all gateway matching device layers meets the preset modularity division condition; all matching devices corresponding to each gateway refer to all model training devices matched therewith; the modularity division condition refers to that the difference in data interaction efficiency between all matching devices in a gateway matching device layer is less than the preset maximum interaction efficiency threshold, and the difference in data interaction efficiency between all matching devices in different gateway matching device layers is greater than the preset minimum interaction efficiency threshold;

[0066] 103. Exchange model parameters among three levels, wherein the exchange mode of model parameters between matching devices in each gateway matching device layer is synchronous mode, the exchange mode of model parameters between each gateway matching device layer and its corresponding gateway is asynchronous mode, the exchange mode of model parameters between each gateway is asynchronous mode, and the exchange mode of model parameters between each gateway and the central server is asynchronous mode; based on the exchange of model parameters, perform joint model training at the three levels until the model trained by the central server reaches a preset convergence standard.

[0067] In this embodiment, each gateway aggregates the model parameters of its corresponding matching device, and the central server aggregates the model parameters of all gateways.

[0068] It can be seen that this embodiment reduces the data transmission volume and delay by performing optimal matching based on the data similarity and transmission delay between the model training device and the gateway; at the same time, the devices in the gateway are layered according to the data interaction efficiency between the model training devices, and synchronous exchange is adopted within the same layer, and asynchronous exchange is adopted between different layers, thereby reducing waiting delays and reducing the impact of the heterogeneity of the model training devices on the efficiency of the exchange of model parameters between each model training device and the central server, thereby improving the efficiency of distributed model training.

[0069] In an optional embodiment, each model training device is matched with a gateway so that the sum of the association indexes between each model training device and its matching gateway in the federated learning system is maximized, including:

[0070] In each round of device gateway matching iteration, each model training device that has not established a matching relationship with any gateway is used as a device to be matched, and the gateway with the highest association index with a device to be matched is used as its target gateway. For each device to be matched, it is determined whether the number of model training devices that have established a matching relationship with its corresponding target gateway in the current round of device gateway matching iteration exceeds the preset maximum number of matches in a single round.

[0071] If it exceeds, then find the model training device with the lowest association index with the target gateway as the low-matching device among all the model training devices that have been matched with it, and judge whether the association index between the low-matching device and the target gateway is less than the association index between the device to be matched and the target gateway. If the association index between the low-matching device and the target gateway is less than the association index between the device to be matched and the target gateway, then the matching relationship between the low-matching device and the target gateway is released, and a matching relationship between the device to be matched and the target gateway is established; if the association index between the low-matching device and the target gateway is greater than or equal to the association index between the device to be matched and the target gateway, then the matching relationship between the device to be matched and the target gateway is not established;

[0072] If in the current round of device gateway matching iteration, the number of model training devices that have established a matching relationship with the target gateway does not exceed the maximum number of matches in a single round, a matching relationship is established between the device to be matched and the target gateway;

[0073] Repeat the device-gateway matching iteration until all model training devices are matched with a gateway.

[0074] Figure 3 An example of a round of device gateway matching iterations is shown when the maximum number of matches per round is set to 1. Figure 3 The federated learning system shown has four model training devices and three gateways. Figure 3The line on the left indicates that the third model training device and the fourth model training device both select the first gateway as the target gateway, and the association index between the first gateway and the third model training device is lower than the association index between the first gateway and the fourth model training device. When the third model training device matches the first gateway, since the first gateway has not matched any model training device in this round of device gateway matching iterations, a matching relationship can be established for the third model training device and the first gateway. When the fourth model training device matches the first gateway, the first gateway has been matched with a model training device in this round of device gateway matching iterations, reaching the maximum number of matches in a single round. By comparing the association index, the matching relationship between the first gateway and the third model training device is released, and a matching relationship between the first gateway and the fourth model training device is established.

[0075] It can be seen that in each round of device gateway matching iteration, this embodiment reduces the probability of gateway matching overload by setting the maximum number of matches in a single round, and achieves the global optimal matching of devices and gateways under the constraint of the maximum number of matches in a single round, thereby improving the efficiency of model parameter exchange between each model training device and the central server, thereby improving the efficiency of model distributed training.

[0076] In another optional embodiment, the method for obtaining the association index includes:

[0077] The training data generated by a model training device is divided into multiple batches, and multiple rounds of model training are performed on the model training device. The average difference in gradients of all batches in one round of model training and the difference in gradients between this round of model training and the previous round of model training are used as the heterogeneity parameters of the model training device in this round of model training. The combination of the heterogeneity parameters of all rounds of model training is used as the heterogeneity parameters of the model training device.

[0078] Calculate the average time it takes for the model training device to perform one model training and the average time it takes for the model training device to send model parameters to a gateway, and use the sum of the two as the transmission delay parameter between the model training device and the gateway;

[0079] For each gateway, the mean value of the heterogeneity parameters of all model training devices matching it is calculated as the heterogeneity parameter of the gateway;

[0080] The data similarity between the heterogeneous parameters of a gateway and the heterogeneous parameters of a model training device is used as the data correlation between the model training device and the gateway;

[0081] When a gateway is not matched with any model training device, the association index between the gateway and any model training device is inversely proportional to the transmission delay parameter between the two. When a gateway is matched with any model training device, the association index between the gateway and the model training device is inversely proportional to the transmission delay parameter between the two and is directly proportional to the data correlation between the two.

[0082] For example, the average difference in gradients of all batches of model training on model training device i in round t is for:

[0083]

[0084] Among them, K i represents the total number of batches of training data processed by model training device i in one round of training, k is the index of the batch in each round of training, Represents the gradient vector corresponding to the k-th batch of training data on model training device i in the t-th round of training, and T is the number of training rounds.

[0085] The gradient difference of model training device i from round t-1 to round t for:

[0086]

[0087] The heterogeneity parameter Γ of model training device i i for:

[0088]

[0089] Among them, flatten() represents the operation of converting a multi-dimensional vector into a one-dimensional vector.

[0090] Transmission delay parameter Ω between model training device i and gateway m (i,m) for:

[0091]

[0092] in, Represents the average time it takes for model training device i to perform one model training. Represents the average time it takes for model training device i to send model parameters to gateway m, Represents transmission delay, RTT (i,m) Represents the propagation delay of model parameters from model training device i to gateway m.

[0093] The heterogeneity parameter Γ of gateway m m for:

[0094]

[0095] Among them, A m Represents the set of all model training devices associated with gateway m, and || represents a conditional symbol.

[0096] JS divergence J (i,m) Represents the data association between model training device i and gateway m:

[0097]

[0098] Among them, KL() represents KL (Kullback-Leibler) divergence, ||||2 represents 2-norm, and | represents a parameter separator.

[0099] When gateway m matches any model training device, its association index Utils with model training device i (i,m) for:

[0100]

[0101] When gateway m does not match any model training device, the association index Utils between it and model training device i (i,m) for:

[0102]

[0103] φ m =1+(|D m | / |D|);

[0104] Among them, θ m Represents the preset device delay threshold, represents the indicator function, δ represents the hyperparameter that adjusts the intensity of the effect of delay on training utility, and φ m Represents the device matching optimization factor, D m represents the number of training data of all model training devices matching gateway m, and D represents the number of training data of all model training devices. m The setting method can be, in Ω (i,m) Select the model training devices with the largest values ​​from the N models, and use the network delay between the two model training devices with the largest network delay as θ m , Υ is the integer ratio of the number of gateway matching device layers corresponding to gateway m to the number of matching devices corresponding to gateway m.

[0105] It can be seen that this optional embodiment takes into account the similarity and transmission delay of the data of the model training device, and can more comprehensively and accurately measure the matching degree between the model training device and the gateway, thereby improving the efficiency of the exchange of model parameters between each model training device and the central server, and thus improving the efficiency of model distributed training. Increasing the device matching preference factor makes the model training device more inclined to connect to the gateway with a smaller number of matching model trainings, optimizes resource allocation, and helps to improve the stability of the federated learning system.

[0106] In yet another optional embodiment, all matching devices corresponding to the gateway are divided into a plurality of gateway matching device layers, so that the modularity between all gateway matching device layers meets a preset modularity division condition, including:

[0107] Each model training device is regarded as a community, and each community is regarded as a node to build a fully connected weight network. The weight of the edge between two nodes represents the training utility between the two nodes.

[0108] Determine whether the training utility between two nodes exceeds the global average utility. If so, disconnect the two nodes. The global average utility refers to the ratio of the sum of the training utilities of all model training devices to the number of model training devices.

[0109] Defining the modularity Q among communities m for:

[0110]

[0111] in, is the sum of the weights of all edges in the weighted network, E a and E b are the degrees of the a-th node and the b-th node respectively, Utils (a,b) represents the training utility between the a-th node and the b-th node, c a and c b Respectively represent the communities to which the a-th node and the b-th node belong, represents the indicator function;

[0112] Repeat the iterative move each node step until moving any node does not make the modularity Q m Increase, and get the updated community; among them, iteratively moving each node means judging whether moving each node to its adjacent community can improve the modularity Q m ,Only when the judgment result is that it can be improved, the node is moved to the adjacent community;

[0113] Iterate the community merging operation until the community merging operation cannot make the modularity Q mThe value of increases, and several final communities are obtained; among them, the community merging operation refers to taking each updated community as a new node, updating the weight network, and for any two new nodes: new node A and new node B, the weight between the two is the sum of the weights of the edges between all nodes constituting new node A and any node constituting new node B before this update of the weight network, and each new node is moved by iteratively moving each node in the weight network until moving any new node does not make the modularity Q m Increase;

[0114] All model training devices corresponding to each final community constitute a gateway matching device layer.

[0115] In one example, the training utility between two nodes at initialization is the actual connection weight of the two nodes. If the two nodes are connected, the training utility is 1, and if the two nodes are not connected, the training utility is 0.

[0116] It can be seen that this optional embodiment can efficiently divide all matching devices corresponding to each gateway into multiple gateway matching device layers by constructing a fully connected weight network and gradually optimizing the community modularity, so that the data interaction efficiency of devices within a layer is high and the difference in data interaction efficiency between layers is significant, thereby optimizing the synchronous and asynchronous exchange process of model parameters and improving the overall training efficiency and resource utilization of the federated learning system.

[0117] Embodiment 2

[0118] See also Figure 4 , Figure 4 1 is a flow chart of another multi-level edge device distributed training method disclosed in an embodiment of the present invention. Figure 4 The described multi-level edge device distributed training method can be applied to the fields of intelligent transportation, intelligent medical care, intelligent agriculture, etc., and the embodiments of the present invention are not limited thereto. Figure 4 As shown, the multi-level edge device distributed training method may include the following operations:

[0119] 201. Match each model training device with a gateway so that the sum of the association index between each model training device and its matching gateway in the federated learning system is maximized. The association index refers to the data similarity and data transmission delay between a model training device and a gateway.

[0120] 202. Divide all matching devices corresponding to each gateway into multiple gateway matching device layers, so that the modularity between all gateway matching device layers meets the preset modularity division condition; all matching devices corresponding to each gateway refer to all model training devices matched therewith; the modularity division condition refers to that the difference in data interaction efficiency between all matching devices in a gateway matching device layer is less than the preset maximum interaction efficiency threshold, and the difference in data interaction efficiency between all matching devices in different gateway matching device layers is greater than the preset minimum interaction efficiency threshold;

[0121] 203. Exchange model parameters among three levels, wherein the exchange mode of model parameters between matching devices in each gateway matching device layer is synchronous mode, the exchange mode of model parameters between each gateway matching device layer and its corresponding gateway is asynchronous mode, the exchange mode of model parameters between each gateway is asynchronous mode, and the exchange mode of model parameters between each gateway and the central server is asynchronous mode;

[0122] 204. Based on the exchange of model parameters, in each round of model training, the training load of each gateway matching device layer is used as one arm in the multi-armed bandit algorithm based on reinforcement learning, and the upper confidence limit strategy is used to adjust the training load of the gateway matching device layer until the model trained by the central server reaches the preset convergence standard.

[0123] In the embodiment of the present invention, for the description of steps 201-203, please refer to the detailed description of steps 101-103 in the first embodiment, which will not be repeated here.

[0124] It can be seen that this embodiment can effectively optimize resource allocation, improve training efficiency, and ensure that the model quickly reaches the preset convergence standard by applying the multi-armed bandit algorithm based on reinforcement learning in each round of model training and using the upper confidence limit strategy to dynamically adjust the training load of the gateway matching device layer, thereby enhancing the overall performance and stability of the federated learning system.

[0125] In another optional embodiment, the upper limit of the confidence interval in the upper confidence limit strategy is the sum of the combined reward of the gateway matching device layer and the predefined exploration factor;

[0126] And, use the upper confidence limit strategy to adjust the training load of the gateway matching device layer, including:

[0127] Sort all gateway matching device layers in ascending or descending order according to the upper limit of the confidence interval, calculate the difference between the upper limits of the confidence intervals of every two adjacent gateway matching device layers, take the gateway matching device layer with the larger upper limit of the confidence interval among the two gateway matching device layers with the largest difference as the reference device layer, take the reference device layer and all gateway matching device layers whose upper limits of the confidence intervals exceed the reference device layer as the upward gateway matching device layers, and take the other gateway matching device layers as the downward gateway matching device layers;

[0128] The training load of each gateway matching device layer in the upward gateway matching device layer is adjusted to K(1+π), and the training load of each gateway matching device layer in the downward gateway matching device layer is adjusted to K(1-π), where K represents the original training load of the gateway matching device layer, and π is a hyperparameter for adjusting the training load.

[0129] For example, in the tth round of training, the hth gateway matches the upper limit of the confidence interval of the device layer UCB h (t) is:

[0130]

[0131] Among them, Δ(t) represents the sum of the number of times all device layers are selected in the first t rounds of training, Δ h (t) represents the number of times the hth gateway matching device layer is selected in the first t rounds of training, CR h (t) represents the combined reward of the hth gateway matching device layer.

[0132] It can be seen that this optional embodiment determines the gateway matching device layer for increasing and decreasing the load through sorting and difference calculation, accurately controls the load distribution, and makes the training resources more reasonably distributed in each device layer, thereby accelerating the convergence of the model and improving the overall performance and stability of the federated learning system.

[0133] In yet another optional embodiment, the combination reward is determined by:

[0134] The ratio of the system reward of a gateway matching device layer to the data reward of the gateway matching device layer is used as the combined reward of the gateway matching device layer;

[0135] Among them, the methods for determining data rewards include:

[0136] The data reward for a gateway matching device layer is the ratio of the average training loss of a gateway matching device layer in the process of exchanging model parameters with its matching gateway to the average training loss of all gateway matching device layers under the gateway.

[0137] Among them, the determination methods of system rewards include:

[0138] The ratio of the average delay of a gateway matching device layer in the process of exchanging model parameters with its matching gateway to the average delay of all gateway matching device layers under the gateway is used as the system reward of the gateway matching device layer.

[0139] For example, the hth gateway matches the data reward DR at the device layer h (t) is:

[0140]

[0141] Among them, S h represents the total number of model parameter exchanges between the hth gateway matching device layer under the gateway and the gateway in the tth round of training between the gateway and the central server, and s represents the number of model parameter exchanges between the hth gateway matching device layer under the gateway and the gateway in the tth round of training between the gateway and the central server. represents the average training loss of the hth gateway matching device layer in all model trainings, H represents the total number of gateway matching device layers under the gateway, ω (t,s) Represents the model parameters exchanged between the hth gateway matching device layer and the gateway for the sth time in the tth round of training between the gateway and the central server.

[0142] The hth gateway matches the system reward SR of the device layer h (t) is:

[0143]

[0144] Among them, U (h,s) represents the delay of the hth gateway matching device layer in the sth model training, U h Represents the latency of the h-th gateway matching device layer in all model trainings.

[0145] The combined reward CR of the hth gateway matching device layer h (t) is:

[0146]

[0147] It can be seen that this optional embodiment determines the combined reward of the gateway matching device layer by the ratio of the system reward to the data reward, and calculates the data reward and the system reward based on the average training loss and the average delay respectively. This can more accurately reflect the performance and efficiency of each device layer, and can not only dynamically adjust the training load and optimize resource allocation more effectively, but also balance the training speed and system delay, thereby improving the overall training efficiency and stability of the federated learning system and accelerating the convergence of the model.

[0148] Embodiment 3

[0149] See also Figure 5 , Figure 5 1 is a schematic diagram of a multi-level distributed training device for edge devices disclosed in an embodiment of the present invention. Figure 5 The device shown can be applied to the fields of intelligent transportation, intelligent medical treatment, intelligent agriculture, etc., and the embodiments of the present invention are not limited thereto. Figure 5 The device shown includes:

[0150] Matching module 501, used to match each model training device with a gateway so that the sum of the correlation index between each model training device and its matching gateway in the federated learning system is maximized, where the correlation index refers to the data similarity and data transmission delay between a model training device and a gateway;

[0151] The layering module 502 is used to divide all matching devices corresponding to each gateway into multiple gateway matching device layers, so that the modularity between all gateway matching device layers meets the preset modularity division condition; all matching devices corresponding to each gateway refer to all model training devices matched therewith; the modularity division condition refers to that the difference in data interaction efficiency between all matching devices in a gateway matching device layer is less than the preset maximum interaction efficiency threshold, and the difference in data interaction efficiency between all matching devices in different gateway matching device layers is greater than the preset minimum interaction efficiency threshold;

[0152] The training module 503 is used to exchange model parameters among three levels, wherein the exchange mode of model parameters between matching devices in each gateway matching device layer is a synchronous mode, the exchange mode of model parameters between each gateway matching device layer and its corresponding gateway is an asynchronous mode, the exchange mode of model parameters between each gateway is an asynchronous mode, and the exchange mode of model parameters between each gateway is an asynchronous mode, and the exchange mode of model parameters between each gateway and the central server is an asynchronous mode; based on the exchange of model parameters, joint model training is performed at the three levels until the model trained by the central server reaches a preset convergence standard.

[0153] It can be seen that this embodiment reduces the data transmission volume and delay by performing optimal matching based on the data similarity and transmission delay between the model training device and the gateway; at the same time, the devices in the gateway are layered according to the data interaction efficiency between the model training devices, and synchronous exchange is adopted within the same layer, and asynchronous exchange is adopted between different layers, thereby reducing waiting delays and reducing the impact of the heterogeneity of the model training devices on the efficiency of the exchange of model parameters between each model training device and the central server, thereby improving the efficiency of distributed model training.

[0154] In an optional embodiment, each model training device is matched with a gateway so that the sum of the association indexes between each model training device and its matching gateway in the federated learning system is maximized, including:

[0155] In each round of device gateway matching iteration, each model training device that has not established a matching relationship with any gateway is used as a device to be matched, and the gateway with the highest association index with a device to be matched is used as its target gateway. For each device to be matched, it is determined whether the number of model training devices that have established a matching relationship with its corresponding target gateway in the current round of device gateway matching iteration exceeds the preset maximum number of matches in a single round.

[0156] If it exceeds, then find the model training device with the lowest association index with the target gateway as the low-matching device among all the model training devices that have been matched with it, and judge whether the association index between the low-matching device and the target gateway is less than the association index between the device to be matched and the target gateway. If the association index between the low-matching device and the target gateway is less than the association index between the device to be matched and the target gateway, then the matching relationship between the low-matching device and the target gateway is released, and a matching relationship between the device to be matched and the target gateway is established; if the association index between the low-matching device and the target gateway is greater than or equal to the association index between the device to be matched and the target gateway, then the matching relationship between the device to be matched and the target gateway is not established;

[0157] If in the current round of device gateway matching iteration, the number of model training devices that have established a matching relationship with the target gateway does not exceed the maximum number of matches in a single round, a matching relationship is established between the device to be matched and the target gateway;

[0158] Repeat the device-gateway matching iteration until all model training devices are matched with a gateway.

[0159] It can be seen that in each round of device gateway matching iteration, this embodiment reduces the probability of gateway matching overload by setting the maximum number of matches in a single round, and achieves the global optimal matching of devices and gateways under the constraint of the maximum number of matches in a single round, thereby improving the efficiency of model parameter exchange between each model training device and the central server, thereby improving the efficiency of model distributed training.

[0160] In another optional embodiment, the method for obtaining the association index includes:

[0161] The training data generated by a model training device is divided into multiple batches, and multiple rounds of model training are performed on the model training device. The average difference in gradients of all batches in one round of model training and the difference in gradients between this round of model training and the previous round of model training are used as the heterogeneity parameters of the model training device in this round of model training. The combination of the heterogeneity parameters of all rounds of model training is used as the heterogeneity parameters of the model training device.

[0162] Calculate the average time it takes for the model training device to perform one model training and the average time it takes for the model training device to send model parameters to a gateway, and use the sum of the two as the transmission delay parameter between the model training device and the gateway;

[0163] For each gateway, the mean value of the heterogeneity parameters of all model training devices matching it is calculated as the heterogeneity parameter of the gateway;

[0164] The data similarity between the heterogeneous parameters of a gateway and the heterogeneous parameters of a model training device is used as the data correlation between the model training device and the gateway;

[0165] When a gateway is not matched with any model training device, the association index between the gateway and any model training device is inversely proportional to the transmission delay parameter between the two. When a gateway is matched with any model training device, the association index between the gateway and the model training device is inversely proportional to the transmission delay parameter between the two and is directly proportional to the data correlation between the two.

[0166] It can be seen that this optional embodiment takes into account the similarity and transmission delay of the data of the model training device, and can more comprehensively and accurately measure the matching degree between the model training device and the gateway, thereby improving the efficiency of the exchange of model parameters between each model training device and the central server, and thus improving the efficiency of model distributed training. Increasing the device matching preference factor makes the model training device more inclined to connect to the gateway with a smaller number of matching model trainings, optimizes resource allocation, and helps to improve the stability of the federated learning system.

[0167] In yet another optional embodiment, all matching devices corresponding to the gateway are divided into a plurality of gateway matching device layers, so that the modularity between all gateway matching device layers meets a preset modularity division condition, including:

[0168] Each model training device is regarded as a community, and each community is regarded as a node to build a fully connected weight network. The weight of the edge between two nodes represents the training utility between the two nodes.

[0169] Determine whether the training utility between two nodes exceeds the global average utility. If so, disconnect the two nodes. The global average utility refers to the ratio of the sum of the training utilities of all model training devices to the number of model training devices.

[0170] Defining the modularity Q among communities m for:

[0171]

[0172] in, is the sum of the weights of all edges in the weighted network, E a and E b are the degrees of the a-th node and the b-th node respectively, Utils (a,b) represents the training utility between the a-th node and the b-th node, ca and c b Respectively represent the communities to which the a-th node and the b-th node belong, represents the indicator function;

[0173] Repeat the iterative move each node step until moving any node does not make the modularity Q m Increase, and get the updated community; among them, iteratively moving each node means judging whether moving each node to its adjacent community can improve the modularity Q m ,Only when the judgment result is that it can be improved, the node is moved to the adjacent community;

[0174] Iterate the community merging operation until the community merging operation cannot make the modularity Q m The value of increases, and several final communities are obtained; among them, the community merging operation refers to taking each updated community as a new node, updating the weight network, and for any two new nodes: new node A and new node B, the weight between the two is the sum of the weights of the edges between all nodes constituting new node A and any node constituting new node B before this update of the weight network, and each new node is moved by iteratively moving each node in the weight network until moving any new node does not make the modularity Q m Increase;

[0175] All model training devices corresponding to each final community constitute a gateway matching device layer.

[0176] It can be seen that this optional embodiment can efficiently divide all matching devices corresponding to each gateway into multiple gateway matching device layers by constructing a fully connected weight network and gradually optimizing the community modularity, so that the data interaction efficiency of devices within a layer is high and the difference in data interaction efficiency between layers is significant, thereby optimizing the synchronous and asynchronous exchange process of model parameters and improving the overall training efficiency and resource utilization of the federated learning system.

[0177] In another optional embodiment, based on the exchange of model parameters, performing joint model training at three levels until the model trained by the central server reaches a preset convergence standard includes:

[0178] Based on the exchange of model parameters, joint model training is performed at three levels. In each round of model training, the training load of each gateway matching device layer is used as one arm in the multi-armed bandit algorithm based on reinforcement learning. The upper confidence bound strategy is used to adjust the training load of the gateway matching device layer until the model trained by the central server reaches the preset convergence standard.

[0179] It can be seen that this embodiment can effectively optimize resource allocation, improve training efficiency, and ensure that the model quickly reaches the preset convergence standard by applying the multi-armed bandit algorithm based on reinforcement learning in each round of model training and using the upper confidence limit strategy to dynamically adjust the training load of the gateway matching device layer, thereby enhancing the overall performance and stability of the federated learning system.

[0180] Embodiment 4

[0181] See also Figure 6 , Figure 6 FIG. 1 is a schematic diagram of another structure of a multi-level edge device distributed training device disclosed in an embodiment of the present invention. Figure 6 As shown, the multi-level edge device distributed training apparatus may include:

[0182] A memory 601 storing executable program codes;

[0183] a processor 602 coupled to the memory 601;

[0184] Furthermore, it may also include an input interface 603 and an output interface 604 coupled to the processor 602;

[0185] The processor 602 calls the executable program code stored in the memory 601 to execute the steps in the multi-level edge device distributed training method described in Embodiment 1 or Embodiment 2 of the present invention.

[0186] Embodiment 5

[0187] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps in the multi-level edge device distributed training method described in Embodiment 1 or Embodiment 2 of the present invention.

[0188] Embodiment 6

[0189] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the multi-level edge device distributed training method described in Embodiment 1 or Embodiment 2.

[0190] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0191] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0192] Finally, it should be noted that the multi-level edge device distributed training method and apparatus disclosed in the embodiment of the present invention only discloses the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-level distributed training method for edge devices, the method is applied to a federated learning system, the federated learning system includes three levels, the three levels sequentially include a device layer composed of multiple model training devices, a gateway layer composed of multiple gateways, and a master control layer composed of a central server, characterized in that: The method comprises: Matching each of the model training devices with one of the gateways so that the sum of the correlation indexes between each of the model training devices and the gateways matched thereto in the federated learning system is maximized, wherein the correlation index refers to the data similarity and data transmission delay between one of the model training devices and one of the gateways; All matching devices corresponding to each of the gateways are divided into multiple gateway matching device layers, so that the modularity between all the gateway matching device layers meets the preset modularity division condition; all the matching devices corresponding to each of the gateways refer to all the model training devices matched therewith; the modularity division condition refers to that the difference in data interaction efficiency between all the matching devices in one gateway matching device layer is less than the preset maximum interaction efficiency threshold, and the difference in data interaction efficiency between all the matching devices in different gateway matching device layers is greater than the preset minimum interaction efficiency threshold; The model parameters are exchanged among the three levels, wherein the exchange mode of the model parameters between the matching devices in each gateway matching device layer is a synchronous mode, the exchange mode of the model parameters between each gateway matching device layer and its corresponding gateway is an asynchronous mode, the exchange mode of the model parameters between each gateway is an asynchronous mode, and the exchange mode of the model parameters between each gateway and the central server is an asynchronous mode; based on the exchange of model parameters, joint model training is performed at the three levels until the model trained by the central server reaches a preset convergence standard.

2. The multi-level edge device distributed training method according to claim 1, characterized in that: Based on the exchange of model parameters, joint model training is performed at the three levels until the model trained by the central server reaches a preset convergence standard, including: based on the exchange of model parameters, joint model training is performed at the three levels, in each round of model training, the training load of each gateway matching device layer is used as one arm in the multi-armed bandit algorithm based on reinforcement learning, and the upper confidence limit strategy is used to adjust the training load of the gateway matching device layer until the model trained by the central server reaches a preset convergence standard.

3. The multi-level edge device distributed training method according to claim 1, characterized in that: The step of matching each of the model training devices with one of the gateways so that the sum of the association indexes between each of the model training devices and the gateway matched thereto in the federated learning system is maximized includes: In each round of device gateway matching iteration, each model training device that has not established a matching relationship with any of the gateways is used as a device to be matched, and the gateway with the highest association index with one of the devices to be matched is used as its target gateway. For each device to be matched, it is determined in turn whether the number of model training devices that have established a matching relationship with the corresponding target gateway in the current round of device gateway matching iteration exceeds the preset single-round maximum matching number; If it exceeds, then find the model training device with the lowest association index among all the model training devices that have been matched with the target gateway as a low-matching device, and judge whether the association index between the low-matching device and the target gateway is less than the association index between the device to be matched and the target gateway; if the association index between the low-matching device and the target gateway is less than the association index between the device to be matched and the target gateway, then cancel the matching relationship between the low-matching device and the target gateway, and establish a matching relationship between the device to be matched and the target gateway; if the association index between the low-matching device and the target gateway is greater than or equal to the association index between the device to be matched and the target gateway, then do not establish a matching relationship between the device to be matched and the target gateway; If, in the device gateway matching iteration of the current round, the number of the model training devices that have established a matching relationship with the target gateway does not exceed the maximum matching number of a single round, a matching relationship is established between the device to be matched and the target gateway; The device-gateway matching iteration is repeatedly performed until all the model training devices establish a matching relationship with one of the gateways.

4. The multi-level edge device distributed training method according to claim 1, characterized in that: The method for obtaining the association index includes: The training data generated by the model training device are divided into multiple batches, and multiple rounds of model training are performed on the model training device. The average difference in gradients of all batches in one round of model training and the difference in gradients between this round of model training and the previous round of model training are combined as the heterogeneity parameters of the model training device in this round of model training, and the combination of the heterogeneity parameters of all rounds of model training is used as the heterogeneity parameters of the model training device; Calculate the average time taken by the model training device to perform one model training and the average time taken by the model training device to send the model parameters to one of the gateways, and use the sum of the two as a transmission delay parameter between the model training device and the gateway; For each of the gateways, calculating the mean of the heterogeneity parameters of all the model training devices matching the gateway as the heterogeneity parameter of the gateway; Using the data similarity between the heterogeneity parameter of one of the gateways and the heterogeneity parameter of one of the model training devices as the data association degree between the model training device and the gateway; When a gateway is not matched with any of the model training devices, the association index between the gateway and any of the model training devices is inversely proportional to the transmission delay parameter between the two. When a gateway is matched with any of the model training devices, the association index between the gateway and the model training device is inversely proportional to the transmission delay parameter between the two and is directly proportional to the data association between the two.

5. The multi-level edge device distributed training method according to claim 1, characterized in that: The step of dividing all the matching devices corresponding to the gateway into a plurality of gateway matching device layers so that the modularity between all the gateway matching device layers meets a preset modularity division condition includes: Each of the model training devices is regarded as a community, and each community is regarded as a node to construct a fully connected weight network, wherein the weight of the edge between two nodes represents the training utility between the two nodes; Determine whether the training utility between the two nodes exceeds the global average utility, and if so, disconnect the connection between the two nodes, wherein the global average utility refers to the ratio of the sum of the training utilities of all the model training devices to the number of the model training devices; Defining the modularity Q among communities m for: in, is the sum of the weights of all edges in the weighted network, E a and E b are the degrees of the a-th node and the b-th node respectively, Utils (a,b) represents the training utility between the a-th node and the b-th node, c a and c b represent the communities to which the a-th node and the b-th node belong respectively, represents the indicator function; Repeat the iterative movement of each node step until moving any of the nodes does not make the modularity Q m Increase, and obtain an updated community; wherein, the iterative movement of each node refers to determining whether moving each node to its adjacent community can improve the modularity Q m ,Only when the judgment result is that it can be improved, the node is moved to the adjacent community; Iterate the community merging operation until the community merging operation cannot make the modularity Q m The value of increases to obtain several final communities; wherein, the community merging operation refers to taking each of the updated communities as a new node to update the weight network, and for any two of the new nodes: new node A and new node B, the weight between the two is the sum of the weights of the edges between all nodes constituting the new node A and any node constituting the new node B before the current update of the weight network, and each of the new nodes is moved by iteratively moving each node in the weight network until the modularity Q is not affected by moving any of the new nodes. m Increase; All the model training devices corresponding to each of the final communities form a gateway matching device layer.

6. The multi-level edge device distributed training method according to claim 2, characterized in that: The upper limit of the confidence interval in the upper confidence limit strategy is the sum of the combined reward of the gateway matching device layer and the predefined exploration factor; And, the using the upper confidence limit strategy to adjust the training load of the gateway matching device layer includes: All the gateway matching device layers are sorted in ascending or descending order according to the upper limit of the confidence interval, and the difference between the upper limits of the confidence interval of every two adjacent gateway matching device layers is calculated, and the gateway matching device layer with the larger upper limit of the confidence interval among the two gateway matching device layers with the largest difference is used as the reference device layer, and the reference device layer and all gateway matching device layers whose upper limits of the confidence interval exceed the reference device layer are used as the upward gateway matching device layers, and the other gateway matching device layers are used as the downward gateway matching device layers; The training load of each gateway matching device layer in the upward gateway matching device layer is adjusted to K(1+π), and the training load of each gateway matching device layer in the downward gateway matching device layer is adjusted to K(1-π), wherein K represents the original training load of the gateway matching device layer, and π is a hyperparameter for adjusting the training load.

7. The multi-level edge device distributed training method according to claim 6, characterized in that: The combination reward is determined in the following ways: The ratio of the system reward of the gateway matching device layer to the data reward of the gateway matching device layer is used as the combined reward of the gateway matching device layer; The data reward is determined in the following ways: The ratio of the average training loss of one gateway matching device layer in the process of exchanging the model parameters with the gateway matched therewith to the average training loss of all the gateway matching device layers under the gateway is used as the data reward of the gateway matching device layer; The method for determining the system reward includes: The ratio of the average delay of one gateway matching device layer in the process of exchanging the model parameters with the gateway matched therewith to the average delay of all the gateway matching device layers under the gateway is used as the system reward of the gateway matching device layer.

8. A multi-level edge device distributed training device, characterized in that: The device is applied in a federated learning system, which includes three levels, which sequentially include a device layer consisting of multiple model training devices, a gateway layer consisting of multiple gateways, and a main control layer consisting of a central server, and is characterized in that the device includes: A matching module, used to match each of the model training devices with one of the gateways, so that the sum of the association indexes between each of the model training devices and the gateways matched thereto in the federated learning system is maximized, wherein the association index refers to the data similarity and data transmission delay between one of the model training devices and one of the gateways; A layering module is used to divide all matching devices corresponding to each of the gateways into multiple gateway matching device layers, so that the modularity between all the gateway matching device layers meets the preset modularity division condition; all the matching devices corresponding to each of the gateways refer to all the model training devices matched therewith; the modularity division condition refers to that the difference in data interaction efficiency between all the matching devices in one gateway matching device layer is less than a preset maximum interaction efficiency threshold, and the difference in data interaction efficiency between all the matching devices in different gateway matching device layers is greater than a preset minimum interaction efficiency threshold; A training module is used to exchange model parameters among the three levels, wherein the exchange mode of the model parameters between the matching devices in each gateway matching device layer is a synchronous mode, the exchange mode of the model parameters between each gateway matching device layer and its corresponding gateway is an asynchronous mode, the exchange mode of the model parameters between each gateway is an asynchronous mode, and the exchange mode of the model parameters between each gateway is an asynchronous mode, and the exchange mode of the model parameters between each gateway and the central server is an asynchronous mode; based on the exchange of model parameters, joint model training is performed at the three levels until the model trained by the central server reaches a preset convergence standard.

9. A multi-level edge device distributed training device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the multi-level edge device distributed training method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, which, when called, are used to execute the multi-level edge device distributed training method as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Hierarchical federated learning framework for efficient communication and optimization method and system thereof

    CN112070240A

  • Hierarchical federated learning method and device based on asynchronous communication, terminal equipment and storage medium

    CN112532451A

  • Wireless communication system for deploying hierarchical federated learning and resource optimization method

    CN114363911A

  • Artificial intelligence (AI) model training method and device in wireless network

    CN116264684A

  • Federal learning optimization method and system based on reinforcement learning

    CN116523079A