A heterogeneous edge device collaboration method and system based on industrial Internet of Things

Abstract

The present invention discloses a heterogeneous edge device collaboration method and system based on industrial Internet of Things, and relates to the technical field of edge device collaboration. The method steps include: constructing a heterogeneous edge device collaboration model, wherein the heterogeneous edge device collaboration model includes a slave edge node, a master edge node, and a deep reinforcement learning node; performing local training on the slave edge node to generate local parameters and training status; the master edge node generates global parameters based on local parameters; the deep reinforcement learning node feeds back training actions to the slave edge node based on the training status; the slave edge node performs communication iterative training of heterogeneous edge node collaboration based on global parameters and training actions, and completes heterogeneous edge node collaboration based on the communication iterative training results. Through the method of the present invention, the time difference between nodes in completing data processing can be effectively reduced, the accuracy and convergence speed of node training can be improved, the problem of node falling behind during node training can be solved, and industrial production can be carried out in an orderly and rapid manner.

Description

Technical Field

[0001] The present invention relates to the technical field of edge device collaboration, and in particular to a heterogeneous edge device collaboration method and system based on industrial Internet of Things. Background Art

[0002] In the industrial Internet of Things environment, the generated data needs to be processed as promptly as possible to monitor the operation of the business or edge devices, and to make predictions and early warnings for the business or edge devices through analysis.

[0003] At present, existing technologies usually use the synchronous update method to predict and warn services or edge devices. For synchronous update, the master edge node must collect the local parameters of all slave edge nodes before performing parameter aggregation. This method ensures consistency and convergence. However, it requires nodes to complete training as simultaneously as possible. Some existing studies assign different data to different nodes to minimize the waiting time for synchronous updates between nodes. Lyu et al. [1] proposed a dynamic method for partitioning streaming data in edge wireless networks to promote distributed machine learning. This method optimizes the partitioning of data sets from different slave edge nodes to ensure consistency and cost efficiency. Wu et al. [2] assigned different batch sizes to model training based on the resources used by each task in a mixed load scenario; this method reduces the total execution time of each round of model training operations, shortens the total training time and improves system efficiency. However, these methods do not consider the heterogeneity of edge nodes, and the availability of nodes may change dynamically, which may lead to problems such as decreased accuracy, increased parameter obsolescence, and slower convergence in node training. In addition, due to the different computing performance of some edge devices, if the data is simply distributed evenly to each device, the time required to complete data processing between devices will vary greatly, causing the problem of falling behind, increasing the total time required to process this batch of data, and causing the subsequent accumulation of data, which is not conducive to the orderly and rapid progress of industrial production.

[0004] References:

[0005] 1: Optimal online data partitioning for geo-distributed machinelearning in edge of wireless networks

[0006] 2: DQ-DPS data partition strategy based on distributed machine learning Summary of the invention

[0007] In order to overcome the problem that the prior art does not take into account the heterogeneity and dynamic changes of edge nodes, resulting in a large difference in the time to complete data processing between edge nodes, which in turn causes falling behind, the present invention proposes a heterogeneous edge device collaboration method and system based on industrial Internet of Things, which can effectively learn the heterogeneity and dynamic changes of edge nodes, reduce the time difference in completing data processing between edge nodes, and thus improve the accuracy and convergence speed of node training.

[0008] To achieve the purpose of the present invention, the present invention adopts the following technical solutions:

[0009] A heterogeneous edge device collaboration method based on industrial Internet of Things, the method comprising the following steps:

[0010] Constructing a heterogeneous edge device collaboration model, wherein the heterogeneous edge device collaboration model includes a slave edge node, a master edge node, and a deep reinforcement learning node;

[0011] Perform local training from edge nodes to generate local parameters and training status;

[0012] The master edge node generates global parameters based on local parameters;

[0013] The deep reinforcement learning node feeds back training actions to the edge nodes based on the training status;

[0014] The edge nodes perform communication iterative training for heterogeneous edge node collaboration according to global parameters and training actions, and complete the heterogeneous edge node collaboration according to the communication iterative training results.

[0015] In the above technical scheme, by performing local training on the slave edge nodes in the heterogeneous edge device collaboration model, the local parameters and training status of each slave edge node in the heterogeneous edge device collaboration model can be generated, so that the master edge node and the deep reinforcement learning node generate and feedback corresponding global parameters and training actions, and then the slave edge nodes perform communication iterative training for heterogeneous edge node collaboration according to the global parameters and training actions to complete the heterogeneous edge node collaboration; the set deep reinforcement learning nodes can effectively learn the heterogeneity and dynamic changes of the slave edge nodes, thereby reducing the time difference in completing data processing between the slave edge nodes, improving the accuracy and convergence speed of node training, solving the problem of node falling behind during node training, and enabling industrial production to proceed in an orderly and rapid manner.

[0016] Furthermore, the process of local training from edge nodes includes:

[0017] Obtaining a data set for training, performing preliminary data division on the data set, and sending the data set to a plurality of slave edge nodes;

[0018] Each of the slave edge nodes performs local training according to the divided data ratio, and generates corresponding local parameters and training states;

[0019] The training status includes the training time and data ratio of the local training, and the data represents the data generated in the industrial Internet of Things environment.

[0020] Furthermore, the computation time of the edge nodes in local training is calculated Communication time The expression is:

[0021]

[0022] Based on calculation time Communication time Get the total time of local training from edge nodes The expression is:

[0023]

[0024] Among them, i represents the i-th slave edge node, k represents the k-th local training, represents the training data divided from edge node i in the kth local training, represents the CPU utilization of edge node i in the kth local training, represents the number of CPU cycles executed on the training data samples from edge node i in the kth local training.

[0025] Furthermore, according to the total time of the kth local training from the edge node Generate training state s from edge nodes k , the expression is:

[0026] s k ={D k-y ,T k-y ,...D k-1 ,T k-1};

[0027] Where y is a natural number, indicating the yth round of local training.

[0028] In the above technical solution, by calculating the calculation time and communication time in the local training process of the edge node, the training status of the edge node is determined. The training status can reflect the time difference for completing data processing between the current edge nodes, so that the deep reinforcement learning node can better understand and learn, and feedback the training action corresponding to the next round of local training based on the training status.

[0029] Furthermore, the process of the deep reinforcement learning node feeding back the training action to the edge node according to the training state includes:

[0030] Based on the training state s from the edge node k , construct several training delay difference problems between edge nodes, the expression is:

[0031]

[0032] The deep reinforcement learning node solves the training delay problem and obtains all training actions a from the edge node for the next round of local training. k , according to the training action a k The expression for guiding the redivision of data ratio and allocating it to each slave edge node is:

[0033]

[0034] Where, ΔT i,i′ Represents the training delay difference between edge node i and edge node i′.

[0035] Furthermore, the process of performing communication iterative training of heterogeneous edge nodes in coordination according to global parameters and training actions from the edge node includes:

[0036] The heterogeneous edge device collaboration model also includes an experience revisit buffer;

[0037] The deep reinforcement learning node uses the ppo-clip algorithm and the actor-critic network, and sets constraints The training delay problem is solved iteratively to obtain the shortest time T required for all rounds of training. The expression is:

[0038]

[0039] Each training action a obtained from the edge node according to each round of iteration k Guide the re-divided data ratio to perform iterative training, and store the result of each round of iterative training as experience data in the experience revisit buffer;

[0040] Wherein, N represents a natural number, and the actor-critic network represents an actor network.

[0041] Furthermore, in the process of iteratively solving the training delay problem, an activation function softmax is set on the actor-critic network, and the sum of the output results of each iterative solution is set to 1.

[0042] Furthermore, in the process of iteratively solving the training delay problem, the reward function r is setk The result of each iteration is evaluated and the expression is:

[0043] r k =-max i,i′∈N ΔT i,i′ ;

[0044] When the reward function r k When the output value of tends to be stable, the shortest time T required for all rounds of training is obtained, the communication iteration training is completed, and the deep reinforcement learning node takes the final training action a k The master edge node sends the final global parameters to the slave edge node, and the slave edge node calculates the final global parameters based on the global parameters and training action a. k Collaborate with heterogeneous edge nodes.

[0045] Furthermore, when the experience data fills up the experience revisit buffer, several rounds of experience data are randomly extracted from the experience revisit buffer to perform optimization training on the actor-critic network of the deep reinforcement learning node, and the experience data in the experience revisit buffer is cleared.

[0046] In the above technical solution, the training action a in the iterative training process is solved by constructing the training delay problem. k , so that the edge node can follow the training action a k Heterogeneous edge nodes are coordinated to reduce the time difference of completing data processing between edge nodes, improve the accuracy and convergence speed of node training, solve the problem of node falling behind during node training, and make industrial production proceed in an orderly and rapid manner; in addition, the adopted PPO-Clip algorithm and actor-critic network can effectively learn the heterogeneity and dynamic changes of edge nodes, and the activation function softmax set can set the sum of the output results of each iterative solution to 1, so as to better observe the effect of iterative training from the data; the reward function r set k The training state with the highest training efficiency in the iterative training process can be screened out and sent to the deep reinforcement learning node, so that the deep reinforcement learning node can feed back the corresponding training action to the slave edge node, so as to reduce the time difference in completing data processing between the slave edge nodes, improve the accuracy and convergence speed of node training, and solve the problem of node falling behind during node training; the set experience revisit buffer can randomly extract several rounds of experience data from the experience revisit buffer when the experience data fills the experience revisit buffer to optimize the actor-critic network of the deep reinforcement learning node, so that the deep reinforcement learning node can better learn the heterogeneity and dynamic changes of the slave edge nodes, thereby improving the performance of the deep reinforcement learning node.

[0047] A heterogeneous edge device collaboration system based on industrial Internet of Things, the system comprising:

[0048] A construction module, used to construct a heterogeneous edge device collaboration model, wherein the heterogeneous edge device collaboration model includes a slave edge node, a master edge node, and a deep reinforcement learning node;

[0049] A training module, used to enable local training from edge nodes;

[0050] A processing module for generating local parameters and training states;

[0051] The processing module is also used to enable the master edge node to generate global parameters according to the local parameters;

[0052] A feedback module is used to enable the deep reinforcement learning node to feed back training actions to the edge nodes according to the training status;

[0053] The training module is also used to enable the slave edge nodes to perform communication iterative training of heterogeneous edge node collaboration according to global parameters and training actions;

[0054] The collaboration module is used to complete the collaboration of heterogeneous edge nodes based on the communication iteration training results.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The present invention proposes a heterogeneous edge device collaboration method and system based on the industrial Internet of Things. By performing local training on the slave edge nodes in the heterogeneous edge device collaboration model, local parameters and training states of each slave edge node in the heterogeneous edge device collaboration model can be generated, so that the master edge node and the deep reinforcement learning node generate and feedback corresponding global parameters and training actions, and then the slave edge nodes perform communication iterative training for heterogeneous edge node collaboration according to the global parameters and training actions to complete the heterogeneous edge node collaboration; the set deep reinforcement learning nodes can effectively learn the heterogeneity and dynamic changes of the slave edge nodes, thereby reducing the time difference in completing data processing between the slave edge nodes, improving the accuracy and convergence speed of node training, solving the problem of node falling behind during node training, and enabling industrial production to proceed in an orderly and rapid manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flowchart of a heterogeneous edge device collaboration method based on the industrial Internet of Things provided in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of the structure of a heterogeneous edge device collaboration model provided in an embodiment of the present application;

[0059] Figure 3A schematic diagram of the communication structure between a slave edge device and a master edge device provided in an embodiment of the present application;

[0060] Figure 4 A diagram showing the data division ratio of each node during the experiment provided in the embodiment of the present application;

[0061] Figure 5 A comparison chart of the time it takes for each node to complete training during the experiment provided in the embodiment of the present application;

[0062] Figure 6 A structural block diagram of a heterogeneous edge device collaboration system based on the Industrial Internet of Things provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0064] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0066] Embodiment 1:

[0067] This embodiment provides a heterogeneous edge device collaboration method based on the industrial Internet of Things. Figure 1 and Figure 2 , the method comprises the following steps:

[0068] S1: Construct a heterogeneous edge device collaboration model, wherein the heterogeneous edge device collaboration model includes a slave edge node, a master edge node, and a deep reinforcement learning node;

[0069] S2: Perform local training from edge nodes to generate local parameters and training status;

[0070] S3: The master edge node generates global parameters based on local parameters;

[0071] S4: The deep reinforcement learning node feeds back training actions to the edge nodes based on the training status;

[0072] S5: The edge nodes perform communication iterative training for heterogeneous edge node collaboration according to global parameters and training actions, and complete the heterogeneous edge node collaboration according to the communication iterative training results.

[0073] In step S2, the process of performing local training from the edge node includes:

[0074] Obtaining a data set for training, performing preliminary data division on the data set, and sending the data set to a plurality of slave edge nodes;

[0075] Each of the slave edge nodes performs local training according to the divided data ratio, and generates corresponding local parameters and training states;

[0076] The training status includes the training time and data ratio of the local training, and the data represents the data generated in the industrial Internet of Things environment.

[0077] Furthermore, the computation time of the edge nodes in local training is calculated Communication time The expression is:

[0078]

[0079] Based on calculation time Communication time Get the total time of local training from edge nodes The expression is:

[0080]

[0081] Among them, i represents the i-th slave edge node, k represents the k-th local training, represents the training data divided from edge node i in the kth local training, represents the CPU utilization of edge node i in the kth local training, represents the number of CPU cycles executed on the training data samples from edge node i in the kth local training.

[0082] Furthermore, according to the total time of the kth local training from the edge node Generate training state s from edge nodes k , the expression is:

[0083] s k ={Dk-y ,T k-y ,...D k-1 ,T k-1};

[0084] Where y is a natural number, indicating the yth round of local training.

[0085] It can be understood that by calculating the calculation time and communication time during the local training process of the edge node, the training status of the edge node can be determined. The training status can reflect the time difference in completing data processing between the current edge nodes, so that the deep reinforcement learning node can better understand and learn, and feedback the training action corresponding to the next round of local training based on the training status.

[0086] For example, see Figure 2 and Figure 3 After each slave edge node completes the local training of this round, it generates local parameters and sends them to the master edge node. After receiving all the local parameters, the master edge node aggregates the parameters to obtain new global parameters. The new global parameters are sent to each slave edge node participating in the training, and the slave edge node uses the new global parameters for local training. Set the collection exist There are N slave edge nodes in the set. Each slave edge node i in the kth round of training will be allocated a data volume suitable for its computing performance and other factors The latency is mainly composed of two parts: one is the computation time of local training from the edge node, and the other is the communication time for parameter transfer between the master edge node and the slave edge node. represents the CPU utilization of edge node i in the kth communication iteration, is the number of CPU cycles executed on the local data samples from edge node i in the kth communication iteration. So the computation time of local training from edge node i in the kth communication iteration is recorded as It is expressed as:

[0087] The communication time to transfer the trained local parameters to the master edge node for global parameter aggregation. The communication time depends on the data transmission rate and parameter size of the local parameters from the edge node. The data transmission rate can be determined by the Shannon formula, and the communication time from edge node i in the kth communication iteration is expressed as: Among them, B represents the average network bandwidth when uploading the model in this communication round, ρ i is the transmission power, h i is the channel gain between the slave edge node i and the master edge node, and N0 represents the background noise.

[0088] In one round of training, the training of a single slave edge node includes both computation time and communication time, so the total time for local training of slave edge node i in the kth round of communication is:

[0089] Considering the problem of falling behind between nodes, we need to consider the difference in time between nodes to complete training, so we use ΔT i,i′ To represent the training delay difference between edge nodes i and i′:

[0090] consider and is the time from the edge node i to start training and end training in the kth round of communication, so the total time of local training from edge node i in the kth round of communication can also be expressed as: In order to dynamically adjust the allocation of training data to slave edge nodes in the next round of communication, the historical training information of these nodes is used. During the learning process of the deep reinforcement learning node, the heterogeneous edge device collaborative model records the training samples of all slave edge nodes in the previous round of communication k. Training time Considering that the state of the collaborative model of heterogeneous edge devices changes over time, in order to reduce the size of the state space, only the state of the most recent y rounds of communication is considered, and the training state space is expressed as: k ={d k-y ,T k -y ,...D k-1 ,T k-1}.

[0091] In step S4, the process of the deep reinforcement learning node feeding back the training action to the edge node according to the training state includes:

[0092] Based on the training state s from the edge node k , construct several training delay difference problems between edge nodes, the expression is:

[0093]

[0094] The deep reinforcement learning node solves the training delay problem and obtains all training actions a from the edge node for the next round of local training. k , according to the training action a k The expression for guiding the redivision of data ratio and allocating it to each slave edge node is:

[0095]

[0096] Where, ΔT i,i′Represents the training delay difference between edge node i and edge node i′.

[0097] In step S5, the process of performing communication iterative training of heterogeneous edge nodes in collaboration from edge nodes according to global parameters and training actions includes:

[0098] The heterogeneous edge device collaboration model also includes an experience revisit buffer;

[0099] The deep reinforcement learning node uses the ppo-clip algorithm and the actor-critic network, and sets constraints The training delay problem is solved iteratively to obtain the shortest time T required for all rounds of training. The expression is:

[0100]

[0101] Each training action a obtained from the edge node according to each round of iteration k Guide the re-divided data ratio to perform iterative training, and store the result of each round of iterative training as experience data in the experience revisit buffer;

[0102] Wherein, N represents a natural number, and the actor-critic network represents an actor network.

[0103] As a preferred embodiment, during the iterative solution to the training delay problem, an activation function softmax is set on the actor-critic network, and the sum of the output results of each iterative solution is set to 1.

[0104] As a preferred embodiment, in the process of iteratively solving the training delay problem, the reward function r is set k The result of each iteration is evaluated and the expression is:

[0105] r k =-max i,i′∈N ΔT i,i′ ;

[0106] When the reward function r k When the output value of tends to be stable, the shortest time T required for all rounds of training is obtained, the communication iteration training is completed, and the deep reinforcement learning node takes the final training action a k The master edge node sends the final global parameters to the slave edge node, and the slave edge node calculates the final global parameters based on the global parameters and training action a. k Collaborate with heterogeneous edge nodes.

[0107] As a preferred embodiment, when the experience data fills the experience revisit buffer, several rounds of experience data are randomly extracted from the experience revisit buffer to optimize the actor-critic network of the deep reinforcement learning node, and the experience data in the experience revisit buffer is cleared.

[0108] Specifically, in the kth round of training, the deep reinforcement learning node collects the training status (i.e., the amount of data and training time of each slave edge node), and the deep reinforcement learning node feeds back the training status information to the actor-critic network, which then outputs the training action. This training action involves allocating the size of the local training data to each slave edge node before the next round of training. After the deep reinforcement learning node outputs the training action, each slave edge node obtains the corresponding amount of local data, collects the latest global parameters from the master edge node for local training, and records the latest training capacity and training time information. After the slave edge node completes the local training, it sends the updated local parameters to the master edge node. After receiving the updated local parameters, the training time of the slave edge node in this round can be calculated; then, the output of the reward function is sent to the deep reinforcement learning node as feedback to evaluate the action of the previous round. During the training process, the deep reinforcement learning node constantly interacts with the environment, and the tuple k ,a k ,r k ,s k+1 > is constantly recorded in the experience replay buffer. When the buffer is full, small batch random sampling is used to effectively utilize this information to train the actor-critic network. By adding a reward function feedback mechanism, the deep reinforcement learning node can adapt to changes in the environment, use historical data in the experience replay buffer to continuously optimize the decision-making strategy, and improve the performance of the deep reinforcement learning node in unknown environments.

[0109] It is understandable that by constructing the training delay problem, the training action a in the iterative training process is solved. k , so that the edge node can follow the training action a k Heterogeneous edge nodes are coordinated to reduce the time difference of completing data processing between edge nodes, improve the accuracy and convergence speed of node training, solve the problem of node falling behind during node training, and make industrial production proceed in an orderly and rapid manner; in addition, the adopted PPO-Clip algorithm and actor-critic network can effectively learn the heterogeneity and dynamic changes of edge nodes, and the activation function softmax set can set the sum of the output results of each iterative solution to 1, so as to better observe the effect of iterative training from the data; the reward function r set k ​The training state with the highest training efficiency in the iterative training process can be screened out and sent to the deep reinforcement learning node, so that the deep reinforcement learning node can feed back the corresponding training action to the slave edge node, so as to reduce the time difference in completing data processing between the slave edge nodes, improve the accuracy and convergence speed of node training, and solve the problem of node falling behind during node training; the set experience revisit buffer can randomly extract several rounds of experience data from the experience revisit buffer when the experience data fills the experience revisit buffer to optimize the actor-critic network of the deep reinforcement learning node, so that the deep reinforcement learning node can better learn the heterogeneity and dynamic changes of the slave edge nodes, thereby improving the performance of the deep reinforcement learning node.

[0110] In this embodiment, by performing local training on the slave edge nodes in the heterogeneous edge device collaboration model, the local parameters and training status of each slave edge node in the heterogeneous edge device collaboration model can be generated, so that the master edge node and the deep reinforcement learning node generate and feedback corresponding global parameters and training actions, and then the slave edge nodes perform communication iterative training for heterogeneous edge node collaboration according to the global parameters and training actions to complete the heterogeneous edge node collaboration; the set deep reinforcement learning nodes can effectively learn the heterogeneity and dynamic changes of the slave edge nodes, thereby reducing the time difference in completing data processing between the slave edge nodes, improving the accuracy and convergence speed of node training, solving the problem of node falling behind during node training, and enabling industrial production to proceed in an orderly and rapid manner.

[0111] Embodiment 2:

[0112] This embodiment provides corresponding experimental data for the content in the first embodiment, which are as follows:

[0113] See Figure Figure 4 , shows the proportion of the training data distributed by the four edge nodes in each round of training to the entire data set. It can be found that at the beginning of training, the data set is evenly distributed to all edge nodes. In the first 40 rounds, the data division ratio of each edge node varies greatly, and then stabilizes. The data ratio allocated to node 1 is stable at around 0.55, and the data ratios allocated to nodes 2, 3, and 4 are stable at around 0.14, 0.26, and 0.04. Correspondingly, see Figure 5 At the beginning, due to the different computing power of each slave edge node and the same amount of training data, the training time of the nodes varies greatly. After about 40 rounds, the training time of each node tends to be consistent, fluctuating around 24 seconds. To verify the rationality of the data partitioning strategy and the consistency of the training time, the proposed method balances the training time by reasonably allocating data in the nodes with heterogeneous computing performance, thereby improving the accuracy and convergence speed of node training, solving the problem of node falling behind during node training, and enabling the orderly and rapid progress of industrial production.

[0114] Embodiment three:

[0115] This embodiment provides a heterogeneous edge device collaboration system based on the industrial Internet of Things. Figure 6 , the system comprising:

[0116] A construction module, used to construct a heterogeneous edge device collaboration model, wherein the heterogeneous edge device collaboration model includes a slave edge node, a master edge node, and a deep reinforcement learning node;

[0117] A training module, used to enable local training from edge nodes;

[0118] A processing module for generating local parameters and training states;

[0119] The processing module is also used to enable the master edge node to generate global parameters according to the local parameters;

[0120] A feedback module is used to enable the deep reinforcement learning node to feed back training actions to the edge nodes according to the training status;

[0121] The training module is also used to enable the slave edge nodes to perform communication iterative training of heterogeneous edge node collaboration according to global parameters and training actions;

[0122] The collaboration module is used to complete the collaboration of heterogeneous edge nodes based on the communication iteration training results.

[0123] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A heterogeneous edge device collaboration method based on industrial Internet of Things, characterized in that: The method comprises the following steps: Constructing a heterogeneous edge device collaboration model, wherein the heterogeneous edge device collaboration model includes a slave edge node, a master edge node, and a deep reinforcement learning node; Perform local training from edge nodes to generate local parameters and training status; The master edge node generates global parameters based on local parameters; The deep reinforcement learning node feeds back training actions to the edge nodes based on the training status; The slave edge node performs communication iterative training for heterogeneous edge node collaboration according to global parameters and training actions, and completes heterogeneous edge node collaboration according to the communication iterative training results; The process of local training from edge nodes includes: Obtaining a data set for training, performing preliminary data division on the data set, and sending the data set to a plurality of slave edge nodes; Each of the slave edge nodes performs local training according to the divided data ratio, and generates corresponding local parameters and training states; Wherein, the training status includes the training time and data ratio of the local training, and the data represents the data generated in the industrial Internet of Things environment; Calculate the computation time of the edge nodes in local training Communication time , the expression is: ； ； Based on calculation time Communication time , get the total time of local training from edge nodes , the expression is: ； Among them, i represents the i-th slave edge node, k represents the k-th local training, Indicated in From edge nodes in the sub-local training The divided training data, Indicated in From edge nodes in the sub-local training CPU utilization, Indicates that from the edge node In the The number of CPU cycles executed on the training data samples in this local training; According to the edge node Total time of local training , generate the training state from the edge node , the expression is: ； Where y is a natural number, indicating the yth round of local training; The process of deep reinforcement learning nodes feeding back training actions to edge nodes based on training status includes: Based on the training status of the edge nodes , construct several training delay difference problems between edge nodes, the expression is: s.t. The deep reinforcement learning node solves the training delay problem and obtains all training actions for the next round of local training from the edge node. , according to the training action The expression for guiding the redivision of data ratio and allocating it to each slave edge node is: ； in, Represents the difference between edge node i and edge node The training delay gap between them.

2. The method for coordinating heterogeneous edge devices based on industrial Internet of Things according to claim 1 is characterized in that: The process of performing communication iterative training of heterogeneous edge nodes in coordination according to global parameters and training actions from edge nodes includes: The heterogeneous edge device collaboration model also includes an experience revisit buffer; The deep reinforcement learning node uses the ppo-clip algorithm and the actor-critic network, and sets constraints , iteratively solve the training delay problem and obtain the shortest time T required for all rounds of training. The expression is: ； Each training action obtained from the edge node according to each round of iteration Guide the re-divided data ratio to perform iterative training, and store the result of each round of iterative training as experience data in the experience revisit buffer; Wherein, N represents a natural number, and the actor-critic network represents an actor network.

3. The method for coordinating heterogeneous edge devices based on industrial Internet of Things according to claim 2 is characterized in that: During the iterative solution to the training delay problem, an activation function softmax is set on the actor-critic network, and the sum of the output results of each iterative solution is set to 1.

4. The method for coordinating heterogeneous edge devices based on industrial Internet of Things according to claim 2 is characterized in that: In the process of iteratively solving the training delay problem, set the reward function The result of each iteration is evaluated and the expression is: ； When the reward function When the output value of tends to be stable, the shortest time T required for all rounds of training is obtained, the communication iteration training is completed, and the deep reinforcement learning node takes the final training action The master edge node sends the final global parameters to the slave edge node, and the slave edge node calculates the final global parameters based on the global parameters and training actions. Collaborate with heterogeneous edge nodes.

5. The method for coordinating heterogeneous edge devices based on industrial Internet of Things according to claim 2 is characterized in that: When the experience data fills up the experience revisit buffer, several rounds of experience data are randomly extracted from the experience revisit buffer to optimize the actor-critic network of the deep reinforcement learning node, and the experience data in the experience revisit buffer is cleared.

6. A heterogeneous edge device collaboration system based on industrial Internet of Things, the system is used to implement the method according to any one of claims 1 to 5, characterized in that: The system comprises: A construction module, used to construct a heterogeneous edge device collaboration model, wherein the heterogeneous edge device collaboration model includes a slave edge node, a master edge node, and a deep reinforcement learning node; A training module, used to enable local training from edge nodes; A processing module for generating local parameters and training states; The processing module is also used to enable the master edge node to generate global parameters according to the local parameters; A feedback module is used to enable the deep reinforcement learning node to feed back training actions to the edge nodes according to the training status; The training module is also used to enable the slave edge nodes to perform communication iterative training of heterogeneous edge node collaboration according to global parameters and training actions; The collaboration module is used to complete the collaboration of heterogeneous edge nodes based on the communication iteration training results.

Images