Federal learning method and system applied to Internet of Things

By selecting the longest-existing nodes from the collection of IoT nodes for parameter aggregation and training, the parameter deviation problem in federated learning is solved and the model accuracy is improved.

CN118917434AInactive Publication Date: 2024-11-08GUANGXI COMM GUIHUA DESIGN CONSULTATION CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410930153.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the field of IoT, federated learning reduces the accuracy of the global model due to parameter deviations caused by different node computing performance.

Method used

By selecting the longest-existing node in the collection of IoT nodes with similar associations as the first IoT node, it is responsible for aggregating the parameters of other nodes and using the global model to train them until the model converges, ensuring the training parameters reliability and stability.

Benefits of technology

Improve the reliability and stability of training parameters in the same set and improve the accuracy of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917434A_ABST
    Figure CN118917434A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a federal learning method and system applied to the Internet of Things, and the method comprises the steps: taking a second Internet of Things node with the longest existence time as a first Internet of Things node in a set of a plurality of second Internet of Things nodes with a similar incidence relation; the first Internet of Things node is responsible for aggregating parameters obtained by training all the second Internet of Things nodes, the second Internet of Things nodes perform local model training based on global parameters, and the server trains the first parameters to obtain global parameters of the next round and issues the global parameters until the model converges. According to the method, the nodes in different sets are divided to have a certain similarity relationship, so that the training parameters in the sets belong to the same distribution, the reliability and stability of the training parameters in the same set are ensured, then the Internet of Things nodes with the longest existence time are subjected to parameter aggregation, the reliability and stability of the first parameter are ensured, and the reliability and stability of the second parameter are ensured. And finally, the model precision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of Internet of Things, and in particular, to a federated learning method and system applied to the Internet of Things. Background Art

[0002] The Internet of Things (IoT) originated in the field of media and is the third revolution in the information technology industry. The Internet of Things refers to connecting any object to the network through information sensor nodes according to agreed protocols, and objects exchange and communicate information through information transmission media to achieve intelligent identification, positioning, tracking, supervision and other functions.

[0003] As a distributed deep learning paradigm, federated learning has broad application prospects in the field of Internet of Things. Federated learning enables data stored on IoT devices to be jointly trained without leaving the local area, thereby effectively solving the "data island" and "data privacy protection" problems caused by geographical isolation and data security of IoT devices. However, due to the characteristics of distributed training in federated learning, different IoT nodes have different computing performances, and there are certain deviations between the parameters generated after training of different IoT nodes, which will reduce the accuracy of the global model. Therefore, how to reduce the deviation between the parameters generated after training of different IoT nodes is a problem that needs to be solved urgently. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The main purpose of the embodiments of the present disclosure is to propose a federated learning method and system applied to the Internet of Things, which can ensure the reliability and stability of training parameters and first parameters in the same set, and ultimately improve the accuracy of the model.

[0006] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present disclosure proposes a federated learning method applied to the Internet of Things, which is applied to a server. The method includes:

[0007] Receive first parameters of multiple first IoT nodes, where the first parameters are obtained by aggregating second parameters of all second IoT nodes in a set in which the first IoT node is located and the first IoT node, and the second parameters are obtained by training the global parameters of a first training number of times by the first IoT node and the second IoT node using a local model, the first IoT node is the one with the longest existence time among all second IoT nodes in the set, and the set includes multiple second IoT nodes including the first IoT node and having a similarity relationship with each other;

[0008] Using a global model to train the first parameter to obtain a global parameter for a second training number of times, where the second training number of times is a next round of training number of times of the first training number of times;

[0009] The global parameters of the second training times are sent to each second IoT node, so that each second IoT node performs the next round of training until the local model of the second IoT node converges.

[0010] An embodiment of the present application provides a federated learning method applied to the Internet of Things. In view of the characteristics of different performances of Internet of Things nodes, the method proposes a federated learning method for collaborative processing of a first Internet of Things node, a second Internet of Things node and a server. The method finds a second Internet of Things node with the longest existence time as the first Internet of Things node among a set of multiple second Internet of Things nodes with similar associations. Multiple first Internet of Things nodes in the Internet of Things are responsible for parameter aggregation obtained by training all second Internet of Things nodes in their respective sets. The second Internet of Things nodes in the set are used to perform parameter training of the local model according to the global parameters sent by the server. The global model of the server trains the first parameters to obtain the next round of global parameters and sends them down until the model converges. The method divides the nodes of the Internet of Things into sets so that the nodes in different sets have a certain similarity relationship, and then the training parameters in the set belong to the same distribution, which can ensure the reliability and stability of the training parameters in the same set. Then, the Internet of Things node with the longest existence time aggregates the parameters to ensure the reliability and stability of the first parameters, and finally improves the accuracy of the model.

[0011] In some implementations of the present application, the set under the first training times is selected in the following manner:

[0012] Determine a target set for a third training number, where the third training number is the previous round of training number of the first training number, and the target set is any set;

[0013] Selecting the IoT node with the longest existence time in the target set as the target IoT node of the target set for the first training number;

[0014] Determine in the Internet of Things the remaining Internet of Things nodes whose distances to the target Internet of Things node are less than a set first threshold and whose similarities to the second parameter of the target Internet of Things node are greater than a set second threshold;

[0015] The target Internet of Things node is used as the first Internet of Things node of the target set, and the remaining Internet of Things nodes are used as the second Internet of Things nodes of the target set.

[0016] In some implementations of the present application, after selecting the IoT node with the longest existence time in the target set as the target IoT node of the target set for the first number of training times, the method further includes:

[0017] Determine a first position of the target Internet of Things node, and obtain a second position of an Internet of Things node other than the target Internet of Things node in the Internet of Things;

[0018] calculating a distance between the first position and the second position;

[0019] Calculate the cosine similarity between the target IoT node and IoT nodes in the IoT except the target IoT node.

[0020] In some embodiments of the present application, the method further comprises:

[0021] receiving a first parameter sent by a third Internet of Things node, wherein the third Internet of Things node is a second Internet of Things node whose existence time is second only to that of the first Internet of Things node and is selected from a set of failed first Internet of Things nodes as a replacement for the failed first Internet of Things node when the first Internet of Things node fails;

[0022] The first parameter is trained by using a global model to obtain the global parameter of a second training number of times, and then the first parameter is sent to the second Internet of Things node.

[0023] In some implementations of the present application, the first IoT node uses synchronous aggregation to perform federated learning.

[0024] In some implementations of the present application, the second Internet of Things node includes a fixed Internet of Things node and a mobile Internet of Things node.

[0025] In some implementations of the present application, the first parameter is a parameter obtained by weighted average aggregation.

[0026] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present disclosure proposes a federated learning device applied to the Internet of Things, the device comprising:

[0027] An information receiving unit, configured to receive first parameters of a plurality of first IoT nodes, wherein the first parameters are obtained by aggregating the second parameters of all second IoT nodes in a set in which the first IoT node is located and the first IoT node, the second parameters are obtained by training the first IoT node and the second IoT node with a global parameter of a first training number of times using a local model, the first IoT node is the one with the longest existence time among all second IoT nodes in the set, and the set includes a plurality of second IoT nodes including the first IoT node and having a similarity relationship with each other;

[0028] A parameter training unit, configured to train the first parameter using a global model to obtain the global parameter for a second training number of times, where the second training number of times is the next round of training number of times of the first training number of times;

[0029] A parameter sending unit is used to send the global parameters of the second training times to each second Internet of Things node, so that each second Internet of Things node performs the next round of training until the local model of the second Internet of Things node converges.

[0030] To achieve the above-mentioned purpose, a third aspect of the embodiments of the present disclosure provides an electronic node, comprising at least one memory;

[0031] at least one processor;

[0032] at least one computer program;

[0033] The computer program is stored in the memory, and the processor executes the at least one computer program to implement:

[0034] A federated learning method applied to the Internet of Things as described in any one of the embodiments of the first aspect.

[0035] To achieve the above objective, a fourth aspect of the embodiments of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute:

[0036] A federated learning method applied to the Internet of Things as described in any one of the embodiments of the first aspect.

[0037] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 It is a flowchart of a federated learning method applied to the Internet of Things provided by an embodiment of the present application;

[0040] Figure 2 This is a schematic diagram of a set partitioning process provided by an embodiment of the present application;

[0041] Figure 3 This is a schematic diagram of the architecture of a federated learning system applied to the Internet of Things provided by an embodiment of the present application;

[0042] Figure 4 This is an architecture diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0045] The Internet of Things (IoT) originated in the field of media and is the third revolution in the information technology industry. The Internet of Things refers to connecting any object to the network through information sensor nodes according to agreed protocols, and objects exchange and communicate information through information transmission media to achieve intelligent identification, positioning, tracking, supervision and other functions.

[0046] Federated learning, as a distributed deep learning paradigm, allows nodes to use local data to train their local deep learning models individually and aggregate them into a global model. Nodes do not send local data directly, but only share local model parameters, which protects node privacy to a certain extent. This flexible learning method is suitable for the Internet of Things.

[0047] At present, since different IoT nodes have different computing performance, there are certain deviations between the parameters generated after training of different IoT nodes, which will reduce the accuracy of the global model.

[0048] Reference Figure 1 An embodiment of the present application provides a federated learning method applied to the Internet of Things, which is applied to a server. The method includes the following steps S110 to S130:

[0049] Step S110, receiving first parameters of multiple first Internet of Things nodes, where the first parameter is a parameter obtained by aggregating second parameters of all second Internet of Things nodes in the set to which the first Internet of Things node belongs, and the second parameter is a parameter obtained by training the global parameter of the first training times by the second Internet of Things node using a local model, the first Internet of Things node is the one with the longest existence time among all the second Internet of Things nodes in the set, and the set includes multiple second Internet of Things nodes including the first Internet of Things node and having a similarity relationship with each other.

[0050] Step S120: Use the global model to train the first parameter to obtain the global parameter of the second training times, where the second training times is the next round of training times of the first training times.

[0051] Step S130: Send the global parameters of the second training times to each second IoT node, so that each second IoT node performs the next round of training until the local model of the second IoT node converges.

[0052] In this embodiment, the server refers to a server used to train the global model, which may be a parameter server, an edge server, or a cloud server.

[0053] In this embodiment, the second IoT node refers to a node of the IoT, for example, various smart terminals, including but not limited to: mobile phones, smart glasses, watches, cameras, monitoring nodes, sensor nodes, etc. The first IoT node is an IoT node selected from a set consisting of multiple second IoT nodes.

[0054] It should be noted that any node (such as a smart terminal) in the Internet of Things that participates in federated learning and has certain computing capabilities (data collection capabilities, model training capabilities, and data transmission capabilities) can be used as a second Internet of Things node. This application selects a second Internet of Things node with higher stability from multiple second Internet of Things nodes as the first Internet of Things node, which can be seen in the introduction of subsequent embodiments for details.

[0055] In this embodiment, the multiple second IoT nodes in the IoT have different computing performances, so there will be deviations between the parameters obtained by training the local model of each second IoT node. Here, the multiple second IoT nodes are divided into multiple sets, at least one node in one set is a first IoT node, and the remaining nodes are second IoT nodes. There is a similarity relationship between the multiple second IoT nodes in each set to ensure that the deviations between the parameters trained by the multiple second IoT nodes in the set are relatively small.

[0056] In this embodiment, both the second IoT node and the first IoT node have a local model, and the first IoT node can also participate in the calculation of the local model.

[0057] For example, the second IoT node receives the global model parameter ω k-1 , perform the kth iteration, train the local model F, and update the second parameter ω k :

[0058] ω k =ω k-1 -ηF(ω k-1 )

[0059] Where η is the learning rate.

[0060] The first IoT node is a parameter that aggregates each second IoT node. In some implementations, the first parameter is a parameter obtained by weighted average aggregation. For example, if there are n second IoT nodes in the set m, then the aggregation formula is:

[0061]

[0062] in, is the first parameter, D i is the weight coefficient of the i-th second IoT node, is the second parameter of the i-th second IoT node, where the weight coefficient can be set based on experience and will not be described in detail here.

[0063] The server is used to receive the first parameter sent by the first IoT node of each set, and then train the global model through the first parameter to obtain the next round of global parameters ω k+1 .

[0064] In this embodiment, for each set, a node with the longest existence time is selected as the first IoT node as the first parameter. The first IoT node selected in this way is the most stable node and can improve the stability of global model training. It should be noted that the node with the longest existence time is the node with the longest cumulative time participating in the task in the IoT. The server will time the cumulative time of each second IoT node. The timing includes the timing of the current task and may also include the timing of historical tasks in the past. For example, there are 10 second IoT nodes participating in the training task S3, namely a1 to a 10 , change a1 to a 10 It is divided into three sets, namely set 1 to set 3, where set 1 includes a1 to a3, set 2 includes a4 to a6, and set 3 includes a7 to a 10 , where nodes a1, a4, and a7 have also participated in the training process of tasks S1 and S2, and node a3 has participated in the process of task S2. Obviously, the cumulative task duration of node a1 is is the cumulative duration of the task at node a3 It is also greater than the cumulative duration of the task at node a1 Right now , then set 1 selects node a1 with the longest cumulative task duration from nodes a1 to a3 as the first IoT node, set 2 selects node a4 with the longest cumulative task duration from nodes a4 to a6 as the first IoT node, and set 3 selects nodes a7 to a 10 Node a7, which has the longest cumulative task duration, is selected as the first IoT node.

[0065] In some embodiments, the second IoT node may be a fixed node, a sensor node, a monitor, etc., or a mobile node, such as a mobile phone, a smart watch, glasses, etc.

[0066] In this embodiment, the functions performed by each node are as follows:

[0067] (1) The server sends the global parameters of the current training times to the first IoT node in each set;

[0068] (2) the first IoT node sends the global parameter to multiple second IoT nodes in the set;

[0069] (3) The second IoT node updates the local model according to the global parameter to obtain a second parameter;

[0070] (4) All second IoT nodes in the set upload their respective second parameters to the first IoT node;

[0071] (5) The first IoT node aggregates the second parameters uploaded by each second IoT node to obtain the first parameter;

[0072] (6) The first IoT node sends the first parameter to the server;

[0073] (7) The server trains the first parameter to obtain the global parameter of the next round;

[0074] Repeat the above steps until the local model and the global model converge. It should be emphasized here that the convergence of the model requires convergence conditions in advance, such as the limit on the number of iterations or the minimization of time, cost, etc. There is no specific limitation here, and the types of local models and global models are not limited here.

[0075] In view of the characteristics of different performances of IoT nodes, this method proposes a federated learning method for collaborative processing of the first IoT node, the second IoT node and the server. This method finds the second IoT node with the longest existence time as the first IoT node in a set of multiple second IoT nodes with similar associations. Multiple first IoT nodes in the IoT are responsible for the aggregation of parameters obtained by training all second IoT nodes in their respective sets. The second IoT nodes in the set are used to train the parameters of the local model according to the global parameters sent by the server. The global model of the server trains the first parameters to obtain the next round of global parameters and sends them down until the model converges. This method divides the nodes of the IoT into sets so that there is a certain similarity relationship between the nodes in different sets, and then makes the training parameters in the set belong to the same distribution, which can improve the reliability and stability of the training parameters in the same set. Then, the IoT node with the longest existence time aggregates the parameters to ensure the reliability and stability of the first parameters, and finally improves the accuracy of the model.

[0076] Reference Figure 2 In some embodiments of the present application, the set under the first training times is selected in the following manner:

[0077] Step S210: determine a target set for a third training number of times, where the third training number of times is a previous round of training number of times of the first training number of times, and the target set is any set.

[0078] Step S220: Select the IoT node with the longest existence time in the target set as the target IoT node of the target set for the first training times.

[0079] Step S230, determine the first position of the target IoT node, obtain the second position of the IoT node other than the target IoT node in the IoT; calculate the distance between the first position and the second position; calculate the cosine similarity between the target IoT node and the IoT nodes other than the target IoT node in the IoT.

[0080] Step S240: Determine in the Internet of Things the remaining Internet of Things nodes whose distance to the target Internet of Things node is less than the set first threshold and whose similarity to the second parameter of the target Internet of Things node is greater than the set second threshold, and use the target Internet of Things node as the first Internet of Things node of the target set, and use the remaining Internet of Things nodes as the second Internet of Things nodes of the target set.

[0081] In this embodiment, the number and type of the plurality of second IoT nodes in the target set may change over time, and the reasons for the change may be:

[0082] (1) If the second IoT node is a mobile node, then as time goes by, the second IoT nodes in the same set may be farther away, resulting in an increase in communication power consumption. Therefore, it is necessary to reallocate each set of second IoT nodes.

[0083] (2) The second IoT node may perform multiple tasks at the same time. When performing multiple tasks, the computing performance of the second IoT node needs to be allocated. In this way, the computing performance for a certain task will be significantly reduced, which will cause deviations between the training parameters of the remaining second IoT nodes in its set. Therefore, it is necessary to reallocate the corresponding set of each second IoT node.

[0084] In this embodiment, firstly, a relatively most stable target IoT node is selected, that is, a second IoT node with the longest existence time in the target set is selected as the first IoT node in the target set. Then, the distances between all second IoT nodes and the first IoT node and the similarity of parameters are checked. The remaining IoT nodes in the IoT whose distances with the target IoT node are less than the set first threshold and whose similarity of the second parameter with the target IoT node is greater than the set second threshold are selected as the second IoT nodes in the target set. Finally, the target set is composed of the selected first IoT node and multiple second IoT nodes.

[0085] This embodiment considers that the second IoT node can be a mobile node. In each round of training, the target set is updated, that is, the constraints of the maximum time, distance and parameter similarity are added to ensure that the parameters between multiple second IoT nodes in the set in each round of training maintain the same distribution, that is, improve the reliability and stability of the training parameters in the same set. In addition, the first IoT node selected is the node with the longest existence time, so it is a relatively stable node. For example, the first IoT node is a fixed node with relatively strong computing performance and will participate in the calculation of each task. Such a configuration can ensure the reliability and stability of the first parameter and improve the accuracy of the model.

[0086] In some embodiments of the present application, the method further comprises:

[0087] Step S310: Receive a first parameter sent by a third IoT node, where the third IoT node is a second IoT node whose existence time is second only to that of the first IoT node, selected from a set of failed first IoT nodes as a replacement for the failed first IoT node when the first IoT node fails.

[0088] Step S320: Use the global model to train the first parameter, and when the global parameter of the second training times is obtained, send the first parameter to the second Internet of Things node.

[0089] This embodiment sets a safety redundancy setting for the first IoT node corresponding to each set, that is, when the first IoT node fails, the parameters of multiple second IoT nodes in the set cannot be aggregated, and the global model in the server at this time has no way to perform training updates. Therefore, a third IoT node whose existence time is second only to the failed first IoT node is selected as the new first IoT node, so as to replace the parameter aggregation operation of the failed first IoT node with the third IoT node, thereby ensuring the safe and normal execution of the training process. Moreover, since the selected third IoT node has a relatively long existence time, the reliability and stability of the first parameters can also be ensured.

[0090] In some embodiments, the first IoT node performs federated learning using synchronous aggregation.

[0091] Because the existing federated learning can be divided into synchronous and asynchronous methods according to the aggregation type, but in the asynchronous federated learning protocol, the server can be the first parameter, but the gradient divergence caused by it will further reduce the performance of the model. Therefore, the nodes selected in each round need to upload the trained model parameters to the server within the specified time slice. The server needs to wait until all selected nodes upload the parameters of the local model before starting aggregation.

[0092] The method provided in the embodiments of the present application can be applied to a variety of scenarios, such as industrial Internet of Things, Internet of Vehicles, smart furniture and power Internet of Things. The following two examples are provided:

[0093] Scenario 1: Applicable to power Internet of Things;

[0094] There are many kinds of power equipment in the power Internet of Things, including but not limited to: maintenance vehicles, servers, sensors, microprocessors, and power terminal equipment. Each device has a local model, and the training task to be performed is to detect abnormal operation data of the power Internet of Things. The conventional federated learning scheme is to train the local model with the parameters sent by each device through the server and upload the trained parameters to the server, so that the server aggregates the parameters and trains them using the global model. However, this method will cause deviations between the parameters calculated by each power device because different power devices have different computing performances, which will result in low training accuracy of the global model.

[0095] In order to solve this defect, different power equipment are divided into corresponding sets (including the initial division and the division in each round of training). During the initial division, the division can be carried out according to the distance. Several power equipment with similar distances are selected to divide into different sets according to the existing clustering algorithm.

[0096] During the training process from the kth round to the k+1th round, the following process is provided: (1) the server generates global parameters used for the kth round of training and sends them to each power device;

[0097] (2) Divide the set according to the training results of the k-1th round;

[0098] For any set, first determine the device that has existed the longest in the set in the k-1th round, and use this device as the first IoT node. Then determine the distance (which can be calculated based on the coordinate position) between all devices in the IoT and the first IoT node and the similarity between local parameters, and find multiple devices with a distance less than a preset first threshold and a similarity less than a second threshold as the second IoT node in the same set as the first IoT node.

[0099] (3) The second IoT node trains the global parameters used for the k-th round of training through the local model to obtain the second parameters used for the k-th round of training, and sends the corresponding second parameters to the first IoT node;

[0100] (4) The first IoT node aggregates the parameters of each second IoT node to obtain a first parameter;

[0101] (5) The first IoT node sends the parameters to the server, and the server uses the global model to train the global parameters used for the k+1th round of training.

[0102] Repeat the steps until the number of iterations is reached or the model converges, then stop iterating.

[0103] Finally, the trained model is used to determine whether there is abnormal operating data in the power Internet of Things, as well as some analysis of the abnormal operating data.

[0104] Scenario 2: Applicable to Internet of Vehicles;

[0105] Different from the power Internet of Things, the vehicle nodes in the Internet of Vehicles are all mobile Internet of Things nodes.

[0106] Assume that the task to be performed is route planning, with a roadside unit or edge server as the carrier of the global model. Each vehicle has an on-board CPU with a local model program set up in the CPU. Then, similar steps can be performed according to the above scenario, which will not be described in detail here.

[0107] Reference Figure 3 , an embodiment of the present application provides a federated learning system applied to the Internet of Things, the system comprising an information receiving unit 1100, a parameter training unit 1200 and a parameter sending unit 1300:

[0108] The information receiving unit is used to receive first parameters of multiple first Internet of Things nodes, where the first parameter is a parameter obtained by the first Internet of Things node aggregating the second parameters of all second Internet of Things nodes in the set to which it belongs, and the second parameter is a parameter obtained by the second Internet of Things node training the global parameter of the first training times using a local model. The first Internet of Things node is the one with the longest existence time among all the second Internet of Things nodes in the set, and the set includes multiple second Internet of Things nodes including the first Internet of Things node and have a similarity relationship with each other.

[0109] The parameter training unit is used to train the first parameter using a global model to obtain the global parameter of a second training number, where the second training number is the next round of training number of the first training number.

[0110] The parameter sending unit is used to send the global parameters of the second training times to each second Internet of Things node, so that each second Internet of Things node performs the next round of training until the local model of the second Internet of Things node converges.

[0111] It should be noted that this embodiment and the above method embodiment are based on the same inventive concept, so the relevant contents of the above method embodiment are also applicable to this embodiment and will not be described in detail here.

[0112] In view of the characteristics of different performances of IoT nodes, this embodiment proposes a federated learning method for collaborative processing of the first IoT node, the second IoT node and the server. This method finds the second IoT node with the longest existence time as the first IoT node among the sets of multiple second IoT nodes with similar associations. Multiple first IoT nodes in the IoT are responsible for the aggregation of parameters obtained by training all second IoT nodes in their respective sets. The second IoT nodes in the set are used to train the parameters of the local model according to the global parameters sent by the server. The global model of the server trains the first parameters to obtain the next round of global parameters and sends them down until the model converges. This method divides the nodes of the IoT into sets so that there is a certain similarity relationship between the nodes in different sets, and then makes the training parameters in the set belong to the same distribution, which can improve the reliability and stability of the training parameters in the same set. Then, the IoT node with the longest existence time aggregates the parameters to ensure the reliability and stability of the first parameters, and finally improves the accuracy of the model.

[0113] like Figure 4 The embodiment of the present application further provides an electronic node, which includes:

[0114] at least one memory;

[0115] at least one processor;

[0116] at least one program;

[0117] The program is stored in the memory, and the processor executes at least one program to implement the above-mentioned federated learning method applied to the Internet of Things implemented in the present disclosure.

[0118] The electronic node may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0119] The electronic node of the embodiment of the present application is introduced in detail below.

[0120] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0121] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage node, a dynamic storage node, or a random access memory (RAM). The memory 1700 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the program code is stored in the memory 1700, and the processor 1600 calls and executes the federated learning method applied to the Internet of Things in the embodiment of this application.

[0122] Input / output interface 1800, used to implement information input and output;

[0123] Communication interface 1900, used to realize communication interaction between this node and other nodes, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0124] Bus 2000 , which transmits information between various components of the node (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0125] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the node via the bus 2000 .

[0126] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned federated learning method applied to the Internet of Things.

[0127] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0128] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0129] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0130] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0131] Those skilled in the art will appreciate that all or some of the steps, systems, and functional modules / units in the nodes of the methods disclosed above may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0132] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or node comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or nodes.

[0133] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic node (which can be a personal computer, server, or network node, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store programs.

[0138] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A federated learning method applied to the Internet of Things, characterized in that: Applied to a server, the method comprises: Receive first parameters of multiple first IoT nodes, where the first parameters are obtained by aggregating second parameters of all second IoT nodes in a set in which the first IoT node is located and the first IoT node, and the second parameters are obtained by training the global parameters of a first training number of times by the first IoT node and the second IoT node using a local model, the first IoT node is the one with the longest existence time among all second IoT nodes in the set, and the set includes multiple second IoT nodes including the first IoT node and having a similarity relationship with each other; Using a global model to train the first parameter to obtain a global parameter for a second training number of times, where the second training number of times is a next round of training number of times of the first training number of times; The global parameters of the second training times are sent to each second IoT node, so that each second IoT node performs the next round of training until the local model of the second IoT node converges.

2. The federated learning method applied to the Internet of Things according to claim 1, characterized in that: The set under the first training times is selected in the following manner: Determine a target set for a third training number, where the third training number is the previous round of training number of the first training number, and the target set is any set; Selecting the IoT node with the longest existence time in the target set as the target IoT node of the target set for the first training number; Determine in the Internet of Things the remaining Internet of Things nodes whose distances to the target Internet of Things node are less than a set first threshold and whose similarities to the second parameter of the target Internet of Things node are greater than a set second threshold; The target Internet of Things node is used as the first Internet of Things node of the target set, and the remaining Internet of Things nodes are used as the second Internet of Things nodes of the target set.

3. The federated learning method applied to the Internet of Things according to claim 2, characterized in that: After selecting the IoT node with the longest existence time in the target set as the target IoT node of the target set for the first number of training times, the method further includes: Determine a first position of the target Internet of Things node, and obtain a second position of an Internet of Things node other than the target Internet of Things node in the Internet of Things; calculating a distance between the first position and the second position; Calculate the cosine similarity between the target IoT node and IoT nodes in the IoT except the target IoT node.

4. The federated learning method applied to the Internet of Things according to claim 1, characterized in that: The method further comprises: receiving a first parameter sent by a third Internet of Things node, wherein the third Internet of Things node is a second Internet of Things node whose existence time is second only to that of the first Internet of Things node and is selected from a set of failed first Internet of Things nodes as a replacement for the failed first Internet of Things node when the first Internet of Things node fails; The first parameter is trained by using a global model to obtain the global parameter of a second training number of times, and then the first parameter is sent to the second Internet of Things node.

5. The federated learning method applied to the Internet of Things according to claim 1, characterized in that: The first IoT node uses synchronous aggregation to perform federated learning.

6. The federated learning method applied to the Internet of Things according to claim 1, characterized in that: The second Internet of Things nodes include fixed Internet of Things nodes and mobile Internet of Things nodes.

7. The federated learning method applied to the Internet of Things according to claim 1, characterized in that: The first parameter is a parameter obtained by weighted average aggregation.

8. A federated learning system applied to the Internet of Things, characterized in that: The system comprises: An information receiving unit, configured to receive first parameters of a plurality of first IoT nodes, wherein the first parameters are obtained by aggregating the second parameters of all second IoT nodes in a set in which the first IoT node is located and the first IoT node, the second parameters are obtained by training the first IoT node and the second IoT node with a global parameter of a first training number of times using a local model, the first IoT node is the one with the longest existence time among all second IoT nodes in the set, and the set includes a plurality of second IoT nodes including the first IoT node and having a similarity relationship with each other; A parameter training unit, configured to train the first parameter using a global model to obtain the global parameter for a second training number of times, where the second training number of times is the next round of training number of times of the first training number of times; A parameter sending unit is used to send the global parameters of the second training times to each second Internet of Things node, so that each second Internet of Things node performs the next round of training until the local model of the second Internet of Things node converges.

9. An electronic node, characterized in that: include: at least one memory; at least one processor; at least one computer program; The computer program is stored in the memory, and the processor executes the at least one computer program to implement: A federated learning method applied to the Internet of Things as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute: A federated learning method applied to the Internet of Things as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Clustering federal multi-task learning method and device for Internet of Things

    CN115293358A

  • Federal learning method and device applied to Internet of Vehicles, equipment and medium

    CN117812564A