Model updating method and related equipment for Internet of Things system

Abstract

The embodiment of the present application proposes a model update method and related equipment for an Internet of Things system. The Internet of Things system includes multiple device nodes. The method includes: first, obtaining group feature perception information of each device node at the current moment; and obtaining the CPU utilization of each device node at the previous moment and the trusted device node sequence at the previous moment; then, inputting all group feature perception information, CPU utilization and the trusted device node sequence at the previous moment into a federal node selection model for node selection to obtain the trusted device node sequence at the current moment; finally, determining the target device node according to the trusted device node sequence at the current moment, obtaining the local model parameters of the data processing model deployed in the target device node, aggregating the local model parameters to obtain the global model parameters, and sending the global model parameters to each device node, thereby improving the reliability of node selection and improving the efficiency of improving model performance during model training.

Description

Technical Field

[0001] The present application relates to the technical field of industrial Internet of Things, and in particular to a model updating method and related equipment of an Internet of Things system. Background Art

[0002] As the amount and sources of data processed by the Industrial Internet of Things increase, traditional centralized learning solutions cannot meet the processing needs of multi-source and large-scale data. Based on this, federated learning, which places the learning process on edge devices for data learning processing, can effectively meet the processing needs of multi-source and large-scale data.

[0003] In related technologies, federated learning in IoT systems typically involves multiple device nodes downloading a global training model from a controller. After each training session, the controller retrieves node training parameters from the multiple device nodes, derives global training parameters based on the multiple node training parameters, and sends these global training parameters to all device nodes for a new round of model training. However, due to the risk of network attacks during model training, the resulting node training parameters can be affected, resulting in low efficiency in improving model performance during federated learning in IoT systems. Summary of the Invention

[0004] The embodiments of the present application provide a model updating method and related devices for an Internet of Things system, which can improve the reliability of model updating and the efficiency of improving model performance in the Internet of Things system.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a model updating method for an Internet of Things system, wherein the Internet of Things system includes a plurality of device nodes, and the method includes:

[0006] Acquire group characteristic perception information of each of the device nodes at a current moment, where the group characteristic perception information is used to characterize the activity level of the device node, the interaction frequency between the device node and the controller, and the communication quality between the device node and other device nodes;

[0007] Obtain the CPU utilization of each device node at the last moment and the sequence of trusted device nodes at the last moment;

[0008] Inputting all the group feature perception information, the CPU utilization rate, and the trusted device node sequence at the previous moment into a federated node selection model to perform node selection, thereby obtaining a trusted device node sequence at the current moment;

[0009] Determine the target device node based on the trusted device node sequence at the current moment, obtain local model parameters of the data processing model deployed in the target device node, aggregate the local model parameters to obtain global model parameters, and send the global model parameters to each of the device nodes so that the device nodes use the global model parameters to update the locally deployed data processing model.

[0010] In some embodiments, the group feature perception information includes node activity, node controller interaction, and communication space constraint information. Acquiring the group feature perception information of each device node at the current moment includes:

[0011] Obtaining the number of interactions between the device node and other device nodes at a current moment, and obtaining the node activity based on a ratio of the number of interactions to the number of device nodes;

[0012] Obtaining the interaction frequency between the device node and the controller at a current moment and the interaction duration of each interaction frequency, and obtaining the node-controller interaction degree based on the interaction frequency and the interaction duration;

[0013] The communication signal-to-noise ratio between the device node and the other device nodes at the current moment is obtained, as well as a minimum signal-to-noise ratio threshold, and the communication space constraint information is obtained based on the communication signal-to-noise ratio and the minimum signal-to-noise ratio threshold.

[0014] In some embodiments, the node controller interactivity includes an average interaction duration, an interaction degree, and a communication probability. Obtaining the node controller interactivity based on the interaction frequency and the interaction duration includes:

[0015] Accumulating all the interaction durations at the current moment to obtain the moment interaction time, and obtaining the average interaction duration based on the ratio of the moment interaction time to the interaction frequency;

[0016] Obtaining the interaction time and the interaction interval time between the device node and the controller, accumulating the interaction time and the interaction interval time to obtain the interaction cost time, and obtaining the interaction degree based on the ratio of the interaction time at the moment to the interaction cost time;

[0017] A decay interaction function is obtained based on an exponential decay function and the degree of interaction, and the communication probability is obtained based on a ratio of a preset communication quantity to the decay interaction function.

[0018] In some embodiments, obtaining the communication signal-to-noise ratio between the device node and the other device nodes at the current moment, and obtaining the minimum signal-to-noise ratio threshold, includes:

[0019] Obtaining a first node channel gain between the device node and a first other device node, obtaining a second node channel gain between the second other device node and the first other device node, obtaining a first transmit power of the device node, and obtaining a second transmit power of the second other device node;

[0020] Obtaining a first received power based on a product of the first node channel gain and the first transmit power, obtaining a second received power based on a product of the second node channel gain and the second transmit power, and obtaining the communication signal-to-noise ratio based on a ratio of the first received power to the second received power;

[0021] Acquire a channel bandwidth, select a minimum communication signal-to-noise ratio from a plurality of communication signal-to-noise ratios as a minimum signal-to-noise ratio, and obtain a minimum communication rate based on a logarithm of the minimum signal-to-noise ratio and multiplying it by the channel bandwidth;

[0022] The minimum signal-to-noise ratio threshold is obtained based on an index of the minimum communication rate of a preset communication quantity.

[0023] In some embodiments, obtaining the communication space constraint information based on the communication signal-to-noise ratio and the minimum signal-to-noise ratio threshold includes:

[0024] Obtaining a minimum signal power based on a product of the second received power and the minimum signal-to-noise ratio threshold;

[0025] The communication space constraint information is obtained based on a ratio of the minimum signal power to the communication signal-to-noise ratio.

[0026] In some embodiments, before inputting all of the group feature perception information, the CPU utilization, and the previous moment trusted device node sequence into a federated node selection model for node selection, the method further includes:

[0027] Obtaining group feature perception information parameters, CPU utilization parameters, trusted device node policy parameters, and local loss value parameters, and obtaining a state space based on the group feature perception information parameters, the CPU utilization parameters, the trusted device node policy parameters, and the local loss value parameters;

[0028] Based on the selected action of the device node, an action space is obtained, and based on the local loss value parameter and the trusted device node policy parameter, a reward function is obtained;

[0029] Constructing an action network and an evaluation network based on a long short-term memory algorithm, constructing a corresponding target action network based on the action network, and constructing a corresponding target evaluation network based on the evaluation network;

[0030] Acquire training samples based on the state space and the action space, train the action network and the evaluation network using the training samples and the reward function, and update the target action network and the target evaluation network during the training process;

[0031] The trained target action network is used as the federated node selection model.

[0032] In some embodiments, obtaining the local loss value parameter includes:

[0033] Obtaining a node loss function of the device node, and obtaining a node cumulative loss value based on the node loss function;

[0034] The local loss value parameter is obtained based on the ratio of the node cumulative loss value to the number of training samples.

[0035] In some embodiments, the step of training the action network and the evaluation network using the training samples and the reward function includes:

[0036] Obtaining action network parameters of the action network and evaluation network parameters of the evaluation network, obtaining a current training state and a current training action corresponding to the training sample, and determining a next training state and a next training action corresponding to the current training state and the current training action based on the trusted device node policy parameters;

[0037] Update action network parameters based on the action value function;

[0038] Obtaining a current reward corresponding to the current training state and the current training action based on the reward function, obtaining a predicted state-action value of the next training state and the next training action based on the action-value function, and updating the state-action value at the current moment in the action-value function based on the current reward and the predicted state-action value;

[0039] The evaluation network parameters are updated based on the evaluation loss function and the updated action value function.

[0040] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a model updating device for an Internet of Things system, wherein the Internet of Things system includes a plurality of device nodes, and the device includes:

[0041] a first acquisition module, configured to acquire group characteristic perception information of each of the device nodes at a current moment, wherein the group characteristic perception information is used to characterize the activity level of the device node, the frequency of interaction between the device node and the controller, and the communication quality between the device node and other device nodes;

[0042] The second acquisition module is used to obtain the CPU utilization of each device node at the last moment and the trusted device node sequence at the last moment;

[0043] A data processing module is used to input all the group feature perception information, the CPU utilization rate and the trusted device node sequence at the previous moment into a federated node selection model to perform node selection and obtain a trusted device node sequence at the current moment;

[0044] A federated learning module is used to determine the target device node based on the current trusted device node sequence, obtain local model parameters of the data processing model deployed in the target device node, aggregate the local model parameters to obtain global model parameters, and send the global model parameters to each of the device nodes so that the device node uses the global model parameters to update the locally deployed data processing model.

[0045] To achieve the above-mentioned objectives, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the model update method of the Internet of Things system as described in the first aspect.

[0046] To achieve the above-mentioned purpose, the fourth aspect of the embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by the processor, it implements the model update method of the Internet of Things system described in the first aspect above.

[0047] The embodiment of the present application proposes a model update method and related equipment for an Internet of Things system. The Internet of Things system includes multiple device nodes, and the method includes: first, obtaining group feature perception information of each device node at the current moment, where the group feature perception information is used to characterize the activity level of the device node, the interaction frequency between the device node and the controller, and the communication quality between the device node and other device nodes; and, obtaining the CPU utilization of each device node at the previous moment and the trusted device node sequence at the previous moment; then, inputting all group feature perception information, CPU utilization and the trusted device node sequence at the previous moment into a federal node selection model for node selection to obtain the trusted device node sequence at the current moment; finally, determining the target device node according to the trusted device node sequence at the current moment, obtaining the local model parameters of the data processing model deployed in the target device node, aggregating the local model parameters to obtain the global model parameters, and sending the global model parameters to each device node, so that the device node uses the global model parameters to update the locally deployed data processing model. The embodiment of the present application utilizes group characteristic perception information that characterizes the activity level, interaction frequency, and communication quality of device nodes, as well as the CPU utilization rate of device nodes as the second input parameter of the federal node selection model, thereby improving the reliability of node selection and reducing the probability of selecting attacked nodes as target device nodes, thereby improving the reliability and accuracy of the local model parameters obtained for the target device nodes, indirectly improving the reliability and accuracy of the global model parameters obtained using the local model parameters, and thus improving the efficiency of improving model performance during the model training process.

[0048] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flowchart of a model updating method for an Internet of Things system provided by one embodiment of the present application.

[0050] Figure 2 yes Figure 1 Flowchart of step 101 in FIG.

[0051] Figure 3 yes Figure 2 Flowchart of step 202 in FIG.

[0052] Figure 4 yes Figure 2 Flowchart of step 203 in FIG.

[0053] Figure 5 yes Figure 2Another flow chart of step 203 in FIG.

[0054] Figure 6 This is a flowchart of constructing a federal node selection model provided by another embodiment of the present application.

[0055] Figure 7 yes Figure 6 Flowchart of step 601 in FIG.

[0056] Figure 8 This is a network training diagram provided by another embodiment of the present application.

[0057] Figure 9 yes Figure 6 Flowchart of step 604 in FIG.

[0058] Figure 10 This is a flowchart of a federated task learning process provided by another embodiment of the present application.

[0059] Figure 11 This is a performance simulation diagram of the first type of reliable node selection provided by another embodiment of the present application.

[0060] Figure 12 This is a performance simulation diagram of the second reliable node selection provided by another embodiment of the present application.

[0061] Figure 13 1 is a schematic diagram of the structure of a model updating device for an Internet of Things system provided by an embodiment of the present application.

[0062] Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0066] First, let’s analyze some of the terms used in this application:

[0067] Federated learning is a distributed machine learning approach that allows model training while keeping data locally stored. This is particularly useful for protecting data privacy and security because data does not need to be centralized on a central server for processing. Federated learning systems train models in parallel on multiple local devices and then aggregate the model parameters to a central server for updates and merging.

[0068] Deep reinforcement learning (DRL) combines deep learning and reinforcement learning to solve complex decision-making and control problems. DRL uses neural networks to process high-dimensional state and action spaces. Common algorithms include Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO). While DRL has achieved remarkable results in areas such as gaming, robotic control, and autonomous driving, it also faces challenges such as low sample efficiency and poor training stability.

[0069] Deep Deterministic Policy Gradient (DDPG) is a deep reinforcement learning algorithm for continuous action spaces. It combines the ideas of policy gradient and Q-value function approximation, implemented using a policy network and a value network. DDPG uses a policy network to generate actions and a value network to evaluate these actions, thereby optimizing the policy and enabling the intelligent agent to effectively learn optimal decisions in high-dimensional continuous action spaces.

[0070] Long Short-Term Memory (LSTM) is a specialized recurrent neural network (RNN) designed to overcome the vanishing and exploding gradient problems of traditional RNNs when processing long sequences. By incorporating memory cells and gating mechanisms (including input, forget, and output gates) to control the flow of information, LSTM is able to retain and extract relevant information from long time series data. This makes LSTM highly effective in tasks such as time series forecasting, natural language processing, and speech recognition.

[0071] As the amount and sources of data processed by the Industrial Internet of Things increase, traditional centralized learning solutions cannot meet the processing needs of multi-source and large-scale data. Based on this, federated learning, which places the learning process on edge devices for data learning processing, can effectively meet the processing needs of multi-source and large-scale data.

[0072] In related technologies, federated learning in IoT systems typically involves multiple device nodes downloading a global training model from a controller. After each training session, the controller retrieves node training parameters from the multiple device nodes, derives global training parameters based on the multiple node training parameters, and sends these global training parameters to all device nodes for a new round of model training. However, due to the risk of network attacks during model training, the resulting node training parameters can be affected, resulting in low efficiency in improving model performance during federated learning in IoT systems.

[0073] In order to improve the reliability of model updates and the efficiency of improving model performance in the Internet of Things system, the embodiment of the present application uses group characteristic perception information that characterizes the activity level, interaction frequency and communication quality of device nodes, and the CPU utilization rate of device nodes as the second input parameter of the federal node selection model, thereby improving the reliability of node selection and reducing the probability of selecting attacked nodes as target device nodes, thereby improving the reliability and accuracy of the local model parameters obtained for the target device nodes, indirectly improving the reliability and accuracy of the global model parameters obtained using the local model parameters, and thus improving the efficiency of improving model performance during the model training process.

[0074] In order to better describe the model update method of the Internet of Things system provided by this application, the following first describes the Internet of Things system applied to the model update method of the Internet of Things system. The Internet of Things system includes a controller and multiple device nodes, each of which is deployed with a data processing model. The controller and the multiple device nodes collaborate to perform federated learning training, that is, in each round of federated learning training, the controller sends global model parameters to each device node; each device node updates and trains the data processing model according to the global model parameters and obtains local model parameters; then the controller selects at least one reliable target device node from the multiple device nodes, obtains the local model parameters of all target device nodes, and obtains new global model parameters based on the local model parameters of the target device nodes to perform a new round of federated learning training.

[0075] The following further describes the model updating method and related devices of the Internet of Things system provided by the embodiment of the present application. The model updating method of the Internet of Things system provided by the embodiment of the present application can be applied to the controller in the Internet of Things system.

[0076] The following will describe in detail the model updating method of the Internet of Things system in the embodiment of the present application. Figure 1 , which is an optional flowchart of the model updating method of the Internet of Things system provided in an embodiment of the present application, Figure 1The method may include but is not limited to steps 101 to 104. It is also understood that this embodiment is for Figure 1 The order of step 101 to step 104 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0077] Step 101: Obtain group feature perception information of each device node at the current moment.

[0078] Step 101 is described in detail below.

[0079] In some embodiments, when performing federated learning in an IoT system, in each round of iterative learning, in order to improve the reliability of the selected target device node, that is, to avoid the selected device node being attacked.

[0080] To achieve satisfactory accuracy in federated learning, continuous model iteration, updates, and testing are essential. The device nodes involved in the training process are primarily used by users and therefore possess significant group-specific attributes, namely, group-aware information. However, during this process, some devices may be unable to participate in iterative updates due to their inability to interact with the server in real time. Therefore, to achieve higher accuracy in selecting target device nodes, it is necessary to consider the group-aware information of device nodes to determine their reliability.

[0081] Before selecting the target device node, the controller needs to obtain the group feature perception information of each device node at the current moment. It can be understood that the group feature perception information is used to characterize the node activity corresponding to the activity level of the device node, the node controller interaction corresponding to the interaction frequency between the device node and the controller, and the communication space constraint information corresponding to the communication quality between the device node and other device nodes, so as to facilitate the subsequent use of the group feature perception information to select the target device node.

[0082] The following will further describe how to obtain group feature perception information.

[0083] Reference Figure 2 , obtaining the group feature perception information of each device node at the current moment, including the following steps 201 to 203.

[0084] Step 201: Obtain the number of interactions between the current device node and other device nodes, and obtain the node activity based on the ratio of the number of interactions to the number of device nodes.

[0085] Step 201 is described in detail below.

[0086] In some embodiments, device nodes in IoT systems demonstrate high network resource utilization efficiency and are highly dependent on them by other device nodes, resulting in significant activity. These device nodes frequently interact with other device nodes and controllers. Therefore, accurately quantifying the node activity of device nodes is crucial for ultimately selecting target device nodes for training.

[0087] Therefore, we first obtain the number of interactions between each device node and other device nodes at the current moment. For example, the number of interactions between the i-th device node and all other device nodes is Where N is the total number of device nodes. Next, based on the ratio of the number of interactions of the i-th device node to the number of device nodes N, the node activity of the i-th device node is obtained as shown in the following formula (1).

[0088]

[0089] Wherein, α is an interaction indicator, which is used to characterize whether there is interaction between device nodes. The interaction indicator α(i, j) of the i-th device node and the j-th device node is shown in the following formula (2).

[0090]

[0091] Step 202: Obtain the interaction frequency between the device node and the controller at the current moment and the interaction duration of each interaction frequency, and obtain the node controller interaction degree based on the interaction frequency and interaction duration.

[0092] Step 202 is described in detail below.

[0093] In some embodiments, the difference in group characteristics between device nodes is also reflected in the degree of interaction between the device node and the controller, that is, the degree of intimacy between the device node and the controller. That is, the interaction frequency NoI between the i-th device node and the controller at the current moment and the interaction duration (END-BEG) of each interaction frequency are obtained, where the starting time of the fre-th interaction is represented by BEG. fre Indicates that the end time of the freth interaction is indicated by END fre Next, we calculate the node controller interaction degree based on the interaction frequency NoI and the interaction duration (END-BEG). The node controller interaction degree includes the average interaction duration, interaction degree, and communication probability. The following describes how to calculate the node controller interaction degree.

[0094] Reference Figure 3 , obtaining the node controller interaction degree based on the interaction frequency and the interaction duration, including the following steps 301 to 303.

[0095] Step 301: Accumulate all interaction durations in the current moment to obtain the moment interaction time, and obtain the average interaction duration based on the ratio of the moment interaction time to the interaction frequency.

[0096] Step 302: Obtain the interaction time and interaction interval time between the device node and the controller, accumulate the interaction time and interaction interval time to obtain the interaction cost time, and obtain the interaction degree based on the ratio of the moment interaction time to the interaction cost time.

[0097] Step 303: Obtain a decay interaction function based on the exponential decay function and the degree of interaction, and obtain a communication probability based on a ratio of a preset communication quantity to the decay interaction function.

[0098] Steps 301 to 303 are described in detail below.

[0099] In some embodiments, after obtaining the interaction frequency NoI and interaction duration (END-BEG) of the i-th device node, all interaction durations of the i-th device node at the current moment are accumulated to obtain the moment interaction time, and based on the ratio of the moment interaction time to the interaction frequency NoI, the average interaction duration of the i-th device node at the current moment is obtained. As shown in the following formula (3).

[0100]

[0101] Next, get the interaction time between the i-th device node and the controller and interaction interval Then accumulate the interaction time and interaction interval Get the interaction cost time of the i-th device node And based on the ratio of the interaction time and interaction cost time to the interaction frequency NoI, the interaction degree θ of the i-th device node is obtained. i,j As shown in the following formula (4).

[0102]

[0103] If there is no interaction between the i-th device node and the controller at the current moment, the interaction level between the i-th device node and the controller is based on the interaction level at the previous moment. However, as time goes by, the reference value of this interaction level gradually decreases. Therefore, based on the exponential decay function and the interaction level, the decay interaction function is obtained as shown in the following formula (5).

[0104]

[0105] Among them, NEWi,j It indicates the updated interaction level, and nt indicates the current moment.

[0106] As the interaction time increases and the interaction degree parameter decreases, the probability that the device node can communicate with the controller also tends to increase monotonically, and finally the function curve with a value between 0 and 1 tends to be horizontal. In special cases, when the device node and the controller are always in an interactive state, each interaction interval is 0, then the interaction probability between the device node and the controller approaches 1, and vice versa. Based on this change trend, the communication probability between the i-th device node and the controller is obtained based on the ratio of the preset communication number (i.e., 2) and the attenuated interaction function (5) as shown in the following formula (6).

[0107]

[0108] Step 203: Obtain the communication signal-to-noise ratio between the device node and other device nodes at the current moment, and obtain the minimum signal-to-noise ratio threshold, and obtain communication space constraint information based on the communication signal-to-noise ratio and the minimum signal-to-noise ratio threshold.

[0109] The following step 203 is described in detail.

[0110] In some embodiments, with the explosive growth of IoT data, traditional cellular technologies that consume a large amount of network resources can no longer fully meet the needs of IoT applications. Therefore, in this application, communication between device nodes and processors uses D2D technology, that is, data is transmitted directly between device nodes. This approach effectively alleviates the data burden on base stations and core networks, greatly improves network resource utilization, expands network capacity, and thus ensures the efficiency and flexibility of the communication network. Therefore, in order to ensure the establishment of reliable D2D links, communication constraints in the spatial domain must be considered.

[0111] That is, in addition to obtaining the node activity and node controller interaction, it is also necessary to obtain the communication signal-to-noise ratio CS between the i-th device node and the j-th device node at the current moment. i,j , and obtain the minimum signal-to-noise ratio threshold between two device nodes to meet communication interaction Then, using the communication signal-to-noise ratio and minimum signal-to-noise ratio threshold The communication space constraint information of the i-th device node is obtained. The following further describes how to obtain the communication space constraint information.

[0112] Reference Figure 4 , obtaining the communication signal-to-noise ratio between the device node and other device nodes at the current moment, and obtaining the minimum signal-to-noise ratio threshold, including the following steps 401 to 404.

[0113] Step 401: Obtain a first node channel gain between the device node and a first other device node, obtain a second node channel gain between a second other device node and the first other device node, obtain a first transmit power of the device node, and obtain a second transmit power of the second other device node.

[0114] Step 402: Obtain a first receiving power based on the product of the first node channel gain and the first transmission power, obtain a second receiving power based on the product of the second node channel gain and the second transmission power, and obtain a communication signal-to-noise ratio based on the ratio of the first receiving power to the second receiving power.

[0115] Step 403: Acquire the channel bandwidth, select the minimum communication signal-to-noise ratio from multiple communication signal-to-noise ratios as the minimum signal-to-noise ratio, and multiply the logarithm of the minimum signal-to-noise ratio by the channel bandwidth to obtain the minimum communication rate.

[0116] Step 404: Obtain a minimum signal-to-noise ratio threshold based on an index of a minimum communication rate of a preset number of communications.

[0117] Steps 401 to 404 are described in detail below.

[0118] In some embodiments, the first node channel gain CG between the i-th device node and the first (ie, j-th) other device node is first obtained. i,j ; and obtain the second node channel gain CG between the second (i.e., kth) other device node and the first (i.e., jth) other device node k,j ; and obtain the first transmission power TP of the i-th device node i , and obtain the second transmission power TP of the second (i.e., kth) other device node k .

[0119] Next, based on the first node channel gain CG between the i-th device node and the first (i.e., j-th) other device node i,j The square of the modulus and the first transmission power TP of the i-th device node i The product of the first received power |CG i,j | 2 TP i ; and, based on the second node channel gain CG between the second (ie, kth) other device node and the first (ie, jth) other device node k,j and the second transmission power TP of the second (i.e., kth) other device node k The product of the second received power |CG k,j | 2 TP k , and based on the first received power |CG i,j |2 TP i and the second received power |CG k,j | 2 TP k The communication signal-to-noise ratio CS between the i-th device node and the first (i.e., j-th) other device node is obtained by the ratio of i,j As shown in the following formula (7).

[0120]

[0121] Among them, σ 2 represents the power of additive white Gaussian noise.

[0122] Next, obtain the channel bandwidth B allocated in the D2D communication link and calculate the communication signal-to-noise ratio CS of multiple device nodes. i,j The minimum communication signal-to-noise ratio is selected as the minimum signal-to-noise ratio min(CS), and the minimum communication rate is obtained based on the logarithm of the minimum signal-to-noise ratio log2[1+min(CS)] and multiplied by the channel bandwidth B. The following formula (8) shows this.

[0123]

[0124] After that, the minimum communication rate based on the preset communication number (i.e. 2) The index of the minimum signal-to-noise ratio threshold is obtained As shown in the following formula (9).

[0125]

[0126] Reference Figure 5 , obtaining communication space constraint information based on the communication signal-to-noise ratio and the minimum signal-to-noise ratio threshold, including the following steps 501 to 502.

[0127] Step 501: Obtain minimum signal power based on the product of the second received power and the minimum signal-to-noise ratio threshold.

[0128] Step 502: Obtain communication space constraint information based on a ratio of minimum signal power to communication signal-to-noise ratio.

[0129] Steps 501 to 502 are described in detail below.

[0130] In some embodiments, after obtaining the communication signal-to-noise ratio CS between the i-th device node and other device nodes, i,: and minimum signal-to-noise ratio threshold Afterwards, based on the second received power |CG between the second (i.e., kth) other device node and other device nodes k,: | 2 TPk Minimum signal-to-noise ratio threshold The product of the minimum signal power is obtained Then based on the minimum signal power The communication signal-to-noise ratio CS between the i-th device node and the j-th device node i,j The ratio of the communication space constraint information is obtained As shown in the following formula (10).

[0131]

[0132] Through the relevant steps of the above-mentioned step 101 (including steps 201 to 203, 301 to 303, 401 to 404, and 501 to 502), the interaction information between the device node and other device nodes and the controller is used to accurately obtain the node activity for describing the interaction activity and interaction dependency between the device nodes, and the node interaction for describing the degree of interaction and intimacy between the device nodes and the device nodes and between the device nodes and the controller; and the communication link information between the device nodes is used to accurately obtain the communication space constraint information for describing the reliable communication requirements between the device nodes and the device nodes in the Internet of Things system, so that when the node activity, node interaction and communication space constraint information are subsequently used to select the target device node, the selection of attacked device nodes, infrequently used device nodes and device nodes whose communication quality cannot meet the communication requirements is reduced, thereby improving the accuracy and reliability of the selection of the target device node.

[0133] Step 102: Obtain the CPU utilization of each device node at the last moment and the sequence of trusted device nodes at the last moment.

[0134] Step 102 is described in detail below.

[0135] In some embodiments, in order to further select a more suitable target device node, the group feature perception information of each device node at the current moment is obtained. In addition, you also need to obtain the CPU utilization of each device node at the last moment And the trusted device node sequence selected at the last moment (i.e., the target device node sequence selected during data model training in the previous federated learning process).

[0136] Step 103: All group feature perception information, CPU utilization, and the trusted device node sequence at the previous moment are input into the federated node selection model to perform node selection, and obtain the trusted device node sequence at the current moment.

[0137] Step 103 is described in detail below.

[0138] In some embodiments, after obtaining the group characteristic perception information of each device node at the current moment, The CPU utilization of each device node at the last moment And the trusted device node sequence selected at the last moment Afterwards, the group feature perception information of each device node at the current moment is The CPU utilization of each device node at the last moment And the trusted device node sequence selected at the last moment Input the pre-built federated node selection model to select nodes and obtain the current trusted device node sequence selected at the current moment. Therefore, when the current trusted device node sequence is subsequently used to select the target device node and obtain the local model parameters of the target device node, the probability of obtaining the attacked local model parameters is reduced, and the validity and reliability of the obtained local model reference are improved, thereby improving the reliability of federated learning and the efficiency of improving model performance during data processing model training.

[0139] In some embodiments, in order to further improve the accuracy of selecting the target device node, it is necessary to pre-build a suitable federated node selection model. The following further describes how to obtain the suitable federated node selection model.

[0140] In some embodiments, the performance evaluation index of federated learning is defined as the accuracy of the global data processing model after aggregation. This accuracy index is specifically quantified by the sum of the loss functions after training. Based on the federated learning process (i.e., the training process of the data processing model of multiple device nodes), the target device node selection accuracy is used as the performance evaluation index, and the data transmission delay on the network link of the Internet of Things system and the training delay of multiple device nodes are used as constraints to achieve the efficiency and accuracy of data processing model training. A node selection optimization problem is constructed as shown in the following formula (11).

[0141]

[0142] Among them, H refers to the number of tasks in the federated learning task, ACC h Refers to the node selection accuracy during the h-th federated learning task, Refers to the data transmission delay during the h-th federated learning task in the IoT system. Refers to the training latency of the data processing model training during the hth federated learning task in the device node, Refers to the maximum delay during the h-th federated learning task, Refers to the maximum data transmission delay, Refers to the maximum training delay, f i h Refers to the COU utilization rate during the hth federated learning task in the i-th device node, The minimum CPU utilization during the hth federated learning task. The core goal of the node selection optimization problem is to select a reliable set of nodes to achieve the best accuracy for data processing model training at the current moment, that is, to minimize the overall federated learning loss function.

[0143] Consider the task of interacting with a controller and device nodes in an IoT system, across a series of states, actions, and rewards within a communication environment. The controller can observe the current state but may not fully understand the current situation. Therefore, it considers sequences of actions and observations during global iterations. This allows the controller to learn a policy that depends on these sequences. Therefore, the target device node selection problem is formulated as a finite Markov decision process, where each sequence is a distinct or independent state. The target device node selection for each training step is then solved using knowledge from reinforcement learning. The following describes how to construct a federated node selection model based on this node selection optimization problem.

[0144] Reference Figure 6 Before inputting all group feature perception information, CPU utilization, and the trusted device node sequence at the previous moment into the federated node selection model for node selection, the model updating method of the Internet of Things system further includes the following steps 601 to 605.

[0145] Step 601: Obtain group feature perception information parameters, CPU utilization parameters, trusted device node policy parameters, and local loss value parameters, and obtain a state space based on the group feature perception information parameters, CPU utilization parameters, trusted device node policy parameters, and local loss value parameters.

[0146] Step 601 is described in detail below.

[0147] In some embodiments, reinforcement learning theory requires constructing a finite Markov decision process, along with a state space, action space, and reward function. The state space refers to the set of all possible states that the IoT system can be in at a given moment. Each state contains all current information about the environment and is used to describe the specific state of the IoT system.

[0148] Therefore, in this embodiment, combined with the relevant situation in the Internet of Things system, first obtain the group feature perception information parameters of each device node at a certain current moment The CPU utilization parameter of each device node at the last moment Trusted device node policy parameters at the last moment And get the local loss value parameter l of each device node at the current moment t (w i ). The following will further describe how to obtain the local loss value parameters of the device node at the current moment.

[0149] Reference Figure 7 , obtaining local loss value parameters, including the following steps 701 to 702.

[0150] Step 701: Obtain a node loss function of a device node, and obtain a node cumulative loss value based on the node loss function.

[0151] Step 702: Obtain a local loss value parameter based on the ratio of the node cumulative loss value to the number of training samples.

[0152] In some embodiments, the node loss function f of each device node is first obtained. ds (w), where the node loss function can be a commonly used loss function, including mean square error, mean absolute error, etc., and w refers to the loss parameter. Next, the node loss function of all device nodes is accumulated to obtain the node cumulative loss value ∑f ds (w), then based on the ratio of the node cumulative loss value to the number of training samples of each device node, the local loss value parameter of each device node is obtained as shown in the following formula (12).

[0153]

[0154] Where ds is the local data set lds of the i-th device node i The data sample in w i are the local model parameters of the i-th device node.

[0155] In some embodiments, based on the obtained group characteristic perception information parameters of each device node at a certain current moment The CPU utilization parameter of each device node at the last moment Trusted device node policy parameters at the last moment And get the local loss value parameter l of each device node at the current moment t (w i ), generating the state space in

[0156] Step 602: Based on the selected action of the device node, an action space is obtained, and a reward function is obtained based on the local loss value parameter and the trusted device node policy parameter.

[0157] Step 602 is described in detail below.

[0158] In some embodiments, the action space refers to the set of all possible actions that the controller can take in each state. At the tth moment of the global iteration of the IoT system, the controller selects an action. In this application, the controller adopts a single node selection scheme. In order to effectively express this selection, a binary vector model is constructed for the node selection scheme at time t. This vector describes the node selection status at each moment. Each element in this vector can only take the value 0 or 1, clearly identifying the selection status of each device node at the current moment t. Specifically, when an element in the vector is 0, it means that the device node with the corresponding value has not been selected during the current node selection process; conversely, if the element value is 1, it means that the device node has been selected for the operation.

[0159] In some embodiments, the reward function defines the immediate reward that the controller receives at each iteration. The reward is a numerical value that represents the feedback the controller receives after taking an action in the current state. The reward of the controller at the current global training iteration can be defined as the current state and the action taken at time. The controller will receive a reward to evaluate its current behavior, and the environment information in the current state will also be updated accordingly. The reward function is based on the local loss value parameter l of each device node. t (w i ) and the current trusted device node policy parameters Get the reward function As shown in the following formula (13).

[0160]

[0161] Step 603: constructing an action network and an evaluation network based on the long short-term memory algorithm, constructing a corresponding target action network based on the action network, and constructing a corresponding target evaluation network based on the evaluation network.

[0162] Step 604: Acquire training samples based on the state space and the action space, and use the training samples and the reward function to train the action network and the evaluation network, and update the target action network and the target evaluation network during the training process.

[0163] Step 605: Use the trained target action network as a federated node selection model.

[0164] Steps 603 to 605 are described in detail below.

[0165] In some embodiments, after obtaining the state space, action space, and reward function, in order to obtain the optimal solution to the node selection optimization problem (11), deep reinforcement learning (DQN) can be applied to this problem. However, as the number of device nodes increases, the processor problem becomes high-dimensional, and applying DQN to solve such problems will aggravate the instability of the control system. The limitation of DQN is that it is only suitable for processing low-dimensional and discrete action spaces. Due to its limited action set, its ability to determine the optimal strategy is limited. Therefore, this application adopts an optimized deep deterministic policy gradient (DDPG) algorithm to solve the problem of selecting reliable nodes.

[0166] Unlike DQN, DDPG can effectively handle problems involving continuous action spaces. DDPG consists of an action network and a critic network. Each network is composed of two neural networks with identical architectures, including a critic network and a target network. Specifically, there are four networks: an action network, a target-action network, a critic network, and a target-criteria network. This application also integrates the Long Short-Term Memory (LSTM) algorithm with DDPG to propose a CoLD algorithm. This algorithm is designed to enhance the stability of the controller during training and has broad application potential, particularly in tasks with temporal and historical dependencies. Compared to the contextual relationships in text-based tasks, network traffic data has longer temporal dependencies. However, traditional sequence models perform poorly in capturing these longer-range temporal features, limiting their application in attack detection. The CoLD algorithm includes two key components: an LSTM-based action network and an LSTM-based critic network. The action network is responsible for generating actions based on the state provided by the environment, while the critic network provides feedback based on the executed actions. The target-action network and target-criteria network are designed to enhance stability during training. The proposed algorithm not only effectively maintains the internal state of the network but also aggregates state observations over time. This capability is crucial for processing tasks that contain historical information, as it enables the controller to infer the actual state and make more accurate decisions.

[0167] Based on the above ideas, this application constructs an action network and an evaluation network based on the long short-term memory algorithm, and then constructs a corresponding target action network based on the action network, and constructs a corresponding target evaluation network based on the evaluation network. Next, training samples are obtained based on the state space and action space, and the training samples and reward function are used to calculate the target evaluation network. Train the action network, evaluation network, target action network, and target evaluation network. Figure 8 As shown in FIG, it is a network training diagram provided by an embodiment of the present application. Figure 8 As shown in the figure, before selecting reliable device nodes in the Internet of Things system, a large number of training samples corresponding to the state space are first obtained, and then these training samples are used to update the relevant network parameters of the action network and the evaluation network in each iteration. Then, after several iterative updates (i.e., soft updates), the relevant parameters of the target action network are periodically updated using the relevant parameters of the action network, and the relevant parameters of the target evaluation network are periodically updated using the relevant parameters of the evaluation network.

[0168] The following will further describe how to update the relevant parameters of the action network and evaluation network.

[0169] Reference Figure 9 , using training samples and reward functions to train the action network and the evaluation network, including the following steps 901 to 903.

[0170] Step 901: Obtain action network parameters of the action network and evaluation network parameters of the evaluation network, obtain the current training state and current training action corresponding to the training sample, and determine the next training state and next training action corresponding to the current training state and current training action based on the trusted device node policy parameters.

[0171] Step 902: Update action network parameters based on the action value function.

[0172] Step 903: Based on the reward function, obtain the current reward corresponding to the current training state and the current training action, based on the action value function, obtain the predicted state-action value of the next training state and the next training action, and based on the current reward and the predicted state-action value, update the state-action value at the current moment in the action value function.

[0173] Step 904: Update the evaluation network parameters based on the evaluation loss function and the updated action-value function.

[0174] Steps 901 to 903 are described in detail below.

[0175] In some embodiments, the action network parameters θ of the action network are first obtained. μ and the evaluation network parameters θ of the evaluation network Q , where μ is the action network indicator indicating the action network, and Q is the action network indicator indicating the action network. Correspondingly, the target action network indicator is μ', and the target action network parameter is θ μ' , the target evaluation network indicator is Q', and the target evaluation network parameter is θ Q' .

[0176] In some embodiments, the action network uses a policy function μ(s|θ μ ) to observe the current training status s of each device node in the infrastructure layer t , and s t The value of is input into the action network and generates the current training action a t , and based on the current training action a t Interact with device nodes.

[0177] In addition, to improve exploration, Gaussian white noise is added To ensure that the policy improves with higher rewards, the policy function maintains the action network parameters θ μ , update the action network parameters θ of the action network by applying the chain rule to the expected return starting from the starting distribution J μ As shown in the following formula (14).

[0178]

[0179] in, is the size of the experience replay pool, Q(s,a) is the action value function. In the reinforcement learning framework, the action value function is used to evaluate the quality of taking an action in a certain state. It is related to the reward function. Closely related.

[0180] Next, based on the trusted device node policy parameters Determine the current training state s t and the current training action a t The corresponding next training state s t+1 and the next training action a t+1 To ensure the correct update of the action network, DDPG uses an evaluation network to evaluate the strategy output by the action network. The evaluation network evaluates the state action value based on the current training action output by the action network and the current training state input, and uses the Bellman equation to learn. Specifically, it is based on the reward function Get the current training state s t and the current training action a t The corresponding current reward r t (s t ,a t ), and obtain the next training state s based on the action value function Q(s,a) t+1 and the next training action a t+1 The predicted state action value Q(s t+1 ,a t+1 |θ Q ), based on the current reward r t (s t ,at ) and predicted state action value Q(s t+1 ,a t+1 |θ Q ) Update the state action value Q(s) at the current moment in the action value function t ,a t |θ Q ) is shown in the following formula (15).

[0181]

[0182] Where γ is the discount factor, ranging from [0,1].

[0183] Next, in order to update the evaluation network parameters θ Q , extract small batches of four-tuple training samples from the experience replay pool and calculate the Q-state action reference value, as shown in the following formula (16).

[0184] y t =r(s t ,a t )+γQ'(s t+1 ,μ'(s t+1 |θ μ' )|θ Q' ), (16)

[0185] Among them, θ μ' is the target action network parameter, θ Q' is the target evaluation network parameter. Therefore, the evaluation loss function of the evaluation network is shown in the following formula (17).

[0186]

[0187] Based on this, the evaluation network parameters θ can be updated using the evaluation loss function (17) and the updated action value function Q(s,a) Q .

[0188] In addition, in order to optimize the actions selected by the action network to obtain higher rewards, the loss function of the action network is the output state action value of the evaluation network. And, in order to improve the learning stability of the state action value function and the policy function, a soft update factor is used. Slowly update the target action network parameters θ μ' and target evaluation network parameters θ Q' As shown in the following formulas (18) and (19).

[0189]

[0190] Through the above steps 901 and 904, the training state corresponding to the training sample is utilized, and the training action obtained by the action network according to the training state, as well as the next training state and the next training action determined according to the trusted device node policy parameters, and the action network parameters are updated in combination with the action value function, and the action value function is updated, and the evaluation network parameters are further updated using the updated action value function, and the target action network parameters and the target evaluation network parameters are soft-updated, thereby effectively improving the discovery of a better node selection strategy and learning, thereby improving the accuracy of the subsequent trusted node selection using the output target action network, so as to improve the training efficiency of the data processing model.

[0191] Through the above steps 601 to 605, the node selection optimization problem (11) constructed to improve the node selection accuracy in the training process of the data processing model in the Internet of Things system is used, and the deep deterministic policy gradient (DDPG) is used to transform the node selection optimization problem (11) into the corresponding state space, action space, and reward function, and based on this, the action network, evaluation network, target action network and target evaluation network are constructed and model training is performed, and finally a reliable federated node selection model is obtained to effectively improve the accuracy of selecting trusted device nodes and significantly reduce the probability of selecting attacked device nodes; in addition, the action network and evaluation network constructed in combination with the long short-term memory algorithm can effectively maintain the internal state of the network and can aggregate state observations over time. This ability is crucial for processing tasks containing historical information because it enables the controller to infer the actual state and make more accurate decisions, thereby significantly improving the reliability and accuracy of the local model parameters of the target device node obtained, indirectly improving the reliability and accuracy of the global model parameters obtained using the local model parameters, and thus improving the efficiency of improving model performance during the model training process.

[0192] In some embodiments, after multiple rounds of training of the action network, the evaluation network, the target action network, and the target evaluation network, the trained target action network is output and used as a federated node selection model. It is understood that based on the idea of ​​reinforcement learning, the trained federated node selection model can output the optimal trusted device node sequence under different states (i.e., group feature perception information, CPU utilization, and the trusted device node sequence at the previous moment), thereby effectively solving the node selection optimization problem (11) mentioned above.

[0193] In some embodiments, after obtaining a reliable federal node selection model, the group feature perception information of all device nodes at the current moment, the CPU utilization of all device nodes at the previous moment, and the trusted device node sequence at the previous moment are input into the federal node selection model for node selection to obtain the trusted device node sequence at the current moment.

[0194] Step 104: Determine the target device node based on the trusted device node sequence at the current moment, obtain the local model parameters of the data processing model deployed in the target device node, aggregate the local model parameters to obtain the global model parameters, and send the global model parameters to each device node so that the device node uses the global model parameters to update the locally deployed data processing model.

[0195] Step 104 is described in detail below.

[0196] In some embodiments, after obtaining the trusted device node sequence at the current moment, the target device node is determined according to the trusted device node sequence at the current moment, the local model parameters of the data processing model deployed in the target device node are obtained, the local model parameters are aggregated to obtain global model parameters, and the global model parameters are sent to each device node so that the device node uses the global model parameters to update the locally deployed data processing model.

[0197] Based on this, there are two independent processes in the process of learning the federated task of the data processing model through the model updating method of the Internet of Things system provided by this application. Figure 10 , is a flowchart of a federated task learning process provided by the embodiment of this application. Figure 10As shown, these two independent processes specifically include: 1. Deep reinforcement learning based on crowd feature perception: This process involves the infrastructure layer (i.e., multiple device nodes), the SDN-based core network layer, the SDN control plane (i.e., controller), and the deep reinforcement learning agent server (which can be either a controller or a server connected to the IoT system). Specifically, the deep reinforcement learning agent requests data and crowd feature perception from the SDN-based core network layer and SDN controller. The SDN-based core network and SDN controller request data from the infrastructure layer and receive a response. The deep reinforcement learning algorithm detects reliable nodes based on this data, eliminating unreliable nodes and feeding these reliable device nodes back to the SDN control layer. Finally, the SDN controller layer aggregates the reliable nodes fed back by the agent to form a secure global FL model. 2. Distributed learning process based on the SDN control layer: Local models (i.e., local models) are trained at the infrastructure layer, and the local model parameters are uploaded to the SDN controller layer. The SDN control layer aggregates the uploaded local model parameters to form a global model. The global model parameters are then distributed to each node in the FL, and this process is repeated until the model converges.

[0198] In addition, the above process can be assisted by the following algorithm 1.

[0199]

[0200] In some embodiments, in order to further verify the reliability of the model update method of the Internet of Things system provided in the embodiments of the present application. The performance indicators of the model update method of the Internet of Things system were tested by the network simulation software Mininet-WiFi. The scenario includes a deep reinforcement learning proxy server, 6 Open vSwitches as the core network of the system, 6 network access points, each access point includes 10 terminal devices, and a total of 60 terminal devices. The simulation process is based on pytorch2.0, python 3.10.0 and OpenAI's Gym environment interface to connect the DRL algorithm and the training environment. Numerical calculations mainly rely on the Numpy open source library. The RYU controller plays a key role in the network framework of FedSPA and realizes centralized network management. The simulation hardware implementation platform is the Ubuntu 22.04.2LTS operating system and the NVIDIA TESLA V100 16G graphics card.

[0201] A convolutional neural network with two convolutional layers, two pooling layers, and two fully connected layers was selected as the FL training model. Experiments were conducted on the MNIST dataset, which consists of 70,000 grayscale images of handwritten digits from 0 to 9 at a resolution of 28*28. The training set contains 60,000 images and labels, and the test set contains 10,000 images and labels. The training set was randomly divided into 60 sub-datasets, each of which was sent to each device node. Unreliable or poorly trained device nodes in the experiment may exhibit malicious behavior. Malicious device nodes were randomly assigned during each global iteration of federated task learning, with probabilities set to 20% and 50%. To simulate poisoning attacks, a prominent pixel pattern was randomly inserted into the MNIST dataset.

[0202] The initial neural network structure constructed consists of 1 LSTM layer, 1 fully connected layer and 1 SoftMax layer. In addition, in order to optimize computational efficiency, the same activation function is used in both the LSTM layer and the fully connected layer. Based on this structure, other parameters are further refined and optimized through model training. The training round is set to 500 and the time series step size is configured to 10. In this paper, the performance of the proposed mechanism will be demonstrated and compared with three other algorithms. (1) FedSAP, the model update method for the Internet of Things system proposed in this paper. (2) FedAvg, randomly selects some device nodes in the federated learning system for model aggregation. (3) There is no group feature perception in FedSAP, and the others are the same as the proposed method. (4) Local training, model training is performed only on the local node of the device, and no federated learning training is performed.

[0203] Reference Figure 11 , is a performance simulation diagram of the first reliable node selection provided by this application. Figure 11 As shown in Figure 2, the performance of the FL global model under four schemes: FedPSA, FedPSA without group feature perception, FedAvg, and local training. The fluctuation of the accuracy when 20% of the device nodes are designated as malicious nodes. Figure 11It is clear that in the initial stages of training, the model accuracy achieved by all four mechanisms is relatively low. This phenomenon occurs because the server lacks information about the FL environment, which in turn affects model training. To ensure model training accuracy, sufficient training cycles are essential. After 10 cumulative iterations, the models trained using the four different mechanisms all showed a stable trend in accuracy, with FedPSA achieving the highest accuracy. The global FL model accuracy for FedPSA, FedPSA without group feature awareness, FedAvg, and locally trained FL remained stable at 91%, 88%, 85%, and 73%, respectively. These results demonstrate that even in the presence of a limited number of malicious device nodes and varying data quality, the FedPSA mechanism maintains robust training performance, while locally learned models struggle to maintain training quality.

[0204] Reference Figure 12 , is a performance simulation diagram of the second reliable node selection provided by this application. Figure 12 As shown in Figure 2, the number of malicious nodes is increased from 20% to 50% of the total number of original device nodes. Figure 12 shown. Figure 12 The accuracy of the global FL model for the four schemes is shown. As the number of global iterations increases, the accuracy of the global model begins to improve for all four schemes. Compared to the other schemes, the FedPSA mechanism achieves significant improvement in global model accuracy after approximately 19 iterations. The global model accuracies for FedPSA, FedPSA without group feature awareness, FedAvg, and locally trained models are 84%, 73%, 69%, and 52%, respectively. The FedPSA mechanism utilizes the group feature-aware DDPG algorithm, balancing data quality and device training capabilities.

[0205] The model updating method and related equipment of the Internet of Things system proposed in the embodiment of the present application, the Internet of Things system includes multiple device nodes, and the method includes: first, obtaining the number of interactions between the device node and other device nodes at the current moment, and obtaining the node activity based on the ratio of the number of interactions to the number of device nodes, obtaining the interaction frequency between the device node and the controller at the current moment and the interaction duration of each interaction frequency, and accumulating all interaction durations in the current moment to obtain the moment interaction time, and obtaining the average interaction duration based on the ratio of the moment interaction time to the interaction frequency, obtaining the interaction time and interaction interval time between the device node and the controller, accumulating the interaction time and interaction interval time to obtain the interaction cost time, and accumulating the interaction time and interaction interval time to obtain the interaction cost time based on the ratio of the interaction time to the interaction frequency. The interaction degree is obtained by the ratio of the interaction time and the interaction cost time at the moment, the attenuation interaction function is obtained based on the exponential attenuation function and the interaction degree, and the communication probability is obtained based on the ratio of the preset communication quantity and the attenuation interaction function, the first node channel gain between the device node and the first other device node is obtained, the second node channel gain between the second other device node and the first other device node is obtained, the first transmission power of the device node is obtained, and the second transmission power of the second other device node is obtained, the first receiving power is obtained based on the product of the first node channel gain and the first transmission power, the second receiving power is obtained based on the product of the second node channel gain and the second transmission power, and the first receiving power and the second receiving power are obtained. The communication signal-to-noise ratio is obtained by the ratio, the channel bandwidth is obtained, the minimum communication signal-to-noise ratio is selected from multiple communication signal-to-noise ratios as the minimum signal-to-noise ratio, the minimum communication rate is obtained based on the logarithm of the minimum signal-to-noise ratio and multiplied by the channel bandwidth, the minimum signal-to-noise ratio threshold is obtained based on the exponent of the minimum communication rate of the preset communication number, and the minimum signal power is obtained based on the product of the second received power and the minimum signal-to-noise ratio threshold, and the communication space constraint information is obtained based on the ratio of the minimum signal power to the communication signal-to-noise ratio; and the CPU utilization of each device node at the previous moment and the trusted device node sequence at the previous moment are obtained; in addition, the group feature perception information parameter, the CPU utilization parameter, the trusted device node strategy parameter, and the local loss value parameter are obtained. , and based on the group feature perception information parameters, CPU utilization parameters, trusted device node policy parameters and local loss value parameters, obtain the state space, based on the device node selection action, obtain the action space, based on the local loss value parameters and the trusted device node policy parameters, obtain the reward function, based on the long short-term memory algorithm to build the action network and the evaluation network, based on the action network to build the corresponding target action network, and based on the evaluation network to build the corresponding target evaluation network, based on the state space and action space to obtain training samples, and use the training samples and reward function to train the action network and evaluation network, and update the target action network and target evaluation network during the training process, and use the trained target action network as the federated node selection model;Then, all group feature perception information, CPU utilization, and the previous trusted device node sequence are input into the federated node selection model for node selection, resulting in the current trusted device node sequence. Finally, the target device node is determined based on the current trusted device node sequence, and the local model parameters of the data processing model deployed in the target device node are obtained. The local model parameters are aggregated to obtain the global model parameters, which are then sent to each device node so that the device node can use the global model parameters to update the locally deployed data processing model.

[0206] The embodiment of the present application utilizes the interaction information between the device node and other device nodes and the controller to accurately obtain the node activity used to describe the interaction activity and interaction dependency between the device nodes, and the node interaction used to describe the degree of interaction and intimacy between the device nodes and the device nodes and between the device nodes and the controller; and utilizes the communication link information between the device nodes to accurately obtain the communication space constraint information used to describe the reliable communication requirements between the device nodes and the device nodes in the Internet of Things system, so as to reduce the selection of attacked device nodes, infrequently used device nodes, and device nodes whose communication quality cannot meet the communication requirements when the node activity, node interaction, and communication space constraint information are subsequently used to select the target device nodes; then, the group feature perception information characterizing the activity level, interaction frequency, and communication quality of the device nodes, and the CPU utilization of the device nodes are used as the second input parameter of the federal node selection model to improve the reliability of the selected nodes and reduce the probability of selecting the attacked nodes as the target device nodes, thereby improving the reliability and accuracy of the local model parameters of the acquired target device nodes; in addition, the node activity, node interaction, and communication space constraint information are utilized to accurately obtain the communication space constraint information used to describe the reliable communication requirements between the device nodes and the device nodes in the Internet of Things system; and In the process of training the data processing model in the Internet of Things system, a node selection optimization problem (11) is constructed to improve the node selection accuracy, and the deep deterministic policy gradient (DDPG) is used to transform the node selection optimization problem (11) into the corresponding state space, action space, and reward function. Based on this, an action network, an evaluation network, a target action network, and a target evaluation network are constructed and the model is trained. Finally, a reliable federated node selection model is obtained to effectively improve the accuracy of selecting trusted device nodes and significantly reduce the probability of selecting attacked device nodes. In addition, the action network and evaluation network constructed in combination with the long short-term memory algorithm can effectively maintain the internal state of the network and can aggregate state observations over time. This ability is crucial for processing tasks containing historical information because it enables the controller to infer the actual state and make more accurate decisions, thereby significantly improving the reliability and accuracy of the local model parameters of the target device node obtained, indirectly improving the reliability and accuracy of the global model parameters obtained using the local model parameters, and thus improving the efficiency of improving model performance during model training.

[0207] The embodiment of the present application also provides a model updating device for an Internet of Things system, which can implement the model updating method of the Internet of Things system. Figure 13 , the apparatus 1300 comprises:

[0208] A first acquisition module 1310 is configured to acquire group feature perception information of each device node at the current moment. The group feature perception information is used to characterize the activity level of the device node, the frequency of interaction between the device node and the controller, and the communication quality between the device node and other device nodes.

[0209] The second acquisition module 1320 is used to obtain the CPU utilization of each device node at the last moment and the trusted device node sequence at the last moment;

[0210] The data processing module 1330 is used to input all group feature perception information, CPU utilization, and the trusted device node sequence at the previous moment into the federated node selection model to perform node selection and obtain the trusted device node sequence at the current moment;

[0211] The federated learning module 1340 is used to determine the target device node based on the current trusted device node sequence, obtain the local model parameters of the data processing model deployed in the target device node, aggregate the local model parameters to obtain the global model parameters, and send the global model parameters to each device node so that the device node uses the global model parameters to update the locally deployed data processing model.

[0212] In some embodiments, the first obtaining module 1310 is further configured to:

[0213] Obtaining the number of interactions between the device node and other device nodes at a current moment, and obtaining the node activity based on a ratio of the number of interactions to the number of device nodes;

[0214] Obtaining the interaction frequency between the device node and the controller at a current moment and the interaction duration of each interaction frequency, and obtaining the node-controller interaction degree based on the interaction frequency and the interaction duration;

[0215] The communication signal-to-noise ratio between the device node and the other device nodes at the current moment is obtained, as well as a minimum signal-to-noise ratio threshold, and the communication space constraint information is obtained based on the communication signal-to-noise ratio and the minimum signal-to-noise ratio threshold.

[0216] In some embodiments, the first obtaining module 1310 is further configured to:

[0217] Accumulating all the interaction durations at the current moment to obtain the moment interaction time, and obtaining the average interaction duration based on the ratio of the moment interaction time to the interaction frequency;

[0218] Obtaining the interaction time and the interaction interval time between the device node and the controller, accumulating the interaction time and the interaction interval time to obtain the interaction cost time, and obtaining the interaction degree based on the ratio of the interaction time at the moment to the interaction cost time;

[0219] A decay interaction function is obtained based on an exponential decay function and the degree of interaction, and the communication probability is obtained based on a ratio of a preset communication quantity to the decay interaction function.

[0220] In some embodiments, the first obtaining module 1310 is further configured to:

[0221] Obtaining a first node channel gain between the device node and a first other device node, obtaining a second node channel gain between the second other device node and the first other device node, obtaining a first transmit power of the device node, and obtaining a second transmit power of the second other device node;

[0222] Obtaining a first received power based on a product of the first node channel gain and the first transmit power, obtaining a second received power based on a product of the second node channel gain and the second transmit power, and obtaining the communication signal-to-noise ratio based on a ratio of the first received power to the second received power;

[0223] Acquire a channel bandwidth, select a minimum communication signal-to-noise ratio from a plurality of communication signal-to-noise ratios as a minimum signal-to-noise ratio, and obtain a minimum communication rate based on a logarithm of the minimum signal-to-noise ratio and multiplying it by the channel bandwidth;

[0224] The minimum signal-to-noise ratio threshold is obtained based on an index of the minimum communication rate of a preset communication quantity.

[0225] In some embodiments, the first obtaining module 1310 is further configured to:

[0226] Obtaining a minimum signal power based on a product of the second received power and the minimum signal-to-noise ratio threshold;

[0227] The communication space constraint information is obtained based on a ratio of the minimum signal power to the communication signal-to-noise ratio.

[0228] In some embodiments, the model updating apparatus 1300 of the Internet of Things system further includes a model building module 1350, which is configured to:

[0229] Obtaining group feature perception information parameters, CPU utilization parameters, trusted device node policy parameters, and local loss value parameters, and obtaining a state space based on the group feature perception information parameters, the CPU utilization parameters, the trusted device node policy parameters, and the local loss value parameters;

[0230] Based on the selected action of the device node, an action space is obtained, and based on the local loss value parameter and the trusted device node policy parameter, a reward function is obtained;

[0231] Constructing an action network and an evaluation network based on a long short-term memory algorithm, constructing a corresponding target action network based on the action network, and constructing a corresponding target evaluation network based on the evaluation network;

[0232] Acquire training samples based on the state space and the action space, train the action network and the evaluation network using the training samples and the reward function, and update the target action network and the target evaluation network during the training process;

[0233] The trained target action network is used as the federated node selection model.

[0234] In some embodiments, the model building module 1350 is further configured to:

[0235] Obtaining a node loss function of the device node, and obtaining a node cumulative loss value based on the node loss function;

[0236] The local loss value parameter is obtained based on the ratio of the node cumulative loss value to the number of training samples.

[0237] In some embodiments, the model building module 1350 is further configured to:

[0238] Obtaining action network parameters of the action network and evaluation network parameters of the evaluation network, obtaining a current training state and a current training action corresponding to the training sample, and determining a next training state and a next training action corresponding to the current training state and the current training action based on the trusted device node policy parameters;

[0239] Update action network parameters based on the action value function;

[0240] Obtaining a current reward corresponding to the current training state and the current training action based on the reward function, obtaining a predicted state-action value of the next training state and the next training action based on the action-value function, and updating the state-action value at the current moment in the action-value function based on the current reward and the predicted state-action value;

[0241] The evaluation network parameters are updated based on the evaluation loss function and the updated action value function.

[0242] In the above embodiments, the description of each embodiment has its own focus. For the parts that are not described in detail in a certain embodiment, the specific implementation of the model updating device of the Internet of Things system is basically the same as the specific implementation of the model updating method of the above-mentioned Internet of Things system, and will not be repeated here.

[0243] In an embodiment of the present application, a model updating device of an Internet of Things system utilizes interaction information between a device node and other device nodes and a controller to accurately obtain node activity for describing the interaction activity and interaction dependency between device nodes, and node interaction for describing the degree of interaction and intimacy between device nodes and between device nodes and controllers; and utilizes communication link information between device nodes to accurately obtain communication space constraint information for describing the requirements for reliable communication between device nodes and device nodes in the Internet of Things system, so as to facilitate the subsequent selection of target device nodes using node activity, node interaction, and communication space constraint information, thereby reducing the selection of attacked device nodes, infrequently used device nodes, and device nodes whose communication quality cannot meet the communication requirements; and then utilizes group feature perception information characterizing the activity level, interaction frequency, and communication quality of device nodes, as well as the CPU utilization rate of device nodes as the second input parameter of the federal node selection model, thereby improving the reliability of node selection and reducing the probability of selecting attacked nodes as target device nodes, thereby improving the reliability and accuracy of the local model parameters of the acquired target device nodes. Accuracy; In addition, the node selection optimization problem (11) constructed to improve the node selection accuracy is used in the training process of the data processing model in the Internet of Things system, and the deep deterministic policy gradient (DDPG) is used to transform the node selection optimization problem (11) into the corresponding state space, action space, and reward function, and based on this, the action network, evaluation network, target action network and target evaluation network are constructed and the model is trained, and finally a reliable federated node selection model is obtained to effectively improve the accuracy of selecting trusted device nodes and significantly reduce the probability of selecting attacked device nodes; In addition, the action network and evaluation network constructed in combination with the long short-term memory algorithm can effectively maintain the internal state of the network and can aggregate state observations over time. This ability is crucial for processing tasks containing historical information because it enables the controller to infer the actual state and make more accurate decisions, thereby significantly improving the reliability and accuracy of the local model parameters of the target device node obtained, indirectly improving the reliability and accuracy of the global model parameters obtained using the local model parameters, and thus improving the efficiency of improving model performance during the model training process.

[0244] An embodiment of the present application further provides an electronic device, including:

[0245] at least one memory;

[0246] at least one processor;

[0247] at least one program;

[0248] The program is stored in the memory, and the processor executes the at least one program to implement the model updating method of the Internet of Things system implemented in this application. The electronic device can be any smart terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), an in-vehicle computer, etc.

[0249] See also Figure 14 , Figure 14 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0250] The processor 1401 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0251] The memory 1402 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 1402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1402, and the processor 1401 calls and executes the model update method of the Internet of Things system of the embodiment of this application;

[0252] Input / output interface 1403, used to implement information input and output;

[0253] Communication interface 1404, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0254] Bus 1405 , which transmits information between various components of the device (e.g., processor 1401 , memory 1402 , input / output interface 1403 , and communication interface 1404 );

[0255] The processor 1401 , the memory 1402 , the input / output interface 1403 and the communication interface 1404 are connected to each other in communication within the device via a bus 1405 .

[0256] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the model updating method of the above-mentioned Internet of Things system is implemented.

[0257] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0258] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0259] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0260] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0261] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0262] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0263] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0264] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0265] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0266] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0267] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0268] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A model updating method for an Internet of Things system, characterized in that: The Internet of Things system includes multiple device nodes, and the method includes: Acquire group characteristic perception information of each of the device nodes at a current moment, where the group characteristic perception information is used to characterize the activity level of the device node, the interaction frequency between the device node and the controller, and the communication quality between the device node and other device nodes; Obtain the CPU utilization of each device node at the last moment and the sequence of trusted device nodes at the last moment; Inputting all the group feature perception information, the CPU utilization rate, and the trusted device node sequence at the previous moment into a federated node selection model to perform node selection, thereby obtaining a trusted device node sequence at the current moment; Determine the target device node based on the trusted device node sequence at the current moment, obtain local model parameters of the data processing model deployed in the target device node, aggregate the local model parameters to obtain global model parameters, and send the global model parameters to each of the device nodes so that the device nodes use the global model parameters to update the locally deployed data processing model.

2. The model updating method of the Internet of Things system according to claim 1, characterized in that: The group feature perception information includes node activity, node controller interaction, and communication space constraint information. Acquiring the group feature perception information of each device node at the current moment includes: Obtaining the number of interactions between the device node and other device nodes at a current moment, and obtaining the node activity based on a ratio of the number of interactions to the number of device nodes; Obtaining the interaction frequency between the device node and the controller at a current moment and the interaction duration of each interaction frequency, and obtaining the node-controller interaction degree based on the interaction frequency and the interaction duration; The communication signal-to-noise ratio between the device node and the other device nodes at the current moment is obtained, as well as a minimum signal-to-noise ratio threshold, and the communication space constraint information is obtained based on the communication signal-to-noise ratio and the minimum signal-to-noise ratio threshold.

3. The model updating method of the Internet of Things system according to claim 2, characterized in that: The node controller interactivity includes an average interaction duration, an interaction degree, and a communication probability. Obtaining the node controller interactivity based on the interaction frequency and the interaction duration includes: Accumulating all the interaction durations at the current moment to obtain the moment interaction time, and obtaining the average interaction duration based on the ratio of the moment interaction time to the interaction frequency; Obtaining the interaction time and the interaction interval time between the device node and the controller, accumulating the interaction time and the interaction interval time to obtain the interaction cost time, and obtaining the interaction degree based on the ratio of the interaction time at the moment to the interaction cost time; A decay interaction function is obtained based on an exponential decay function and the degree of interaction, and the communication probability is obtained based on a ratio of a preset communication quantity to the decay interaction function.

4. The model updating method of the Internet of Things system according to claim 2, characterized in that: The obtaining of the communication signal-to-noise ratio between the device node and the other device nodes at the current moment, and obtaining the minimum signal-to-noise ratio threshold, includes: Obtaining a first node channel gain between the device node and a first other device node, obtaining a second node channel gain between the second other device node and the first other device node, obtaining a first transmit power of the device node, and obtaining a second transmit power of the second other device node; Obtaining a first received power based on a product of the first node channel gain and the first transmit power, obtaining a second received power based on a product of the second node channel gain and the second transmit power, and obtaining the communication signal-to-noise ratio based on a ratio of the first received power to the second received power; Acquire a channel bandwidth, select a minimum communication signal-to-noise ratio from a plurality of communication signal-to-noise ratios as a minimum signal-to-noise ratio, and obtain a minimum communication rate based on a logarithm of the minimum signal-to-noise ratio and multiplying it by the channel bandwidth; The minimum signal-to-noise ratio threshold is obtained based on an index of the minimum communication rate of a preset communication quantity.

5. The model updating method of the Internet of Things system according to claim 4, characterized in that: The obtaining the communication space constraint information based on the communication signal-to-noise ratio and the minimum signal-to-noise ratio threshold includes: Obtaining a minimum signal power based on a product of the second received power and the minimum signal-to-noise ratio threshold; The communication space constraint information is obtained based on a ratio of the minimum signal power to the communication signal-to-noise ratio.

6. The model updating method of the Internet of Things system according to claim 1, characterized in that: Before inputting all of the group feature perception information, the CPU utilization rate, and the previous moment trusted device node sequence into the federated node selection model for node selection, the method further includes: Obtaining group feature perception information parameters, CPU utilization parameters, trusted device node policy parameters, and local loss value parameters, and obtaining a state space based on the group feature perception information parameters, the CPU utilization parameters, the trusted device node policy parameters, and the local loss value parameters; Based on the selected action of the device node, an action space is obtained, and based on the local loss value parameter and the trusted device node policy parameter, a reward function is obtained; Constructing an action network and an evaluation network based on a long short-term memory algorithm, constructing a corresponding target action network based on the action network, and constructing a corresponding target evaluation network based on the evaluation network; Acquire training samples based on the state space and the action space, train the action network and the evaluation network using the training samples and the reward function, and update the target action network and the target evaluation network during the training process; The trained target action network is used as the federated node selection model.

7. The model updating method of the Internet of Things system according to claim 6, characterized in that: The obtaining of local loss value parameters includes: Obtaining a node loss function of the device node, and obtaining a node cumulative loss value based on the node loss function; The local loss value parameter is obtained based on the ratio of the node cumulative loss value to the number of training samples.

8. The model updating method of the Internet of Things system according to claim 6, characterized in that: The step of training the action network and the evaluation network using the training samples and the reward function includes: Obtaining action network parameters of the action network and evaluation network parameters of the evaluation network, obtaining a current training state and a current training action corresponding to the training sample, and determining a next training state and a next training action corresponding to the current training state and the current training action based on the trusted device node policy parameters; Update action network parameters based on the action value function; Obtaining a current reward corresponding to the current training state and the current training action based on the reward function, obtaining a predicted state-action value of the next training state and the next training action based on the action-value function, and updating the state-action value at the current moment in the action-value function based on the current reward and the predicted state-action value; The evaluation network parameters are updated based on the evaluation loss function and the updated action value function.

9. A model updating device for an Internet of Things system, characterized in that: The Internet of Things system includes multiple device nodes, and the apparatus includes: a first acquisition module, configured to acquire group characteristic perception information of each of the device nodes at a current moment, wherein the group characteristic perception information is used to characterize the activity level of the device node, the frequency of interaction between the device node and the controller, and the communication quality between the device node and other device nodes; The second acquisition module is used to obtain the CPU utilization of each device node at the last moment and the trusted device node sequence at the last moment; A data processing module is used to input all the group feature perception information, the CPU utilization rate and the trusted device node sequence at the previous moment into a federated node selection model to perform node selection and obtain a trusted device node sequence at the current moment; A federated learning module is used to determine the target device node based on the current trusted device node sequence, obtain local model parameters of the data processing model deployed in the target device node, aggregate the local model parameters to obtain global model parameters, and send the global model parameters to each of the device nodes so that the device node uses the global model parameters to update the locally deployed data processing model.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the model updating method of the Internet of Things system according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the model updating method of the Internet of Things system according to any one of claims 1 to 8 is implemented.

Images