Industrial Internet of Things Channel Allocation Method Optimized for Information Freshness

Abstract

The present invention discloses an industrial Internet of Things channel allocation method for information freshness optimization, which relates to the field of wireless communication technology. The specific steps of the method are as follows: in the case where the Internet of Things nodes cannot obtain channel state information, each Internet of Things node selects its transmission channel by using the multi-armed bandit method according to the historical transmission results; when a wireless channel is selected by multiple Internet of Things nodes simultaneously and conflicts occur, the central controller makes an allocation decision by using the matching method of the maximum information age reduction. Based on the reinforcement learning and matching theory system, the present invention can effectively solve the problem of channel allocation for information age optimization in the industrial Internet of Things when the channel state information is unknown under multi-channel conditions.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technologies, and in particular, to a channel allocation method for industrial Internet of Things. Background Art

[0002] AoI (Age of Information) is defined as the time elapsed since the most recently received data packet at the target node was generated. It is a metric for evaluating data freshness and measures the freshness of data from the perspective of the target node. Most research on the age of information assumes that the global prior knowledge of the channel is known, such as transmission outage probability, statistical channel information, etc. In the industrial Internet of Things scenario, sensor devices cannot obtain the channel state information of the current environment. Therefore, optimizing the average age of information in the industrial Internet of Things under unknown channel state information has become a research hotspot.

[0003] In 2020, Rajat Talak et al. published "Improving Age of Information in Wireless Networks With Perfect Channel State Information" in IEEE / ACM Transactions on Networking, which considered the AoI minimization problem for networks with general interference constraints and time-varying channels, proposed two strategies, namely the virtual queue-based strategy and the age-based strategy, constructed a feasible scheduling set in a traversal manner, and derived a scheduling strategy for minimizing the average age of information.

[0004] In 2021, Xiaolin Wang et al. published "AoI-Aware Control and Communication Co-Design for Industrial IoT Systems" in IEEE Internet of Things Journal. In the field of industrial Internet of Things, the specific expression of the average AoI under a finite number of retransmissions was first derived, then the impact of the average AoI on control performance was studied, the joint cost of communication energy consumption and control cost was obtained, and a communication strategy including data arrival rate and code length was designed.

[0005] In 2021, Jia Wang et al. published "Sleep-Wake Sensor Scheduling for Minimizing AoI-Penalty in Industrial Internet of Things" in IEEE Internet of Things Journal. Considering a typical industrial Internet of Things application, where multiple sensors monitor some time-varying physical processes and report the measurements to a central controller through unreliable wireless channels. To save energy, each sensor may switch to the sleep mode for a period of time after successfully sending a data packet. A maximum-weight-based scheduling strategy was developed, and the achieved AoI performance is close to that of non-sleep sensors.

[0006] None of the above three methods can solve the channel allocation problem for minimizing the age of information in the industrial Internet of Things when the channel state information is unknown under multi-channel conditions. Summary of the Invention

[0007] The object of the present invention is to overcome the deficiencies of the above-mentioned background technology, and provide an age of information optimization method based on reinforcement learning and matching theory, which solves the channel allocation problem for minimizing the average age of information in the industrial Internet of Things when the channel state information is unknown.

[0008] The present invention adopts the following technical solutions to achieve the above-mentioned invention object:

[0009] The present invention proposes an industrial Internet of Things channel allocation method for optimizing information freshness. When allocating wireless channels to nodes in a state where the Internet of Things nodes cannot obtain the channel state information in the network, a multi-armed bandit method is used for decision-making. When a conflict occurs because a wireless channel is selected by multiple Internet of Things nodes simultaneously, the central controller uses a matching method with the maximum age of information reduction for allocation decision-making. The industrial Internet of Things mentioned above includes a central controller, N Internet of Things nodes, and j available wireless channels; with the central controller as the center, the Internet of Things nodes are scattered around the central controller.

[0010] Furthermore, due to the limitation of spectrum resources, the number of Internet of Things nodes considered in the present invention is more than the number of available wireless channels. In addition, data collisions will occur when multiple Internet of Things nodes transmit data packets on the same wireless channel. Therefore, each wireless channel can only be assigned to one Internet of Things node. For minimizing the average age of information of the Internet of Things, the following optimization problem is constructed:

[0011]

[0012] where, A avg is the average age of information of the system network, and A n (t) represents node mn The AoI value at time slot t, where T is the number of time slots, N is the number of nodes, and m n represents an IoT node, j represents a wireless channel, and the binary variable d n,j,t = 1 indicates that node m n selects wireless channel j to transmit a data packet in the current time slot t; otherwise, d n,j,t = 0. Condition 1) means that a wireless channel can only be selected by one node, and condition 2) means that a node can only select one wireless channel to transmit data. It can be seen from the P1 problem that when the number of time slots T approaches infinity, the average age of information will tend to a stable value, and the lower the average age of information in the network, the higher the information freshness of the network.

[0013] Furthermore, the multi-armed bandit method is as follows: Model the IoT nodes as players of a multi-armed bandit and the wireless channels as arms. The IoT nodes estimate the transmission success probability of the wireless channels by observing the historical feedback of the wireless channels. The entire transmission process of the system network can be divided into two stages:

[0014] The first stage: When 1 ≤ t ≤ N, we ensure that each wireless channel is scheduled by all IoT nodes through N rounds of time slots.

[0015]

[0016] Among them, R n,j (t) in Equation (3) represents the reward value when node m n selects wireless channel j. If node m n successfully transmits a data packet on wireless channel j, then R n,j (t) = 1; otherwise, R n,j (t) = 0; represents the sample estimated value of the transmission success probability when node m n selects channel j before time slot t. in Equation (4) represents the number of times node m n has selected channel j before time slot t.

[0017] The second stage: When t ≥ N, each IoT node constructs a descending preference list for the wireless channels according to the preference estimated value. The formula for the preference estimated value is:

[0018]

[0019] In Equation (5), represents the preference estimated value of the node for the wireless channel. The IoT node selects the most preferred wireless channel to transmit a data packet to the central controller according to the preference list.

[0020] Furthermore, the central controller makes a matching decision based on the AoI reduction amount of the conflicting IoT nodes and assigns the wireless channel to the IoT node with the largest AoI reduction amount. The specific steps are as follows:

[0021] S1. First, determine the preference list of IoT nodes for wireless channels: The IoT nodes calculate the preference estimation values of each channel according to Equation (5) respectively Arrange the preference estimation values in descending order to obtain the preference list of wireless channels;

[0022] S2. Determine the matching result between the wireless channels and the IoT nodes, which specifically includes the following steps:

[0023] Step S201. The IoT nodes send transmission requests to the wireless channels in the order of the preference list;

[0024] Step S202. If a wireless channel is selected for transmission by only one IoT node, they are directly matched; otherwise, the wireless channel will be put into the Ω set and go to Step S203;

[0025] Step S203. If Ω is not an empty set, the central controller directly calculates the AoI reduction amount D n (t) of the conflicting IoT nodes. The specific formula is:

[0026]

[0027] where D n (t) represents the AoI reduction amount of the IoT node in the current time slot, represents the AoI of the data packet to be transmitted by this IoT node;

[0028] Step S204. Match the wireless channel with the IoT node having the largest AoI reduction amount D n (t). The remaining IoT nodes go to Step S201 and continue to send transmission requests to other wireless channels in the order of the preference list;

[0029] Step S205. When all the wireless channels are matched with the IoT nodes, the matching is completed. At this time, the IoT nodes start to transmit data packets.

[0030] Furthermore, the specific calculation steps of Step S203 are as follows:

[0031] For the IoT node, it is known that since the data packet has not been transmitted to the central controller, the central controller does not know the value. In each time slot, the process of the node collecting and updating data packets follows a probability of λ nBernoulli distribution, and the central controller retains the time slot of the last successful packet transmission of the IoT node. Therefore, the central controller can estimate the expected value of the AoI of the packet to be transmitted by the IoT node, and further estimate the AoI reduction amount. The probability distribution model is as follows:

[0032]

[0033] where B n (t) represents the difference between the current time slot and the time slot when the last packet from IoT node m n was received.

[0034] Therefore, the expected value of the AoI value n of the packet to be transmitted by node m can be expressed as:

[0035]

[0036] By estimating the value, the central controller calculates the AoI reduction amount of the conflicting node according to Equation (6).

[0037] Furthermore, after each IoT node completes a transmission, it will update the reward value R n,j (t) of each wireless channel, the sample estimate value the number of selections The central controller updates the AoI value A n (t) of the system network.

[0038] Furthermore, the industrial IoT channel allocation method described above includes the following:

[0039] The IoT node is responsible for collecting packets in the system network, establishing a preference list for wireless channels, sending a transmission request to the wireless channel, and transmitting the packet to the central controller through the wireless channel;

[0040] The central controller allocates the wireless channel, receives the packet sent by the IoT node, and updates the age of information of the system network.

[0041] The present invention adopts the above technical solutions and has the following beneficial effects:

[0042] (1) Aiming at the problem that nodes in the industrial IoT optimized based on the age of information cannot obtain channel state information, the method of multi-armed bandit is considered to enable the IoT node to explore and learn the transmission success probability of wireless channels in the current network, greatly improving the probability of the node matching a wireless channel with good quality, and at the same time improving the freshness of network information.

[0043] (2) During the process of IoT node matching channels, there will be a conflict problem where multiple IoT nodes simultaneously select a wireless channel. The sensing node with the largest AoI reduction is assigned to this wireless channel to improve the average information freshness in the network. Description of the Drawings

[0044] Figure 1 This is the industrial IoT system model optimized based on the Age of Information of the present invention.

[0045] Figure 2 This is the AoI simulation comparison diagram of the present invention.

[0046] Figure 3 This is the simulation comparison diagram of the average AoI of the present invention with the increase in the number of channels. Detailed Embodiment

[0047] The technical solution of the invention will be described in detail below with reference to the drawings.

[0048] The industrial IoT channel allocation method for optimizing information freshness disclosed by the present invention makes decisions using the multi-armed bandit method when allocating wireless channels to nodes in a state where IoT nodes cannot obtain channel state information in the network. When a conflict occurs where a wireless channel is simultaneously selected by multiple IoT nodes, the central controller makes an allocation decision using the matching method with the largest Age of Information reduction. The industrial IoT therein includes a central controller, N IoT nodes, and J available wireless channels; as Figure 1 shown, with the central controller as the center, the IoT nodes are dispersed around the central controller.

[0049] The transmission process of the industrial IoT optimized based on the Age of Information can be divided into two stages. In the first stage, within the time slots 1 ≤ t ≤ N, through N rounds of time slots, it is ensured that each wireless channel is scheduled by all IoT nodes, and the AoI of the IoT nodes at the central controller end is updated according to Equation (1), and R n,j (t),

[0050]

[0051] R n,j (t) ~ Ber(θ j )(2),

[0052]

[0053] where, A n (t) represents the AoI value of node m n at the current time slot t, and u n (t) represents node mn The generation time slot for successfully transmitting data packets; R n,j (t) represents node m n The reward value for selecting wireless channel j. If node m n successfully transmits a data packet on wireless channel j, then R n,j (t) = 1; otherwise, R n,j (t) = 0; Represents the sample estimated value of the successful transmission probability of node m n selecting channel j before time slot t; Represents the number of times node m n has selected channel j before time slot t.

[0054] In the second - stage time slots t ≥ N, each IoT node constructs a descending - order preference list for wireless channels according to the preference estimation value. The formula for the preference estimation value is:

[0055]

[0056] In Equation (5), represents the preference estimation value of the IoT node for the wireless channel. The IoT node selects the most preferred wireless channel to transmit data packets to the central controller according to the preference list.

[0057] When a wireless channel is selected by multiple IoT nodes and a conflict occurs, the central controller makes a matching decision based on the AoI reduction amount of the conflicting IoT nodes, and allocates this wireless channel to the IoT node with the largest AoI reduction amount. The AoI reduction amount of IoT node m n can be expressed as:

[0058]

[0059] In Equation (6), D n (t) represents the AoI reduction amount of IoT node m n in the current time slot, represents the AoI of the data packet to be transmitted by node m n this time. The central controller estimates the expectation of the AoI of the data packet to be transmitted by the node according to the probability distribution, and then estimates the AoI reduction amount according to Equation (6). The expected value of the AoI value n of the data packet to be transmitted by node m can be expressed as:

[0060]

[0061] To minimize the average age of information of the system network, the present invention uses the central controller to make a matching decision based on the AoI reduction amount of the conflicting IoT nodes. The specific steps are as follows:

[0062] S1. First, determine the preference list of the Internet of Things (IoT) nodes for wireless channels: The IoT nodes calculate the preference estimation value of each channel according to Equation (5) respectively Arrange the preference estimation values in descending order to obtain the preference list of wireless channels;

[0063] S2. Determine the matching result between the wireless channels and the IoT nodes, which specifically includes the following steps:

[0064] Step S201. The IoT nodes send transmission requests to the wireless channels in the order of the preference list;

[0065] Step S202. If a wireless channel is selected for transmission by only one IoT node, direct matching is performed between the two; otherwise, the wireless channel will be put into the Ω set and go to Step S203;

[0066] Step S203. If Ω is not an empty set, the central controller calculates the AoI reduction amount D of the conflicting nodes according to Equation (6) n (t);

[0067] Step S204. Match the wireless channel with the IoT node having the maximum AoI reduction amount D n (t), and the remaining IoT nodes go to Step S201 and continue to send transmission requests to other wireless channels in the order of the preference list;

[0068] Step S205. When all wireless channels are matched with the IoT nodes, the matching is completed, and at this time, the IoT nodes start to transmit data packets.

[0069] In this embodiment, the simulation conditions are as follows: The number of IoT nodes is N = 10, the number of wireless channels varies in the interval [3 - 9], the data generation probability λ of the IoT nodes n = 0.5, the transmission success probability of the wireless channels varies in the interval [0.1 - 0.75], and the number of simulation iterations: 1000.

[0070] When the IoT nodes cannot obtain the channel state information, the industrial IoT channel allocation method optimized for information freshness, the random matching method, the UCB algorithm, and the method of directly matching with known channel state information of the present invention are respectively used to compare the AoI in the system network under the same number of IoT nodes and channels. The simulation results are obtained as Figure 2 shown. At about 600 time slots, through exploration and learning, the AoI of the system network can be reduced to the AoI of the network with known channel state information. At the same time, the AoI of the system network in the method of the present invention is significantly lower than the AoI of the system network in the random matching method and the UCB method, which indicates that the method of the present invention can obtain a higher information freshness in an environment where the channel state information is unknown.

[0071] When the wireless channel is selected by multiple Internet of Things nodes simultaneously and conflicts occur, under the condition of the same number of Internet of Things nodes, the method of the present invention, the random matching method, the UCB algorithm, and the method of directly matching with known channel state information are respectively used. Under different numbers of channels, the average AoI of the system network is compared. The simulation results shown as Figure 3 below are obtained. As the number of wireless channels increases, the average AoI in the system network decreases. This shows that the increase in wireless channels improves the information freshness of the system network. The average AoI in the system network of the method of the present invention is very close to the average AoI of the method with known channel state information, and at the same time is significantly lower than the average AoI in the system network of the random matching method and the UCB method. This shows that the method of the present invention can obtain a higher information freshness in an environment where the channel state information is unknown.

[0072] The above embodiments are only used to illustrate the technical idea of the present invention, and the protection scope of the present invention cannot be limited thereby. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. Industrial Internet of Things channel allocation method optimized for information freshness. The Industrial Internet of Things includes a central controller, several Internet of Things nodes, and several available wireless channels. Centered around the central controller, the Internet of Things nodes are dispersed around the central controller. The Internet of Things nodes are responsible for collecting data packets in the system network, establishing a preference list for wireless channels, sending transmission requests to the wireless channels, and transmitting the data packets to the central controller through the wireless channels. The central controller allocates the wireless channels, receives the data packets sent by the Internet of Things nodes, and updates the age of information of the system network. It is characterized in that When allocating wireless channels to IoT nodes in a state where the IoT nodes cannot obtain channel state information in the network, the IoT nodes use the multi-armed bandit method for decision-making; when a wireless channel is selected by multiple nodes simultaneously and a conflict occurs, the central controller uses the matching method of the maximum age-of-information reduction to make an allocation decision, and allocates the wireless channel to the IoT node with the largest AoI reduction, where the IoT node m n 's AoI reduction is expressed as: Among them, D n (t) represents the reduction in the AoI of the IoT node m at the current time slot n . represents the AoI of the data packet to be transmitted by node m this time, A n (t) represents the AoI value of node m n at time slot t, expressed as: n u (t) represents the generation time slot of the successfully transmitted data packet of node m n . n ​ 2. The industrial Internet of Things channel allocation method according to claim 1, characterized in that In the Internet of Things network, the method for minimizing the age of information at the central controller end is as follows: Among which A avg is the average age of information of the system network, T is the number of time slots, N is the number of nodes, and m n represents an Internet of Things node, j represents a wireless channel, and the binary variable d n,j,t = 1 indicates that node m n selects the wireless channel j to transmit data packets in the current time slot t, otherwise d n,j,t = 0.

3. The industrial Internet of Things channel allocation method according to claim 1, characterized in that When using the multi-armed bandit method for decision-making, each Internet of Things sensor node needs to construct a descending-order preference list for the wireless channels according to the preference estimation value. The formula for the preference estimation value is: Among them, represents the preference estimation value of the node for the wireless channel, represents the sample estimation value of the successful transmission probability of node m n selecting channel j before time slot t, represents the number of times node m n selects channel j before time slot t.

4. The industrial Internet of Things channel allocation method according to claim 3, wherein The central controller end makes a matching decision based on the AoI reduction amount of the conflicting Internet of Things nodes. The specific steps are as follows: S1. First, determine the preference list of nodes for wireless channels: Each node calculates the preference estimate value of each channel according to Equation (2). Arrange the preference estimate values in descending order to obtain the preference list of wireless channels; S2. Determine the matching result between the wireless channel and the Internet of Things node, which specifically includes the following steps: Step S201. The Internet of Things node sends a transmission request to the wireless channel in the order of the preference list. Step S202. If the wireless channel is selected for transmission by only one node, a direct match is made between the two; otherwise, the wireless channel will be put into the Ω set and go to step S203. Step S203: If Ω is not an empty set, the central controller directly calculates the AoI reduction amount D of the conflicting IoT nodes n (t); Step S204: Match this wireless channel with the IoT node having the maximum AoI reduction amount D n (t), and transfer the remaining nodes to step S201 to continue sending transmission requests to other wireless channels in the order of the preference list; Step S205. When all the wireless channels are matched with the Internet of Things nodes, the matching is completed, and at this time, the Internet of Things node starts to transmit the data packet.

5. The industrial Internet of Things channel allocation method according to claim 4, wherein The specific calculation steps in step S203 are as follows: Suppose that in each time slot, the process of the Internet of Things node collecting updated data packets follows a Bernoulli distribution with a probability of λ n and the central controller retains the time slot of the last successful data packet transmission of the Internet of Things node. Therefore, the central controller can estimate the expected value of the AoI of the data packet to be transmitted by the Internet of Things node, and further estimate the reduction amount of AoI; where the probability distribution model is as follows: Among them, B n (t) represents the difference between the current time slot and the time slot when the last data packet received from the Internet of Things node m n was generated; Therefore, node m n The AoI value of the data packet to be transmitted is expected to be expressed as: By estimating the value, the central controller calculates the AoI reduction of the conflicting IoT nodes according to Equation (3).

6. The industrial Internet of Things channel allocation method according to claim 4, characterized in that After each IoT node completes transmission, it will update the reward value R of each wireless channel separately n,j (t), the sample estimate Number of selections The central controller updates the AoI value A of the system network n (t).

Images