Reliability metric oriented multi-dimensional resource allocation method for low power internet of things

Abstract

The application discloses a low-power Internet of Things multi-dimensional resource allocation method for reliability measurement, which comprises the following steps: S1, establishing a system model of the low-power Internet of Things; S2, establishing a reliability constraint model in the low-power Internet of Things model; S3, setting constraint conditions in the optimization problem and the reliability constraint model; S4, establishing a deep reinforcement learning network; S5, importing the system model of the low-power Internet of Things into the deep reinforcement learning network; S6, randomly selecting at least one transmission action in the low-power Internet of Things model of S5; S7, judging whether the data transmission action satisfies the constraint condition and performing rewards and punishments; S8, saving each item of information of the transmission action as a sample and storing the sample into an experience replay pool of the deep reinforcement learning network; S9, repeating steps S6 to S8 at least once; S10, calculating a target value by randomly selecting a small sample in the experience replay pool to update network parameters; and S11, repeating S4 to S10 to obtain a stable transmission power and channel quantity allocation scheme.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of Internet of Things, and particularly relates to a multi-dimensional resource allocation method for low-power Internet of Things oriented to reliability measurement. BACKGROUND

[0002] With the increasing demand for the number of sensing devices in the actual application deployment of low-power Internet of Things, research on wireless resource allocation, such as power, time domain, frequency domain, etc., has attracted attention. At the same time, many rescue and early warning related application scenarios in Internet of Things pay great attention to the timeliness of data collection, that is, the deadline of data. Therefore, completing real-time monitoring of environmental or target data and successfully transmitting within a limited time is the key to avoiding irreparable losses caused by data loss and high latency.

[0003] The reliability of the network in low-power Internet of Things still needs to be focused on.

[0004] When performing resource allocation, there is a certain conflict between the reliability of data transmission in the wireless link and the performance of low power consumption. Considering the attenuation of energy in the wireless link, too low transmit power of the terminal device may cause the receiving node to fail to parse the signal, thereby causing data loss, but too high transmit power cannot guarantee the low power consumption requirement of the device. Even if some communication protocols use retransmission mechanism to give the unsuccessfully transmitted data a second chance, multiple retransmissions will inevitably increase the additional energy consumption, produce additional latency, and reduce the life cycle of the device. Therefore, how to reasonably and efficiently allocate resources so that the terminal device maintains low power consumption performance while still completing reliable data transmission before the deadline is a very important research problem in low-power Internet of Things.

[0005] In the existing research on resource allocation of low-power Internet of Things, for tasks with a deadline and guaranteed data transmission reliability, the commonly used time division multiple access technology often cannot establish a scheduling that meets the requirements. Because in the premise condition setting of time slot allocation, it is usually assumed that the data in a single time slot is successfully transmitted, without considering the data packet loss caused by wireless link loss. At the same time, a long time is wasted on waiting for time slots, which is not conducive to the guarantee of transmission within the deadline. SUMMARY

[0006] To solve the problems raised in the background art, the application provides a joint optimization method of channel allocation and power control for minimizing the total energy consumption of terminal devices in Internet of Things under the dual constraints of data transmission reliability and deadline.

[0007] To achieve the above object, the application provides the following technical scheme:

[0008] The multi-dimensional resource allocation method for low-power Internet of Things oriented to reliability measurement comprises:

[0009] S1. Establish a system model for low-power Internet of Things (IoT);

[0010] S2. Establish a reliability constraint model in the low-power IoT model;

[0011] S3. Set up the optimization problem of minimizing the total power consumption of terminal devices, and various constraints in the reliability constraint model;

[0012] S4. Establish a deep reinforcement learning network and initialize the parameters of the initial network and the target network;

[0013] S5. Import the low-power IoT system model into the deep reinforcement learning network and initialize the scene state of the low-power IoT system model.

[0014] S6. Randomly select at least one transmission action in the low-power IoT model of step S5.

[0015] S7. Determine whether the transmission time and reliability of the data transmission action meet the constraints. If yes, reward; if no, punish.

[0016] S8. Save the transmission action, reward, and current and next time-step state as a sample and store it in the experience replay pool of the deep reinforcement learning network;

[0017] S9. Repeat steps S6 to S8 at least once;

[0018] S10. In the experience replay pool of step S8, randomly select a small sample to calculate the target value, and update the network parameters through the loss function.

[0019] S11. Repeat steps S4 to S10 a fixed number of times.

[0020] Preferably, the system model in step S1 includes:

[0021] Suppose there are N end devices, and the set of end nodes is denoted as I = {1,...,N};

[0022] Assume the convergence node in 3D space is fixed at (X,Y,Z) and cannot be moved, and the position of end device i is...

[0023] Transmission distance from terminal device i to the aggregation node

[0024] There are two uplink transmission modes for end devices: one is that the aggregation node provides feedback and acknowledgment, in which the end device retransmits data packets a limited number of times until it receives an acknowledgment character from the aggregation node; the other is that data packets are sent only once without feedback and acknowledgment.

[0025] S1.1: Establish a communication model;

[0026] Define B as the total allocatable communication bandwidth. b is the bandwidth of each sub-channel i If the number of communication channels allocated to end device i is determined, then the maximum uplink transmission rate r from end device i to the aggregation node is determined. i for

[0027] Where δ 2 Indicates the noise power of the terminal device. h represents the transmit power of terminal device i. i Channel gain;

[0028] In suburban scenarios, the channel gain h i Expressed in dB

[0029] Where G[dB] is the path loss at the reference distance d0, ε is the path loss exponent, which is related to the propagation environment. In suburban environments, the path loss is approximately ε = 3.7, and X0 is the shadow fading correction exponent, which is a zero-mean Gaussian random variable with a standard deviation of σ0, following a normal distribution X0~N(0,σ0). 2 );

[0030] S1.2: Establish a task model;

[0031] Suppose Within a given timeframe, if terminal device i needs to complete the transmission of all data packets generated by the perceived event, then...

[0032] Where t i β1 t represents the time it takes for terminal device i to successfully send data that does not require feedback. i β2 This indicates the time t is required for terminal device i to successfully send the data that needs to be returned. i β1 =(1-ω)M i ·L i / r i , t i β2 = (1+K)·ωM i ·L i / r i , where M i L is the total number of data packets that end device i needs to transmit. iWhere is the length of the data packet, K is the number of retransmissions, and ω is the feedback coefficient of the end device sending the data packet, which is the proportion of the data packets that need to be acknowledged and retransmitted to the total number of data packets sent.

[0033] S1.3: Establish an energy consumption model;

[0034] Terminal device i generates M within time t. i The energy consumption for successfully transmitting data packets is E. i (t)=E i α (t)+E i β (t);

[0035] Where E i α (t) represents the energy consumption of end node i for data sensing, E i β (t) represents the energy consumption of end node i when transmitting data;

[0036] Based on the characteristics of the periodic sensing data of low-power IoT mid-range devices, η is defined as the ratio of the duration of sensing events by the mid-range device to the total duration.

[0037] in Power consumption of the terminal device's sensing module;

[0038] The energy consumption of end node i transmitting data is expressed as:

[0039] in The operating power of the communication module used. For the transmit power, the total operating power consumption of the sensing end node is

[0040] Preferably, in step S2, the reliability constraint model includes:

[0041] Signal-to-noise ratio (SIN) i Exceeding a certain threshold γ t The signal-to-noise ratio is expressed as

[0042] Define the probability of successful signal transmission from terminal device i as follows: This can then be expressed in terms of probability as

[0043] In step S1.1, h i Substituting the expression and expressing it in dB, we get:

[0044] make Since X0~N(0,σ0) 2 Then we can further obtain

[0045] Where Φ(x) is the cumulative distribution function of the standard normal distribution, defined as

[0046] Since the data sent by the end node is divided into two transmission modes: one that supports retransmission and the other that sends data only once, the probability Φ of all data successfully transmitted by end node i is... i For Φ i =ω·(1-(1-Φ(φ) n )) K )+(1-ω)·Φ(φ n ).

[0047] Preferably, in step S3, the set of the number of channel allocations for the N end devices is set as B = {b i The set of transmit powers of terminal device i, where i ∈ N.

[0048] An optimization problem is proposed to minimize the total energy consumption of IoT mid-level devices while satisfying deadline and data transmission reliability constraints. The optimization problem is as follows:

[0049] Optimization issues:

[0050] Constraints:

[0051] C2:ω·(1-(1-Φ(φ n )) K )+(1-ω)·Φ(φ n )≥Φ min

[0052]

[0053] C4:0≤E i ≤E imax ;

[0054] Where C1 guarantees that the total number of channel allocations will not exceed the total number of allocatable channels, and Φ in C2 min Given a threshold for data transmission success rate, serving as a data transmission reliability constraint, C3 is the data transmission deadline constraint, and C4 is the node battery capacity constraint, with a numerical upper bound E. imax .

[0055] Preferably, in step S4, the experience replay pool D, the number of time slots, and the total power of the device in the reinforcement learning network are initialized. Initialize the estimated Q-network parameters θ and the target Q-network parameters in the deep reinforcement learning network. Make Determine the discount factor γ, ∈ Greedy exploration probability, and update the step size.

[0056] Preferably, in step S6, an action a, where a ∈ A, is randomly selected based on probability ∈ A, and A is the set of all valid actions; otherwise, an action a is selected based on probability ∈ A. i =π(a i |s i Select the action, where s i For the current state of a low-power IoT scenario, π(a i |s i ) refers to s i The action with the highest value among the existing strategy choices.

[0057] Preferably, in step S7, if the data transmission time or data transmission rate of the terminal device does not meet the constraints, the reward is calculated as follows: r(s) i ,a i )=r(s i ,a i )-p, where p is the penalty.

[0058] Preferably, in step S8, behavior a i Rewards r i Current s i And the state s at the next moment i+1 Save as a sample (s) i ,a i ,r i ,s i+1 ), and store them in the experience replay pool D.

[0059] Preferably, in step S10, a batch of samples (s) is randomly selected from the M samples in the experience replay pool D. i ,a i ,r i ,s i+1 For each of the extracted samples, the target value is calculated. Then through the loss function The parameters θ of the estimated Q-network are updated, and after F iterations, the parameters θ in the estimated Q-network are then passed to the target Q-network. Used for updates.

[0060] The beneficial effects of this invention are as follows:

[0061] 1. In the optimization problem of this invention, a transmission reliability index is introduced to consider the data packet loss caused by wireless link loss, so as to ensure the reliability constraint of data transmission while allocating resources.

[0062] 2. The optimization method based on deep reinforcement learning proposed in this invention optimizes the channel allocation of the end device while realizing real-time dynamic control of the end device's transmit power. It minimizes the total power consumption of the end device while ensuring that the task is completed on time and the data transmission is reliable. Attached Figure Description

[0063] Figure 1 This is a schematic diagram illustrating the main steps of the present invention;

[0064] Figure 2 This is a schematic diagram illustrating the main energy consumption of the low-power IoT terminal node of the present invention;

[0065] Figure 3 This is a model diagram of the low-power Internet of Things system of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] To facilitate understanding of the technical content of this invention by those skilled in the art, the invention will be further described below with reference to the accompanying drawings.

[0068] Example 1

[0069] This invention provides a joint optimization method for channel allocation and power control that minimizes the total energy consumption of IoT mid-level devices under the dual constraints of data transmission reliability and deadline. It consists of three main parts: system model establishment, problem model transformation, and method solution. Figure 1 As shown, the specific steps include:

[0070] S1: Establish a system model for low-power IoT, specifically:

[0071] like Figure 1 As shown, there are N end devices, and the set of end nodes can be denoted as I = {1,...,N}. The positions of the sink node and end devices in three-dimensional space are described. The sink node is fixed at (X,Y,Z) and cannot be moved. The position of end device i is given as (x,Y,Z). i ,y i ,z i ), The transmission distance from terminal device i to the aggregation node

[0072] There are two uplink transmission modes for end devices. One mode requires feedback from the aggregation node. If the end device does not receive the acknowledgment character from the aggregation node, it will retransmit the data packet a limited number of times until it receives it. The other mode does not require feedback and the data packet is sent only once.

[0073] S1.1: Establish a communication model, defining B as the total allocatable communication bandwidth. b is the bandwidth of each sub-channel i If the number of communication channels allocated to end device i is determined, then the maximum uplink transmission rate r from end device i to the aggregation node is determined. i for Where δ 2 Indicates the noise power of the terminal device. h represents the transmit power of terminal device i. i Let h be the channel gain. Considering suburban scenarios where the wireless links between nodes are line-of-sight, and path loss is dominated by multipath and shadowing phenomena, the channel gain h can be... i Expressed in dB Where G[dB] is the path loss at the reference distance d0, ε is the path loss exponent, which is related to the propagation environment; in a suburban environment, the path loss ε = 3.7. X0 is the shadow fading correction exponent, a zero-mean Gaussian random variable with a standard deviation of σ0, following a normal distribution X0~N(0,σ0). 2 ).

[0074] S1.2: Establish a task model, assuming... Within a given timeframe, if terminal device i needs to complete the transmission of all data packets generated by the perceived event, then... Where t i β1 t represents the time it takes for terminal device i to successfully send data that does not require feedback. i β2 Let t be the time it takes for terminal device i to successfully send the data that requires feedback. i β1 =(1-ω)M i ·L i / r i , t i β2 = (1+K)·ωM i ·L i / r i , where M i L is the total number of data packets that end device i needs to transmit. i Let K be the length of the data packet, K be the number of retransmissions, and ω be the feedback coefficient of the end device sending the data packet, which is the proportion of data packets that need to be acknowledged and retransmitted out of the total number of data packets sent.

[0075] S1.3: Establish an energy consumption model, where terminal device i generates M within time t. i The energy consumption for successfully transmitting data packets is E. i (t)=E i α (t)+E i β (t), where E i α (t) represents the energy consumption of end node i for data sensing, E i β (t) represents the energy consumption for transmitting data to end node i. Based on the characteristics of the periodic sensing data operation mode of low-power IoT devices, η is defined as the ratio of the duration of sensing events to the total duration of the end device. in The operating power of the terminal device's sensing module. in For the operating power of the communication module, To achieve the required transmission power, transmitting data that needs feedback will trigger an additional communication module cycle to receive the acknowledgment character from the aggregation node. Therefore, the total operating energy consumption of the sensor nodes in this IoT system is...

[0076] S2: Introduce transmission reliability indicators and establish a reliability constraint model:

[0077] In a single communication where the uplink data is transmitted by the device i, to ensure accurate signal reception, the signal-to-noise ratio (SNR) of the received signal is typically required to be [value missing]. i Exceeding a certain threshold γ t The signal-to-noise ratio can be expressed as Define the probability of successful signal transmission from terminal device i as follows: This can then be expressed in terms of probability as h in S11 i Substituting the expression and expressing it in dB, we get: make Since X0~N(0,σ0) 2 Then we can further obtain Where Φ(x) is the cumulative distribution function of the standard normal distribution, defined as As shown in the above formula, the probability of successful signal transmission is related to the device's transmission power and the distance between nodes. With a fixed distance, the probability of successful signal transmission increases with the device's transmission power. Since the data transmitted by the end node has two transmission modes—one supporting retransmission and the other transmitting only once—the probability Φ of successful transmission of all data from end node i is...i For Φ i =ω·(1-(1-Φ(φ) n )) K )+(1-ω)·Φ(φ n ).

[0078] S3: Set up an optimization problem that minimizes the total power consumption of terminal devices, and define various constraints in the model:

[0079] Based on the above system model, the set of channel allocation numbers for N end devices is defined as B = {b}. i The set of transmit powers of terminal device i, where i ∈ N. An optimization problem is proposed to minimize the total energy consumption of IoT mid-level devices while satisfying deadline and data transmission reliability constraints. The optimization problem is as follows:

[0080] Optimization issues:

[0081] Constraints:

[0082] C2:ω·(1-(1-Φ(φ n )) K )+(1-ω)·Φ(φ n )≥Φ min

[0083]

[0084] C4:0≤E i ≤E imax

[0085] C1 guarantees that the total number of channel allocations will not exceed the total number of allocatable channels; Φ in C2 min C1 is a threshold for the given data transmission success rate, serving as a data transmission reliability constraint; C2 is a data transmission deadline constraint; C3 is a constraint on the node's battery capacity, with a numerical upper bound E. imax .

[0086] S4: Establish a deep reinforcement learning network and initialize the parameters of the initial network and the target network:

[0087] Initialize the experience playback pool D, the number of time slots N, and the total power of the device. Initialize deep reinforcement learning to estimate Q-network parameters θ and target Q-network parameters. Make Determine the discount factor γ, ∈ the greedy exploration probability, update the step size, etc.

[0088] S5: Initialize the state of the low-power IoT scenario in the deep reinforcement learning network to s1;

[0089] S6: Perform actions based on status and rewards;

[0090] Based on probability ∈, randomly select an action a∈A; otherwise, based on a i =π(a i |s i Select action, π(a) i |s i ) refers to state s i The action with the highest value among the existing strategy choices.

[0091] S7: Determine whether the data transmission time and reliability meet the constraints; if not, apply a penalty.

[0092] If the data transmission time or data transmission rate of the end device does not meet the constraints, the reward is calculated as follows: r(s) i ,a i )=r(s i ,a i )-p, where p is the penalty.

[0093] S8: Action a i Rewards r i Current s i And the state s at the next moment i+1 Save as a sample (s) i ,a i ,r i ,s i+1 ), and store it in the experience replay pool D;

[0094] S9: Repeat steps S6 to S8 a fixed number of times;

[0095] S10: In the experience replay pool D of S9, a small sample is randomly selected to calculate the target value, and the network parameters are updated through the loss function;

[0096] A batch of samples (s) is randomly selected from the M samples in the experience replay pool. i ,a i ,r i ,s i+1 For each of the extracted samples, the target value is calculated. Then through the loss function The parameters θ of the estimated Q-network are updated, and after F iterations, the parameters θ in the estimated Q-network are then passed to the target Q-network. Used for updates.

[0097] S11: Repeat steps S4 to S10 a fixed number of times to obtain a relatively stable transmission power and channel number allocation scheme.

[0098] Example 2

[0099] Figure 3 This is the low-power IoT system model of the present invention, such as Figure 3 As shown, the model consists of a base station and several end nodes, with a star topology. The base station acts as a aggregation node, and the end devices periodically collect data and transmit it directly to the aggregation node via a single hop. The end nodes in this network are battery-powered, and the remaining battery power varies among different end nodes. Each end device periodically broadcasts on the "feedback channel," providing real-time information about the network, including its location, remaining energy, channel status, and the amount of data being transmitted by the sensor nodes. The aggregation node simultaneously integrates network information, and based on the obtained information, makes appropriate transmission strategies regarding transmit power and channel allocation at the data processing center, and periodically interacts with the end devices.

[0100] Example 3

[0101] Figure 2 This is a schematic diagram illustrating the main energy consumption of the low-power IoT terminal node of this invention. (See diagram below.) Figure 2 As shown, the energy consumption of the sensing device includes the energy consumed in data sensing and generation, data processing, and data transmission and reception. Because the processor module of the sensing device has very low power consumption, it is usually not analyzed. The main components of the sensing device include a sensing module, a communication module, a power supply, and a signal transceiver module. The sensing module is used to sense and generate various data. After the sensing module detects information about the surrounding environment and generates data, the communication module starts working. The operating power of the sensing module of sensing device i is defined as... The operating power of the communication module is Transmission power is

[0102] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A low-power IoT multi-dimensional resource allocation method oriented towards reliability measurement, characterized in that, The method includes: S1: Establish a system model for low-power Internet of Things (IoT); S2: Establish a reliability constraint model in the low-power IoT model; S3: Set up the optimization problem of minimizing the total power consumption of terminal devices, and various constraints in the reliability constraint model; S4: Build a deep reinforcement learning network and initialize the parameters of the initial network and the target network; S5: Import the low-power IoT system model into the deep reinforcement learning network and initialize the scene state of the low-power IoT system model; S6: Randomly select at least one transmission action in the low-power IoT model of step S5; S7: Determine whether the transmission time and reliability of the data transmission action meet the constraints. If yes, reward; if no, punish. S8: Save the transmission action, reward, and current and next time-step state as a sample and store it in the experience replay pool of the deep reinforcement learning network; S9: Repeat steps S6 to S8 at least once; S10: In the experience replay pool of step S8, a small sample is randomly selected to calculate the target value, and the network parameters are updated through the loss function; S11: Repeat steps S4 to S10 at least once.

2. The low-power IoT multi-dimensional resource allocation method based on reliability measurement according to claim 1, characterized in that, The system model in step S1 includes: Suppose there are N end devices, and the set of end nodes is denoted as I = {1,...,N}; Assume the convergence node in 3D space is fixed at (X,Y,Z) and cannot be moved, and the position of end device i is... Transmission distance from terminal device i to the aggregation node There are two uplink transmission modes for end devices: one is that the aggregation node provides feedback and acknowledgment, in which the end device retransmits data packets a limited number of times until it receives an acknowledgment character from the aggregation node; the other is that data packets are sent only once without feedback and acknowledgment. S1.1: Establish a communication model; Define B as the total allocatable communication bandwidth. b is the bandwidth of each sub-channel i If the number of communication channels allocated to end device i is determined, then the maximum uplink transmission rate r from end device i to the aggregation node is determined. i for Where δ 2 Indicates the noise power of the terminal device. h represents the transmit power of terminal device i. i Channel gain; In suburban scenarios, the channel gain h i Expressed in dB Where G[dB] is the path loss at the reference distance d0, ε is the path loss exponent, which is related to the propagation environment. In suburban environments, the path loss is approximately ε = 3.7, and X0 is the shadow fading correction exponent, which is a zero-mean Gaussian random variable with a standard deviation of σ0, following a normal distribution X0~N(0,σ0). 2 ); S1.2: Establish a task model; Suppose Within a given timeframe, if terminal device i needs to complete the transmission of all data packets generated by the perceived event, then... in This indicates the time it takes for terminal device i to successfully send data that does not require feedback. This indicates the time required for terminal device i to successfully send the data and provide feedback. Where M i L is the total number of data packets that end device i needs to transmit. i Where is the length of the data packet, K is the number of retransmissions, and ω is the feedback coefficient of the end device sending the data packet, which is the proportion of the data packets that need to be acknowledged and retransmitted to the total number of data packets sent. S1.3: Establish an energy consumption model; Terminal device i generates M within time t. i The energy consumption for successfully transmitting data packets is in Energy consumption for data sensing at end node i. Energy consumption for transmitting data to end node i; Based on the characteristics of the periodic sensing data of low-power IoT mid-range devices, η is defined as the ratio of the duration of sensing events by the mid-range device to the total duration. in Power consumption of the terminal device's sensing module; The energy consumption of end node i transmitting data is expressed as: in The operating power of the communication module used. For the transmit power, the total operating power consumption of the sensing end node is 3. The low-power IoT multi-dimensional resource allocation method based on reliability measurement according to claim 2, characterized in that, In step S2, the reliability constraint model includes: Signal-to-noise ratio (SIN) i Exceeding a certain threshold γ t The signal-to-noise ratio is expressed as Define the probability of successful signal transmission from terminal device i as follows: This can then be expressed in terms of probability as In step S1.1, h i Substituting the expression and expressing it in dB, we get: make Since X0~N(0,σ0) 2 Then we can further obtain Where Φ(x) is the cumulative distribution function of the standard normal distribution, defined as Since the data sent by the end node is divided into two transmission modes: one that supports retransmission and the other that sends data only once, the probability Φ of all data successfully transmitted by end node i is... i For Φ i =ω·(1-(1-Φ(φ) n )) K )+(1-ω)·Φ(φ n ).

4. The low-power IoT multi-dimensional resource allocation method based on reliability measurement according to claim 3, characterized in that, In step S3, the set of the number of channel allocations for N end devices is set as B = {b i The set of transmit powers of terminal device i, where i ∈ N. An optimization problem is proposed to minimize the total energy consumption of IoT mid-level devices while satisfying deadline and data transmission reliability constraints. The optimization problem is as follows: Optimization issues: Constraints: C1: C2:ω·(1-(1-Φ(φ n )) K )+(1-ω)·Φ(φ n )≥Φ min C3: C4:0≤E i ≤E imax ; Where C1 guarantees that the total number of channel allocations will not exceed the total number of allocatable channels, and Φ in C2 min Given a threshold for data transmission success rate, serving as a data transmission reliability constraint, C3 is the data transmission deadline constraint, and C4 is the node battery capacity constraint, with a numerical upper bound E. imax .

5. The low-power IoT multi-dimensional resource allocation method based on reliability measurement according to claim 1, characterized in that, In step S4, the experience replay pool D, the number of time slots, and the total power of the device in the reinforcement learning network are initialized. Initialize the estimated Q-network parameters θ and the target Q-network parameters in the deep reinforcement learning network. Make Determine the discount factor γ, ∈ Greedy exploration probability, and update the step size.

6. The low-power IoT multi-dimensional resource allocation method based on reliability measurement according to claim 1, characterized in that, In step S6, an action a is randomly selected based on probability ∈ A, where A is the set of all valid actions; otherwise, action a is selected based on probability ∈ A. i =π(a i |s i Select the action, where s i For the current state of a low-power IoT scenario, π(a i |s i ) refers to s i The action with the highest value among the existing strategy choices.

7. The low-power IoT multi-dimensional resource allocation method based on reliability measurement according to claim 6, characterized in that, In step S7, if the data transmission time or data transmission rate of the end device does not meet the constraints, the reward is calculated as follows: r(s) i ,a i )=r(s i ,a i )-p, where p is the penalty.

8. The low-power IoT multi-dimensional resource allocation method based on reliability measurement according to claim 7, characterized in that, In step S8, behavior a i Rewards r i Current s i And the state s at the next moment i+1 Save as a sample (s) i ,a i ,r i ,s i+1 ), and store them in the experience replay pool D.

9. The low-power IoT multi-dimensional resource allocation method based on reliability measurement according to claim 8, characterized in that, In step S10, a batch of samples (s) is randomly selected from the M samples in the experience replay pool D. i ,a i ,r i ,s i+1 For each of the extracted samples, the target value is calculated. Then through the loss function The parameters θ of the estimated Q-network are updated, and after F iterations, the parameters θ in the estimated Q-network are then passed to the target Q-network. Used for updates.

Images