Low-Latency Anti-Interference Video Transmission Method for Mobile Internet of Things Based on Reinforcement Learning

Abstract

A method for low-latency and anti-interference video transmission in mobile Internet of Things based on reinforcement learning belongs to the fields of wireless communication, Internet of Things and information security. Aiming at the device mobility and the demand for low video latency, a method for low-latency and anti-interference video transmission in mobile Internet of Things based on reinforcement learning is provided, which can effectively reduce the communication latency and packet loss rate of wireless video transmission of mobile devices, improve the network throughput and save communication energy consumption. By using channel, network state and mobile device location information, as well as communication performance parameters, a reinforcement learning algorithm is adopted to dynamically optimize the mobile trajectory of the device, the transmission power of wireless video transmission, the channel and the code rate, so as to cope with the dynamic wireless environment and hostile interference attacks during the movement and improve the video quality and fluency. It can effectively reduce the communication latency and packet loss rate of wireless video transmission, improve the network throughput and save communication energy consumption.

Description

Technical Field

[0001] The present invention belongs to the fields of wireless communication, Internet of Things, and information security, and specifically relates to a low-latency anti-interference video transmission method for mobile Internet of Things based on reinforcement learning. Background Technique

[0002] Wireless video transmission in the mobile Internet of Things supports applications such as intelligent inspection in the power Internet of Things system, environmental monitoring in smart cities, and cloud video monitoring. It has the advantages of flexible deployment and easy function expansion. However, the wireless transmission environment is complex and changeable. The frequent movement of Internet of Things devices leads to changes in network topology and channel conditions. It also faces factors such as environmental noise, co-channel interference, and even hostile interference signals, resulting in a decline in communication performance. Moreover, especially under the transmission conditions where the wireless channel bandwidth resources are limited and the interference signal intensity is high, the clarity and smoothness of video data are difficult to meet the service quality requirements. Therefore, designing a low-latency code rate adaptive video transmission system with wireless communication anti-interference ability to ensure video service quality is of great significance.

[0003] Chinese Patent Application CN114172874A proposes a video code rate adaptive method based on bandwidth information. It estimates the transmission delay of the buffer queue in the video stream according to the uplink bandwidth and code rate, and dynamically adjusts it jointly with the code rate threshold to achieve low-latency video stream transmission of the system, but it is difficult to cope with hostile interference attacks. Yilin Xiao et al. [Yilin Xiao; Liang Xiao; Zefang Lv; Guohang Niu; Yuzhen Ding; Wenyuan Xu, Learning based low-latency vIoT video streaming against jamming and interference, IEEE Wireless Comm. Magazine, vol. 28, no. 4, pp. 12 - 18, Sep. 2021] propose a reinforcement learning-based video stream encoding and signal modulation scheme for drones. By receiving the signal strength and video PSNR, it adjusts the video code rate and modulation method to improve the peak signal-to-noise ratio of the video stream and reduce the transmission delay at the same time, but it does not consider the direct relationship between the coordinates and road topology of Internet of Things devices and the channel state in the mobile scenario.

[0004] Power control and channel selection can cope with the dynamics of the wireless environment and various types of interference. A. Gouissem et al. [A. Gouissem, K. Abualsaud, E. Yaacoub, T. Khattab and M. Guizani, "Game Theory for Anti-Jamming Strategy in Multichannel Slow Fading IoT Networks," in IEEE Internet of Things Journal, vol. 8, no. 23, pp. 16880-16893, Dec. 1, 2021] proposed an anti-jamming power allocation scheme for IoT devices based on the Colonel Blotto game, which dynamically adjusts the transmit power according to the slow fading effect of the channel and the interference power level, reducing the energy consumption of communication devices while ensuring communication quality, but fails to take into account the optimization of video transmission performance. Chinese patent application CN112291495A proposed a wireless transmission method based on channel state and video quality, which combines the AC algorithm and the channel estimation algorithm based on pilot information to optimize the video bit rate, transmit power and channel, effectively reducing the bit error rate of data transmission. However, in the mobile IoT scenario, hostile interference and the device's travel trajectory will also affect the channel quality.

[0005] Therefore, the present invention considers the relationship between the real-time position information of the node and the channel state, and focuses on solving the problem of poor wireless video transmission performance caused by strong interference and fast time-varying channels in the mobile IoT environment. It uses reinforcement learning to jointly optimize power control, bit rate selection, channel switching and trajectory control to improve the anti-jamming ability and video transmission performance of the communication system. Summary of the Invention

[0006] The object of the present invention is to provide a low-latency anti-jamming video transmission method for mobile IoT based on reinforcement learning, which can effectively reduce the communication latency and packet loss rate of wireless video transmission of mobile devices, improve network throughput, and save communication energy consumption, aiming at the device mobility and low video latency requirements. Using channel, network state and mobile device location information, as well as communication performance parameters, the reinforcement learning algorithm is used to dynamically optimize the mobile trajectory of the device, the transmit power, channel and bit rate of wireless video transmission, to cope with the dynamics of the wireless environment and hostile interference attacks during movement, and to improve video quality and smoothness.

[0007] The present invention includes the following steps:

[0008] Step 1: The available transmission channels for wireless video of the IoT mobile device are c ∈ {1, 2,..., Ω}, where Ω represents the total number of available channels; the video bit rate is μ ∈ {iμ max / M|1 ≤ i ≤ M}, where μ maxis the maximum code rate, uniformly quantized into M levels; the transmission power is p ∈ {ip max / N|1 ≤ i ≤ N}, where p max is the maximum transmission power, quantized into N levels; the movable distances along the X-axis and Y-axis each time are denoted as d x , d y ∈ {id max / W|-W ≤ i ≤ W}, where d max is the maximum moving distance, quantized into 2W + 1 levels; the transmission interval is T seconds, μ (0) = μ max , p (0) = p max ,

[0009] Step 2: Construct neural network A and network B, and denote their network parameters as θ Α and θ B ; Initialize the delay threshold as τ max , packet loss rate threshold ρ max , transmission delay τ (0) , packet loss rate η (0) , network throughput t (0) , location and network training parameters γ, ε, where γ, ε ∈ (0, 1);

[0010] Step 3: At the k-th time slot, read the received signal strength r (k) , calculate the location

[0011] Step 4: Construct the state s (k) = [r (k) , t (k-1) , τ (k-1) , η (k-1) , l (k) and input it into neural network A to obtain the estimated value Q(s (k) , a (k) ); With a probability of 1 - ε, select the transmission strategy with the maximum Q i value; With a probability of ε, randomly select any transmission strategy;

[0012] Step 5: The IoT mobile device switches the transmission channel to c (k) , moves a distance along the X-axis and a distance along the Y-axis, performs video encoding at the code rate μ (k) , and sends the video data at the transmission power p (k) ; Obtain the transmission delay τ (k) , packet loss rate η(k) and throughput t (k) ;

[0013] Step 6: Calculate where I(·) is an indicator function: taking the value of 1 when the variable is true and 0 otherwise; w, ν, and are weight parameters;

[0014] Step 7: Store the experience e (k) ={s (k) , a (k) , u (k)} into the cache pool H. After Z time slots, randomly take out a historical experience e (i) from the cache pool H, and input the state vector s (i) into the neural network B, and denote the output as Q B (s (i) , a'; θ B );

[0015] Step 8: Update the weight parameters of the neural network A using the stochastic gradient descent algorithm That is:

[0016]

[0017] And every L time slots, let

[0018] Step 9: Repeat Steps 3 - 8 until the communication delay is less than τ max , and the packet loss rate is lower than ρ max .

[0019] Compared with the prior art, the present invention takes into account the device mobility and the low - latency requirements of video, and proposes a low - latency anti - interference video transmission method for mobile Internet of Things based on reinforcement learning. Using channel, network state, and mobile device location information, as well as communication performance parameters, the reinforcement learning algorithm is used to dynamically optimize the device movement trajectory, the transmission power of wireless video, the channel, and the code rate to cope with the dynamic wireless environment and hostile interference attacks during the movement, improving the video quality and smoothness. The present invention effectively reduces the communication delay and packet loss rate of wireless video transmission of mobile devices, improves the network throughput, and saves communication energy consumption. Brief Description of the Drawings

[0020] Figure 1 This is the communication delay analysis of the system of the present invention.

[0021] Figure 2 This is the packet loss rate analysis of the system of the present invention. Detailed Embodiment

[0022] The following embodiments will further illustrate the present invention in conjunction with the drawings.

[0023] The embodiments of the present invention include the following steps:

[0024] Step 1: For the wireless video available transmission channel c of the Internet of Things mobile device, c ∈ {1, 2,..., 13}, where Ω = 13 represents the total number of available channels; the video bit rate is μ ∈ [100, 200, 300, 400] Kbps, where μ max = 400 Kbps is the maximum bit rate, uniformly quantized into M = 4 levels; the transmit power is p ∈ [10, 14, 18, 22] dBm, where p max = 22 dBm is the maximum transmit power, uniformly quantized into N = 4 levels; the movable distances along the X-axis and Y-axis each time are denoted as d x , d y ∈ [-5, 0, 5] m, where d max = 5 m is the maximum movable distance, quantized into 3 levels; the transmission interval is T = 2 s, μ (0) = μ max , p (0) = p max ,

[0025] Step 2: Construct neural network A and network B, each consisting of 3 fully connected layers, with their initial network parameters being θ Α = θ B = 0. Network A and B both contain 6 input neurons, 256 hidden neurons, and 144 output neurons; let the delay threshold be τ max = 0.2 s, the packet loss rate threshold ρ max = 0.01%, the initial transmission delay τ (0) , the packet loss rate η (0) , the network throughput t (0) , are all zero, and the position The network training parameters γ = 0.5, ε = 0.1.

[0026] Step 3: At the k-th time slot, read the received signal strength r (k) , and calculate the position

[0027] Step 4: Construct s (k) = [r (k) , t (k-1) , τ (k-1) , η (k-1) , l (k) and input it into neural network A to obtain the estimated value Q(s (k) , a (k) ); select the transmission strategy with the maximum Q i value with a probability of 1 - ε Randomly select any transmission strategy with probability ε.

[0028] Step 5: The IoT mobile device switches the transmission channel to c (k) and moves along the X-axis Moves along the Y-axis At code rate μ (k) Video coding, with transmission power p (k) Send video data; obtain the transmission delay τ according to the feedback result at the receiving end (k) , packet loss rate η (k) and throughput t (k) .

[0029] Step 6: Calculate where I(·) is the indicator function: it takes the value 1 when the variable is true, otherwise 0. Let w = 10, ν = 0.05 and be the weight parameters.

[0030] Step 7: Store the experience e (k) ={s (k) ,a (k) ,u (k)} into the cache pool H. After Z = 10 time slots, randomly take out a historical experience e (i) from the cache pool H, input the state vector s (i) into the neural network B, and denote the output as Q B (s (i) ,a';θ B ).

[0031] Step 8: Update the weight parameters of the neural network A using the stochastic gradient descent algorithm That is:

[0032]

[0033] And every L = 100 time slots, let

[0034] Step 9: Repeat steps 3 - 8 until the communication delay is less than 50 ms and the packet loss rate is lower than 0.02%.

[0035] The present invention utilizes the location information of the mobile device, the channel state between the mobile device and the receiving end, and the communication performance parameters, and adopts a reinforcement learning algorithm to dynamically optimize the mobile device trajectory, transmission power, transmission channel, and video coding rate, so as to defend against wireless hostile interference attacks and cope with the changes in network topology and channel conditions caused by the movement of IoT devices, and improve the quality and smoothness of video data transmission. As Figure 1 and 2 shown, compared with the benchmark scheme, the system communication delay of the proposed scheme in the present invention is reduced by 48%, and the packet loss rate is reduced by 26%.

Claims

1. A method for low-latency anti-interference video transmission in a mobile Internet of Things based on reinforcement learning, characterized in that including the following steps: Step 1: The available transmission channels for the wireless video of the Internet of Things mobile device are \(c\in\{1,2,\cdots,\Omega\}\), where \(\Omega\) represents the total number of available channels; the video bit rate is \(\mu\in\{i\mu\) max / M|1\leq i\leq M\}\), where \(\mu\) max is the maximum bit rate and is uniformly quantized into \(M\) levels; The transmission power is p ∈ {ip max / N|1 ≤ i ≤ N}, where p max is the maximum transmission power, quantized into N levels; the movable distances along the X-axis and Y-axis each time are denoted as d x , d y ∈ {id max / W|-W ≤ i ≤ W}, where d max is the maximum movable distance, quantized into 2W + 1 levels; the video stream data transmission interval is T seconds, and let the initial bit rate μ (0) = μ max , the initial transmission power p (0) = p max , and the movable distances and Step 2: Construct neural networks A and B, and denote their network parameters as θ Α and θ B ; Initialize the delay threshold as τ max , packet loss rate threshold ρ max , transmission delay τ (0) , packet loss rate η (0) , network throughput t (0) , position and discount rate γ and exploration rate ε, where γ, ε ∈ (0, 1); Step 3: At the k-th time slot, read the received signal strength r (k) , and calculate the current position Step 4: Construct state s (k) = [r (k) , t (k-1) , τ (k-1) , η (k-1) , l (k) Input into neural network A to obtain the estimated value Q of the state-action pair A (s (k) , a (k) ); Select the transmission strategy with the maximum Q i value with a probability of 1 - ε Randomly select any transmission strategy with a probability of ε; Step 5: The IoT mobile device switches the transmission channel to c (k) , move along the X-axis Move along the Y-axis At the code rate μ (k) Perform video encoding and send video data at the transmit power p (k) ; Obtain the transmission delay τ (k) , packet loss rate η (k) and throughput t (k) ; Step 6: Calculate where, I(·) is the indicator function: taking the value of 1 when the variable is true, otherwise 0; w, ν and are weight parameters; Step 7: Store the experience e (k) ={ (k) s, a (k) , u (k)} into the cache pool H. After Z time slots, randomly take out a historical experience e (i) from the cache pool H, and input the state vector s (i) into the neural network B, and denote the output as Q B (s (i) , a'; θ B ); Step 8: Update the weight parameters of neural network A using the stochastic gradient descent algorithm That is: And every L time slots apart, let Step 9: Repeat steps 3 to 8 until the communication delay is less than τ max , and the packet loss rate is lower than ρ max .

Images