Monitoring video transmission method and system based on wireless communication

Abstract

The invention provides a monitoring video transmission method and system based on wireless communication, and relates to the technical field of monitoring video transmission, and the method comprises the steps: obtaining and preprocessing network state data, the network state data comprising bandwidth, time delay and packet loss rate; establishing a network state prediction model based on the LSTM neural network, and predicting a future network state according to the network state prediction model; based on the future network state, a multi-objective optimization algorithm is adopted to perform transmission path selection; and for the selected transmission path, video coding parameters are adjusted, the coding parameters comprise a code rate, a resolution ratio and a frame rate, and a self-adaptive transmission protocol is adopted to dynamically adjust the transmission rate according to a real-time network state. According to the method, different paths can be dynamically selected, network congestion or degradation is avoided, and the video transmission quality and rate are adaptively and flexibly adjusted according to the current network state, so that stable transmission of the video quality under different network conditions is ensured.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a monitoring video transmission method and system based on wireless communication. Background Art

[0002] In a complex and ever-changing network environment, ensuring the quality of video transmission and user experience is a major challenge. The fluctuation and instability of network status will lead to the decline of video transmission quality, and problems such as freezing and blurring will occur. At the same time, under different network conditions, the optimal video encoding parameters and transmission strategies are also different. How to accurately predict the changing trend of network status and dynamically optimize the transmission path, adjust the encoding parameters and transmission strategy accordingly is a key issue that needs to be solved urgently. Summary of the invention

[0003] In view of the problems existing in the above-mentioned existing monitoring video transmission method, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is how to provide a monitoring video transmission method based on wireless communication.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: obtaining network status data and performing preprocessing, wherein the network status data includes bandwidth, delay and packet loss rate; establishing a network status prediction model based on an LSTM neural network, and predicting the future network status according to the network status prediction model; based on the future network status, adopting a multi-objective optimization algorithm to select a transmission path, and adjusting video encoding parameters for the selected transmission path, wherein the encoding parameters include bit rate, resolution and frame rate; adopting an adaptive transmission protocol to dynamically adjust the transmission rate according to the real-time network status.

[0006] As a preferred solution of the monitoring video transmission method based on wireless communication described in the present invention, wherein: the network state prediction model is established based on the LSTM neural network, and the future network state is predicted according to the network state prediction model, including: establishing a network state prediction model based on LSTM, the network state prediction model includes an input layer, an LSTM layer and a fully connected layer; integrating the preprocessed network state data to form a network state vector; inputting the network state vector into the input layer of the LSTM basic model; adding one or more LSTM layers, each layer including a number of memory units; the output of the LSTM layer is mapped to the predicted future network state after passing through a fully connected layer.

[0007] As a preferred solution of the monitoring video transmission method based on wireless communication described in the present invention, wherein: based on the future network state, a multi-objective optimization algorithm is used to select the transmission path, including: minimizing delay, minimizing packet loss rate and maximizing bandwidth as optimization goals; based on the network topology distribution, a Dijkstra algorithm is used to find multiple feasible transmission paths, each of which is composed of a transmission node and a link; all feasible transmission paths are predicted by the network state prediction model to determine the future network state of each transmission path; the future network state of each transmission path is mapped to the objective function value, the quality of the path is evaluated by the NSGA-III algorithm, several transmission paths are randomly selected as the initial population, and the fitness of each transmission path is calculated according to the future network state; based on the fitness value, individuals are selected from the current population for crossover and mutation using the tournament selection method; when the preset number of iterations is reached, a transmission path is selected from the Pareto optimal solution set.

[0008] As a preferred solution of the monitoring video transmission method based on wireless communication described in the present invention, wherein: the video encoding parameters are adjusted for the selected transmission path, including: obtaining network status data of the selected transmission path; if the network delay exceeds a preset first threshold, the frame rate of the video encoding is reduced by a preset first proportion; if the network packet loss rate exceeds a preset second threshold, the bit rate of the video encoding is reduced by a preset second proportion; if the network available bandwidth is lower than the bit rate of the video encoding, the bit rate and resolution of the video encoding are simultaneously reduced by a preset third proportion; using a preset video encoding method, the video source data is encoded according to the adjusted encoding parameter values ​​to obtain encoded video data.

[0009] As a preferred solution of the monitoring video transmission method based on wireless communication described in the present invention, if it is detected that the network status data has changed and does not meet the preset video transmission quality requirements, then return to the step of adjusting the video encoding parameters, and re-judge and adjust the video encoding parameters according to the latest network status data.

[0010] As a preferred solution of the monitoring video transmission method based on wireless communication described in the present invention, the adaptive transmission protocol is used to dynamically adjust the transmission rate according to the real-time network status, including: obtaining the real-time network status of the selected transmission path; inputting the real-time network status into a pre-built random forest model to obtain the health score of the network; if the health score is lower than a preset health threshold, triggering the rate adjustment mechanism of the adaptive transmission protocol to adjust the transmission rate.

[0011] As a preferred solution of the monitoring video transmission method based on wireless communication described in the present invention, the rate adjustment mechanism of the adaptive transmission protocol includes taking the real-time network state as the state space, taking the transmission rate adjustment range as the action space, and determining the optimal transmission rate through the Q-Learning algorithm; inputting the historical network state parameters into the network state prediction model to predict the future network state within a period of time in the future; taking the prediction result as one of the states of the Q-Learning algorithm to adjust the transmission rate in advance.

[0012] In the second aspect, an embodiment of the present invention provides a monitoring video transmission method system based on wireless communication, the system including a data collection module for acquiring network status data and performing preprocessing; an LSTM network status prediction module for predicting the future network status according to the network status prediction model; a path selection module for selecting a transmission path according to the future network status; and a parameter adjustment module for adjusting video encoding parameters and transmission rate.

[0013] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the monitoring video transmission method based on wireless communication as described in the first aspect of the present invention is implemented.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the monitoring video transmission method based on wireless communication as described in the first aspect of the present invention is implemented. The technical solution provided by the embodiment of the present invention may include the following beneficial effects: by adopting a multi-objective optimization algorithm for transmission path selection, combined with the future network status predicted by the LSTM model, different paths can be dynamically selected to avoid network congestion or degradation, and the quality and rate of video transmission can be flexibly adjusted according to the current network status to ensure stable transmission of video quality under different network conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0016] Figure 1 The present invention is a flow chart of a monitoring video transmission method based on wireless communication. DETAILED DESCRIPTION

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0018] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0020] like Figure 1 , this embodiment provides a monitoring video transmission method based on wireless communication, which includes the following steps: Step S1, obtaining network status data and performing preprocessing.

[0021] Specifically, the network interface card (NIC) and the wireless access point (AP) are used to obtain network status data in real time. The data sources include the TCP / UDP layer in the network protocol stack and the wireless channel monitoring information (such as CSI). The acquisition frequency is set for data acquisition. The network status data includes bandwidth, latency and packet loss rate. The preprocessing steps include using Python's Pandas library to clean the data and remove missing values ​​and outliers; using the Min-Max normalization method to normalize the data and scale the data to the [0,1] interval.

[0022] Step S2, establishing a network status prediction model based on the LSTM neural network, and predicting the future network status according to the network status prediction model.

[0023] Step S201, establishing a network status prediction model based on LSTM, wherein the network status prediction model comprises an input layer, an LSTM layer and a fully connected layer.

[0024] Step S202, the preprocessed network status data is integrated to form a network status vector. For example, at time t, the network status vector is: F(t)=[Bnorm(t), Dnorm(t), Pnorm(t), B(t), D(t), P(t)], where Bnorm(t), Dnorm(t) and Pnorm(t) are the normalized bandwidth, delay and packet loss rate at time t, respectively, and B(t), D(t), P(t) are the bandwidth, delay and packet loss rate at time t, respectively.

[0025] Step S203: input the network state vector into the input layer of the LSTM basic model, where the input dimension is the number of features of the network state, and the number of features is 6 in this embodiment.

[0026] Step S204, add one or more LSTM layers, each layer contains a number of memory cells. In this embodiment, only one LSTM layer is used, containing 128 memory cells. Step S205, the output of the LSTM layer is mapped to the predicted future network state after passing through a fully connected layer. In this embodiment, the network state at time T in the future is predicted, and the output dimension is 3 (bandwidth, delay, and packet loss rate), which is expressed as, ; Among them, W o is the weight matrix of the fully connected layer, b o is the bias term, h T is the hidden state of LSTM at the last time step, and Xpred(T) is the network state at time T.

[0027] It should be noted that in order to ensure that the prediction model can accurately reflect the changes in network status, the model needs to be trained first, using the ReLU activation function and the mean square error as the loss function. The Adam optimization algorithm is used to train the model, and the model parameters are continuously adjusted to enable it to accurately predict the changing trend of the network status. The training data comes from the historical network status data. The specific training process can use existing technologies and will not be described here.

[0028] Step S3: Based on the future network status, a multi-objective optimization algorithm is used to select a transmission path.

[0029] Step S301, minimizing delay, minimizing packet loss rate and maximizing bandwidth are taken as optimization goals.

[0030] Step S302, generate a network topology distribution map, the nodes in the map represent routers, switches or base stations in the network, the links represent communication links between adjacent nodes, and based on the network topology distribution map, use the Dijkstra algorithm to find multiple feasible transmission paths, each of which is composed of transmission nodes and links. The Dijkstra algorithm is a commonly used shortest path algorithm that can calculate multiple feasible paths from the source node to the target node based on the weight of each link. In this embodiment, the weight can be comprehensively calculated based on indicators such as delay, packet loss rate and bandwidth in the network topology, and the specific calculation method is not limited.

[0031] It should be noted that the Dijkstra algorithm can quickly find multiple feasible paths, ensuring that there are always multiple paths to choose from in a complex wireless network environment. Through path identification based on network topology, new transmission paths can be discovered in real time to avoid bottlenecks or interruptions that may be caused by a single path. This provides more options for subsequent multi-objective optimization, thereby better realizing path selection that is adaptive to network status.

[0032] Step S303: predict all feasible transmission paths through the network status prediction model to determine the future network status of each transmission path.

[0033] Step S304, mapping the future network state of each transmission path to an objective function value, each objective function corresponds to delay minimization, packet loss rate minimization and bandwidth maximization, evaluating the quality of the path through the NSGA-III algorithm, randomly selecting several transmission paths as the initial population, and calculating the fitness of each transmission path according to its future network state.

[0034] Step S305, based on the fitness value, use the tournament selection method to select individuals from the current population for crossover and mutation, where the tournament selection method is a genetic algorithm selection mechanism that can randomly select a number of individuals (paths) from the current population, and compete according to the fitness value to select the best individual for crossover and mutation operations. The crossover operation generates a new path by combining two paths, and the mutation operation generates a new individual by making a small-scale random modification to the links on the path.

[0035] Step S306: When the preset number of iterations is reached, a transmission path is selected from the Pareto optimal solution set.

[0036] Step S4, adjusting the video encoding parameters for the selected transmission path, the encoding parameters including bit rate, resolution, frame rate, using an adaptive transmission protocol, and dynamically adjusting the transmission rate according to the real-time network status.

[0037] Step S401, obtaining network status data of a selected transmission path.

[0038] Step S402: If the network delay exceeds a preset first threshold, the frame rate of the video encoding is reduced by a preset first ratio, wherein the first threshold is an important indicator for measuring the real-time performance of video transmission. Generally, the expected delay should be between 100 and 500 milliseconds (ms). It should be noted that the specific threshold can be set by the usage scenario and requirements, and the first ratio can also be set according to the requirements. Generally, it is selected between 10% and 30%.

[0039] Step S403: if the network packet loss rate exceeds a preset second threshold, the bit rate of the video encoding is reduced by a preset second ratio, where the packet loss rate refers to the ratio of data packets lost during video transmission to the total data packets. For video transmission, a reasonable threshold of the packet loss rate should generally be controlled between 1% and 5%.

[0040] Step S404: if the network available bandwidth is lower than the bit rate of the video encoding, the bit rate and resolution of the video encoding are simultaneously reduced by a preset third ratio, where the available bandwidth refers to the maximum data transmission rate that can be used for video transmission on the current transmission path. When the available bandwidth is lower than the bit rate of the video encoding, the system needs to reduce the bit rate and resolution. When the network available bandwidth is lower than the bit rate of the video encoding, the bit rate and resolution of the video encoding are simultaneously reduced by 30%.

[0041] Step S405 , adopting a preset video encoding method, encoding the video source data according to the adjusted encoding parameter values ​​to obtain encoded video data.

[0042] It should be noted that if it is detected that the network status data changes and does not meet the preset video transmission quality requirements, then return to the step of adjusting the video encoding parameters, and re-judge and adjust the video encoding parameters according to the latest network status data. For the encoding parameters that need to be adjusted, dynamic adjustment is performed within their value range to obtain the adjusted encoding parameter values. Preferably, the value range of the video encoding frame rate is 10-60 frames / second, the value range of the bit rate is 100-8000kbps, and the value range of the resolution is 360P-1080P. The video source data is encoded according to the adjusted encoding parameter values ​​to obtain the encoded video data adapted to the current network transmission conditions. The encoded video data is transmitted through the selected transmission path, and the Ping command and Iperf tool are used to monitor the changes in parameters such as network delay, packet loss rate and bandwidth in real time. When it is detected that the network transmission conditions have changed and no longer meet the video transmission quality requirements, the video encoding parameters are re-judged and adjusted according to the latest network status information to dynamically adapt to the changes in network transmission conditions and ensure the quality of video transmission.

[0043] The adaptive transmission protocol is used to dynamically adjust the transmission rate according to the real-time network status, including: Get real-time network status of selected transmission paths.

[0044] Inputting the real-time network status into a pre-built random forest model to obtain a health score of the network; Specifically, the real-time network status parameters of the current network are obtained, and the network congestion degree is determined by the random forest model to obtain the network health score. In this embodiment, the scoring formula can be: ; In the formula, It is expressed as the network health score, n is the number of decision trees (the number of characteristic network state feature vectors), X i represents the input i-th network state feature vector, and RF represents the random forest model.

[0045] If the health score is lower than a preset health threshold, the rate adjustment mechanism of the adaptive transmission protocol is triggered to adjust the transmission rate.

[0046] The preset health threshold is set according to the situation. The rate adjustment mechanism of the adaptive transmission protocol includes: The real-time network state is used as a state space, which includes delay, packet loss rate, and bandwidth, and the transmission rate adjustment range is used as an action space, which includes operations such as increasing the rate and decreasing the rate, and the optimal transmission rate is determined by a Q-Learning algorithm; Input historical network status parameters into the network status prediction model to predict the future network status within a certain period of time; The prediction result is used as one of the states of the Q-Learning algorithm to adjust the transmission rate in advance.

[0047] The transmission rate is dynamically adjusted through the Q-Learning algorithm, with the network state parameters as the state space and the rate adjustment range as the action space. The optimal strategy is found through continuous trial and error learning, which improves bandwidth utilization while ensuring QoS. During the rate adjustment process, the BBR congestion control algorithm is also used to dynamically adjust the congestion window size by estimating the network bandwidth and RTT in real time, and quickly converge to the optimal transmission rate. For application layer data of different priorities, differentiated scheduling is performed through the WFQ algorithm to ensure the transmission quality of key data. To further improve the timeliness of rate adjustment, the LSTM neural network is used to predict the bandwidth and delay change trends in the future, and the historical network state parameters are used as input features. The prediction results are used as one of the states of Q-Learning to guide early rate adjustment.

[0048] This embodiment also provides a monitoring video transmission method system based on wireless communication, which includes: Data collection module, used to obtain network status data and perform preprocessing; LSTM network state prediction module, used to predict the future network state based on the network state prediction model; A path selection module, used to select a transmission path according to the future network status; The parameter adjustment module is used to adjust the video encoding parameters and transmission rate.

[0049] The present invention adopts a multi-objective optimization algorithm to select the transmission path. Combined with the future network status predicted by the LSTM model, different paths can be dynamically selected to avoid network congestion or degradation, and the quality and rate of video transmission can be flexibly adjusted according to the current network status to ensure the stable transmission of video quality under different network conditions.

[0050] This embodiment also provides a computer device, which is suitable for the monitoring video transmission method based on wireless communication, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the monitoring video transmission method based on wireless communication proposed in the above embodiment.

[0051] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0052] The present embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for transmitting surveillance video based on wireless communication as proposed in the above embodiment is implemented. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A monitoring video transmission method based on wireless communication, characterized in that: include: Acquire network status data and perform preprocessing, wherein the network status data includes bandwidth, delay and packet loss rate; Establishing a network status prediction model based on the LSTM neural network, and predicting the future network status according to the network status prediction model; Based on the future network state, a multi-objective optimization algorithm is used to select a transmission path; The video encoding parameters including bit rate, resolution and frame rate are adjusted for the selected transmission path. An adaptive transmission protocol is used to dynamically adjust the transmission rate according to the real-time network status.

2. The monitoring video transmission method based on wireless communication according to claim 1, characterized in that: The method of establishing a network state prediction model based on the LSTM neural network and predicting the future network state according to the network state prediction model includes: Establishing a network status prediction model based on LSTM, wherein the network status prediction model comprises an input layer, an LSTM layer and a fully connected layer; Integrate the preprocessed network state data to form a network state vector; Inputting the network state vector into the input layer of the LSTM base model; Add one or more LSTM layers, each containing several memory cells; The output of the LSTM layer is mapped to the predicted future network state after passing through a fully connected layer.

3. The monitoring video transmission method based on wireless communication according to claim 1, characterized in that: Based on the future network state, a multi-objective optimization algorithm is used to select a transmission path, including: Minimizing latency, minimizing packet loss, and maximizing bandwidth are the optimization goals; Based on the network topology distribution, the Dijkstra algorithm is used to find multiple feasible transmission paths, where each transmission path consists of transmission nodes and links; Predicting all feasible transmission paths through the network state prediction model to determine the future network state of each transmission path; The future network state of each transmission path is mapped to the objective function value, and the quality of the path is evaluated through the NSGA-III algorithm. Several transmission paths are randomly selected as the initial population, and the fitness of each transmission path is calculated according to its future network state. Based on the fitness value, individuals are selected from the current population for crossover and mutation using the tournament selection method; When the preset number of iterations is reached, a transmission path is selected from the Pareto optimal solution set.

4. The monitoring video transmission method based on wireless communication according to claim 1, characterized in that: The step of adjusting the video encoding parameters for the selected transmission path includes: Obtain network status data for selecting a transmission path; If the network delay exceeds a preset first threshold, reducing the frame rate of the video encoding by a preset first ratio; If the network packet loss rate exceeds a preset second threshold, reducing the bit rate of the video encoding by a preset second ratio; If the network available bandwidth is lower than the bit rate of the video encoding, reducing the bit rate and resolution of the video encoding by a preset third ratio; The video source data is encoded using a preset video encoding method according to the adjusted encoding parameter values ​​to obtain encoded video data.

5. The monitoring video transmission method based on wireless communication according to claim 4 is characterized in that: If it is detected that the network status data has changed and does not meet the preset video transmission quality requirement, the process returns to the step of adjusting the video encoding parameters, and re-determines and adjusts the video encoding parameters according to the latest network status data.

6. The monitoring video transmission method based on wireless communication according to claim 1, characterized in that: The adaptive transmission protocol is used to dynamically adjust the transmission rate according to the real-time network status, including: Get the real-time network status of the selected transmission path; Inputting the real-time network status into a pre-built random forest model to obtain a health score of the network; If the health score is lower than a preset health threshold, the rate adjustment mechanism of the adaptive transmission protocol is triggered to adjust the transmission rate.

7. The monitoring video transmission method based on wireless communication according to claim 5, characterized in that: The rate adjustment mechanism of the adaptive transmission protocol includes: The real-time network state is used as the state space, the transmission rate adjustment range is used as the action space, and the optimal transmission rate is determined by a Q-Learning algorithm; Inputting historical network status parameters into a network status prediction model to predict future network status within a certain period of time; The prediction result is used as one of the states of the Q-Learning algorithm to adjust the transmission rate in advance.

8. A monitoring video transmission method system based on wireless communication for implementing any of the methods 1 to 7, characterized in that: include: Data collection module, used to obtain network status data and perform preprocessing; LSTM network state prediction module, used to predict the future network state based on the network state prediction model; A path selection module, used to select a transmission path according to the future network status; The parameter adjustment module is used to adjust the video encoding parameters and transmission rate.

Images