Video multi-level adaptive compression cross-layer transmission optimization method and system based on deep learning

Abstract

The invention discloses a video multi-level adaptive compression cross-layer transmission optimization method and system based on deep learning, and relates to the technical field of video transmission, and the method comprises the steps: designing a compression level selection model based on deep learning, outputting an optimal compression level based on original video data and network environment parameters, and obtaining a compression strategy corresponding to the optimal compression level; designing a transmission strategy selection model, and generating a transmission strategy based on the video quality parameters and the network environment parameters; a global utility function is defined, a global utility value when the compression strategy and the transmission strategy are assumed to be executed is calculated, an optimization objective function is designed, and the optimization objective function inputs the compression strategy and the transmission strategy and outputs the optimized compression strategy and the optimized transmission strategy; condition judgment logic is set, and a final strategy is selected based on the compression strategy and the transmission strategy before and after optimization, so that global optimization of video compression and transmission is realized.

Description

Technical Field

[0001] The present application relates to the field of video transmission technology, and in particular to a method and system for optimizing cross-layer transmission of multi-level adaptive compression of videos using deep learning. Background Art

[0002] Video multi-level compression cross-layer transmission is a comprehensive method for complex video data streams. Its core goal is to achieve efficient video compression and reliable cross-layer data transmission optimization through the collaboration of edge computing nodes and terminals in a network environment with limited or unstable bandwidth. The key to this technology is the deep integration of video compression algorithms, transmission protocols and edge computing capabilities.

[0003] The patent application with publication number CN111510774A discloses an intelligent terminal image compression algorithm that combines edge computing and deep learning. The intelligent terminal image compression algorithm that combines edge computing and deep learning deploys an image compression model during the data transmission process between the terminal and the cloud, compresses the image data and video data obtained by the edge terminal, and retains key frames and motion information to reduce data transmission, reduce bandwidth pressure, save a lot of storage space and search time for locating abnormal behavior.

[0004] Traditional video compression and transmission strategies are relatively fixed and cannot be dynamically adjusted according to actual conditions, resulting in poor compression and transmission effects and waste of resources. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one purpose of the present application is to propose a deep learning video multi-level adaptive compression cross-layer transmission optimization method and system, which realizes the global optimization of video compression and transmission.

[0006] One aspect of the present application provides a deep learning video multi-level adaptive compression cross-layer transmission optimization method, including:

[0007] Step S100: designing a compression level selection model based on deep learning, outputting an optimal compression level based on original video data and network environment parameters, and obtaining a compression strategy corresponding to the optimal compression level;

[0008] Step S200: Designing a transmission strategy selection model to generate a transmission strategy based on video quality parameters and network environment parameters;

[0009] Step S300: define a global utility function, calculate the global utility value when the compression strategy and the transmission strategy are assumed to be executed, and design an optimization objective function, wherein the optimization objective function inputs the compression strategy and the transmission strategy and outputs the optimized compression strategy and the transmission strategy;

[0010] Step S400: setting conditional judgment logic, and selecting a final strategy based on the compression strategy and transmission strategy before and after optimization;

[0011] The design is based on a deep learning-based compression level selection model. Based on the original video data and network environment parameters, the specific method of outputting the optimal compression level is:

[0012] Step S110: constructing a compression level selection model based on deep learning, wherein the compression level selection model adopts a combined structure of a convolutional neural network and a long short-term memory network;

[0013] Step S120: Acquire real-time original video data and network environment parameters;

[0014] Step S130: The convolutional neural network extracts features from the video frames of the original video data to obtain a feature vector for each frame. The feature vectors of all frames constitute a frame feature sequence. The frame feature sequence extracted by the convolutional neural network is input into the long short-term memory network to extract video dynamic features of the frame feature sequence.

[0015] Step S140: combining the video dynamic features with the network environment parameters to form a comprehensive feature vector;

[0016] Step S150: using a decision maker in a compression level selection model to map the comprehensive feature vector into a probability distribution of compression levels, and outputting the compression level with the maximum probability value as the optimal compression level of the original video data; the decision maker is a fully connected layer and a softmax activation function;

[0017] The training process of the compression level selection model based on deep learning is:

[0018] Step S111: collect various types of original video samples from the data set, use network simulation tools to compress and transmit the original video samples based on different network environments, and record network environment parameters;

[0019] Step S112: for each set of original video samples and network environment parameters, annotate the optimal compression level label;

[0020] Step S113: using a cross entropy loss function to measure the difference between the probability distribution of the optimal compression level predicted by the compression level selection model based on deep learning and the actual optimal compression level label;

[0021] Step S114: using a stochastic gradient descent algorithm to update model parameters through back propagation, taking minimizing the value of the cross entropy loss function as the optimization goal, training the compression level selection model based on deep learning, and obtaining a trained compression level selection model;

[0022] The specific method of obtaining the compression strategy corresponding to the optimal compression level is:

[0023] Step S160: defining compression level and compression parameters, wherein the compression parameters include quantization parameter QP, intra-frame prediction mode Intra, inter-frame prediction mode Inter, and entropy coding mode Entropy, and designing a mapping table between compression level and compression parameters based on the compression parameters;

[0024] Step S170: using the optimal compression level as an index, searching for corresponding compression parameters in a mapping table, and combining the compression parameters as a compression strategy;

[0025] The specific method of designing a transmission strategy selection model and generating a transmission strategy based on video quality parameters and network environment parameters is as follows:

[0026] Step S210: defining a state space based on video quality parameters and network environment parameters, wherein the state includes application layer state s app , transport layer status s trans , network layer status s net ;

[0027] The application layer status is a video quality parameter, specifically including video encoding parameters, frame rate, and resolution;

[0028] The transport layer status and network layer status are network environment parameters. The transport layer status includes current bandwidth, packet loss rate, and delay. The network layer status includes network topology, link quality, and congestion level.

[0029] Step S220: define an action space, wherein the action includes application layer actions a app , transport layer action a trans 、Network layer action a net ;

[0030] Step S230: Define a reward function r, based on the application layer reward r app , transport layer reward r trans 、Network layer reward r net Design comprehensive reward functions;

[0031] Step S240: Designing an agent system, wherein the agent includes an application layer agent, a transport layer agent, and a network layer agent, and using a deep reinforcement learning model to build a transmission strategy selection model;

[0032] Step S250: At each time step, according to the states and policy networks corresponding to each layer observed by the current agent, generate actions of the corresponding layer, combine the actions generated by the agent into a transmission strategy and output it;

[0033] The design method of the reward function r is:

[0034] Step S231: Count the peak signal-to-noise ratio of the video at the application layer as a measure of video quality, based on the peak signal-to-noise ratio PSNR of each time step and the maximum peak signal-to-noise ratio PSNR max Calculate the normalized value of video quality and count the number of times the video playback stalls count Stall duration duration Calculate the video fluency impact factor and obtain the video startup time. Calculate the video startup time impact factor and calculate the product of the video startup time impact factor, the normalized value of the video quality and the video fluency impact factor to obtain the application layer reward r. app ;

[0035] Step S232: Count the time interval delay from the sending to the receiving of the video frame at the transport layer, and calculate its difference from the target delay delay. target The influence factor of transmission delay is calculated based on the difference, the proportion of video frames lost during transmission is counted, the influence factor of packet loss rate is calculated, and the product of the normalized value of video quality, the influence factor of transmission delay and the influence factor of packet loss rate is calculated to obtain the transport layer reward r trans ;

[0036] Step S233: Obtain the network bandwidth utilization at the network layer, calculate the impact factor of the network bandwidth utilization based on the network bandwidth utilization and the target utilization, obtain the queuing delay in the network, calculate the impact factor of the queuing delay based on the queuing delay and the target queuing delay, calculate the impact factor of resource consumption according to the resource consumption of the transmission process, calculate the product of the impact factor of the network bandwidth utilization, the impact factor of the queuing delay, and the impact factor of the resource consumption, and obtain the network layer reward r net ;

[0037] Step S234: Calculate a comprehensive reward function based on the application layer reward, the transport layer reward, and the network layer reward;

[0038] The training process of the transmission strategy selection model is:

[0039] Step S241: Collect historical data of the application layer, transport layer and network layer, including state transition data, action selection and performance feedback data;

[0040] Step S242: For each agent, initialize the parameters of its strategy network and value network;

[0041] Step S243: Initialize the experience replay cache of the agent; the experience replay cache is used to store state transition data during the training process;

[0042] Step S244: for each agent, according to the current state s and the policy network π(a|s;θ), an action a is generated, the generated action is executed, the agent interacts with the environment, and an immediate reward function r and the state s′ of the next time step are obtained;

[0043] Step S245: Store the state transition data (s, a, r, s′) in the experience replay cache, and randomly extract a batch of state transition data (s i ,a i ,r i ,s′ i ), for each agent, calculate the temporal difference error, use the temporal difference error to update the parameter φ of the value network, and minimize the mean square error loss; where s i 、a i 、r i , s′ i is the state, action, reward function and state of the next time step in the state transition data of the randomly selected time step i;

[0044] Step S246: Use the policy gradient algorithm to update the parameters θ of the policy network to maximize the expected return;

[0045] Step S247: repeat the above steps S245 to S246 until the preset number of training rounds is reached to obtain a trained transmission strategy selection model;

[0046] The specific method of defining the global utility function, calculating the global utility value when assuming the compression strategy and the transmission strategy are implemented, and designing the optimization objective function is as follows:

[0047] Step S310: Define a global utility function U(s) based on video quality parameters, transmission delay, bandwidth utilization, compression ratio, and resource consumption. c ,s t );

[0048] Step S320: Calculate the global utility function value when the compression strategy and the transmission strategy are assumed to be executed, wherein the global utility function value includes the global utility function value of the compression strategy. and the global utility function value of executing the transmission strategy And the global utility function value when the compression strategy and the transmission strategy are executed simultaneously

[0049] Step S330: Design an optimization objective function, take the current compression strategy and transmission strategy as input, take maximizing video quality, minimizing transmission delay, maximizing bandwidth utilization, and minimizing total resource consumption as optimization goals, and output the optimized compression strategy s c ′ and transmission strategy s t ′;

[0050] The specific method of setting the conditional judgment logic is:

[0051] Step S410: The conditional judgment logic includes:

[0052] if and where θ 1 is the first utility threshold, then the compression strategy before optimization is directly executed and transmission strategy U(s c ′,s t ′) is the optimized compression strategy s c ′ and optimized transmission strategy s t ′’s global utility function value;

[0053] if and where θ 2 is the second utility threshold, then only the transmission strategy is optimized and the compression strategy is executed and optimized transmission strategies t ′;

[0054] if and Then only the compression strategy is optimized and the optimized compression strategy s is executed c ′ and transmission strategy

[0055] Otherwise, the compression strategy and transmission strategy are optimized at the same time, and the optimized compression strategy s is executed. c ′ and transmission strategy s t ′;

[0056] Step S420: Select the compression strategy and transmission strategy to be executed based on the conditional judgment logic output to obtain the final strategy;

[0057] The method for adjusting the first utility threshold and the second utility threshold is: respectively obtaining the average global utility function values ​​of the compression strategy and the transmission strategy before optimization, the compression strategy before optimization and the transmission strategy after optimization, and the compression strategy after optimization and the transmission strategy before optimization in the historical time at the current moment Based on the current first utility threshold θ 1 and the second utility threshold θ 2 , combined with the average global utility function value, the first utility threshold and the second utility threshold are adjusted to obtain the adjusted first utility threshold θ 1 ′ and the second utility threshold θ 2 ′.

[0058] One aspect of the present application provides a deep learning video multi-level adaptive compression cross-layer transmission optimization system, including:

[0059] The compression strategy generation module is used to design a compression level selection model based on deep learning, output the optimal compression level based on the original video data and network environment parameters, and obtain the compression strategy corresponding to the optimal compression level;

[0060] A transmission strategy generation module is used to design a transmission strategy selection model and generate a transmission strategy based on video quality parameters and network environment parameters;

[0061] A global optimization calculation module, used to define a global utility function, calculate a global utility value when assuming that a compression strategy and a transmission strategy are executed, and design an optimization objective function, wherein the optimization objective function inputs the compression strategy and the transmission strategy and outputs an optimized compression strategy and the transmission strategy;

[0062] The final strategy output module is used to set the conditional judgment logic and select the final strategy based on the compression strategy and transmission strategy before and after optimization.

[0063] Compared with the prior art, the deep learning video multi-level adaptive compression cross-layer transmission optimization method and system proposed in this application have the following advantages:

[0064] This application designs a compression level selection model, which uses a convolutional neural network to extract video frame features, combines a long short-term memory network to capture the temporal dynamic characteristics of the video, and then comprehensively considers network environment parameters to intelligently predict the optimal compression level. It fully considers the differences in video content and network conditions, overcomes the limitations of traditional fixed compression parameters, and the dynamic adjustment of the compression level can minimize the video bit rate while ensuring video quality, saving transmission bandwidth and improving network utilization efficiency.

[0065] This application constructs a multi-agent system, which uses deep reinforcement learning technology from three perspectives: application layer, transmission layer, and network layer, to autonomously learn and generate the optimal transmission strategy combination. The all-round and multi-level transmission strategy optimization breaks the traditional single-layer optimization idea, and can more comprehensively adapt to the complex and changeable network environment. It comprehensively considers factors at all levels and dynamically adjusts the transmission strategy. It can minimize transmission delays, improve bandwidth utilization, reduce freezes and frame losses, and greatly improve the user's viewing experience while ensuring the quality of video playback.

[0066] The jointly optimized compression and transmission strategy designed in this application can better balance multiple goals such as video quality, latency, and bandwidth utilization, achieve optimal resource allocation and performance overall, and avoid waste of resources and loss of effects.

[0067] This application is based on the compression and transmission strategies before and after optimization, performs logical judgment based on global utility, and dynamically decides to execute the strategies before and after optimization, thus avoiding the waste of resources and increased latency caused by blind optimization, and achieving global optimization of video compression and transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A method flow chart of a method for optimizing cross-layer transmission of multi-level adaptive compression of videos through deep learning provided in this application;

[0069] Figure 2 A flow chart of the method for generating the transmission strategy provided in this application;

[0070] Figure 3 A flow chart of the optimization method for optimizing the objective function provided in this application;

[0071] Figure 4 Functional module diagram of the deep learning video multi-level adaptive compression cross-layer transmission optimization system provided in this application. DETAILED DESCRIPTION

[0072] In order to better understand the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application and do not limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0073] In the accompanying drawings, the size, dimensions and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not strictly drawn to scale. As used herein, the terms "substantially", "approximately" and similar terms are used as terms of approximation, not as terms of degree, and are intended to illustrate the inherent deviations in measurements or calculations that will be recognized by those of ordinary skill in the art. In addition, in this application, the order in which the steps are described does not necessarily represent the order in which these processes occur in actual operation, unless otherwise specified or can be derived from the context.

[0074] It should also be understood that expressions such as "including", "comprising", "having", "containing" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.

[0075] Unless otherwise specified, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that, unless clearly stated in this application, words defined in common dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0076] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0077] Example 1

[0078] like Figure 1 As shown, the deep learning video multi-level adaptive compression cross-layer transmission optimization method provided by this application includes:

[0079] Step S100: designing a compression level selection model based on deep learning, outputting an optimal compression level based on original video data and network environment parameters, and obtaining a compression strategy corresponding to the optimal compression level;

[0080] The design is based on a deep learning-based compression level selection model. Based on the original video data and network environment parameters, the specific method of outputting the optimal compression level is:

[0081] Step S110: constructing a compression level selection model based on deep learning, wherein the compression level selection model adopts a combined structure of a convolutional neural network and a long short-term memory network;

[0082] The training process of the compression level selection model based on deep learning is:

[0083] Step S111: collect various types of original video samples from the data set, use network simulation tools to compress and transmit the original video samples based on different network environments, and record network environment parameters;

[0084] Step S112: for each set of original video samples and network environment parameters, annotate the optimal compression level label;

[0085] The labeling method of the optimal compression level label adopts a method of taking the average value of scores given by multiple experts.

[0086] Step S113: using a cross entropy loss function to measure the difference between the probability distribution of the optimal compression level predicted by the compression level selection model based on deep learning and the actual optimal compression level label;

[0087] Step S114: using a stochastic gradient descent algorithm to update model parameters through back propagation, taking minimizing the value of the cross entropy loss function as the optimization goal, training the compression level selection model based on deep learning, and obtaining a trained compression level selection model;

[0088] The compression level selection model based on deep learning utilizes video dynamic features and network environment parameters to learn the underlying rules of compression level selection through an end-to-end training method, and can adjust the compression strategy according to real-time dynamic input to improve the quality and efficiency of video transmission.

[0089] Step S120: Acquire real-time original video data and network environment parameters;

[0090] The network environment parameters include current bandwidth, packet loss rate, delay, network topology, link quality, and congestion level. The network environment parameters refer to relevant network status data of the transport layer and the network layer;

[0091] Step S130: The convolutional neural network extracts features from the video frames of the original video data to obtain a feature vector for each frame. The feature vectors of all frames constitute a frame feature sequence. The frame feature sequence extracted by the convolutional neural network is input into the long short-term memory network to extract video dynamic features of the frame feature sequence.

[0092] The video dynamic features refer to the temporal dynamic features of the video sequence learned by the long short-term memory network.

[0093] Step S140: combining the video dynamic features with the network environment parameters to form a comprehensive feature vector;

[0094] The comprehensive feature vector represents the status of the current video content and the network environment;

[0095] Step S150: using a decision maker in a compression level selection model to map the comprehensive feature vector into a probability distribution of compression levels, and outputting the compression level with the maximum probability value as the optimal compression level of the original video data; the decision maker is a fully connected layer and a softmax activation function;

[0096] The fully connected layer in the decision maker includes multiple hidden layers, and the specific number of layers can be adjusted according to the complexity of the task;

[0097] The specific method of obtaining the compression strategy corresponding to the optimal compression level is:

[0098] Step S160: defining compression level and compression parameters, wherein the compression parameters include quantization parameter QP, intra-frame prediction mode Intra, inter-frame prediction mode Inter, and entropy coding mode Entropy, and designing a mapping table between compression level and compression parameters based on the compression parameters;

[0099] The mapping table is designed based on human experience and domain knowledge; preferably, the mapping table for designing compression levels and compression parameters based on compression parameters is specifically:

[0100] CL=1: QP=20, Intra=3, Inter=2, Entropy=1;

[0101] CL=2: QP=25, Intra=3, Inter=1, Entropy=1;

[0102] CL=3: QP=30, Intra=2, Inter=1, Entropy=0;

[0103] CL=4: QP=35, Intra=2, Inter=0, Entropy=0;

[0104] CL=5: QP=40, Intra=1, Inter=0, Entropy=0;

[0105] Among them, CL is the compression level;

[0106] Step S170: using the optimal compression level as an index, searching for corresponding compression parameters in a mapping table, and combining the compression parameters as a compression strategy;

[0107] The above-mentioned compression strategy generation method based on the mapping table is seamlessly integrated with the compression level selection model to achieve rapid mapping from compression level to compression strategy, providing strong support for subsequent video transmission and optimization.

[0108] The above step S100 aims to select the optimal compression level based on the deep learning model according to the network status and video characteristics. The optimal compression level can achieve local optimality at the compression level, ensure video quality, and adapt to the limitations of network transmission.

[0109] Step S200: Designing a transmission strategy selection model to generate a transmission strategy based on video quality parameters and network environment parameters;

[0110] like Figure 2 As shown, the specific method of designing a transmission strategy selection model and generating a transmission strategy based on video quality parameters and network environment parameters is:

[0111] Step S210: defining a state space based on video quality parameters and network environment parameters, wherein the state includes application layer state s app , transport layer status s trans , network layer status s net ;

[0112] The application layer status is a video quality parameter, specifically including video encoding parameters, frame rate, and resolution;

[0113] The transport layer status and network layer status are network environment parameters. Specifically, the transport layer status includes current bandwidth, packet loss rate, and delay, and the network layer status includes network topology, link quality, and congestion level.

[0114] The state space s is expressed as s=[s app ,s trans ,s net ];

[0115] Step S220: define an action space, wherein the action includes application layer actions a app , transport layer action a trans 、Network layer action a net ;

[0116] The application layer action includes adjusting video quality parameters;

[0117] The transport layer actions include transmission rate and retransmission mechanism;

[0118] The network layer action includes transmission path and network traffic;

[0119] The action space a is expressed as a=[a app ,a trans ,a net ];

[0120] Step S230: Define a reward function r, based on the application layer reward r app , transport layer reward r trans 、Network layer reward r net Design comprehensive reward functions;

[0121] The design method of the reward function r is:

[0122] Step S231: Count the peak signal-to-noise ratio of the video at the application layer as a measure of video quality, based on the peak signal-to-noise ratio PSNR of each time step and the maximum peak signal-to-noise ratio PSNR maxCalculate the normalized value of video quality and count the number of times the video playback stalls count Stall duration duration Calculate the video fluency impact factor, obtain the video startup time, calculate the video startup time impact factor, calculate the product of the video startup time impact factor, the normalized value of the video quality, and the video fluency impact factor, and obtain the application layer reward;

[0123] The calculation formula for the application layer reward is: in, is the normalized value of video quality, is the factor affecting video fluency, is the influencing factor of video startup time, γ is the balancing factor for controlling the impact of lag on application layer rewards, and δ is the balancing factor for controlling the impact of startup time on application layer rewards;

[0124] The values ​​of the balance factor for controlling the degree of influence of the jamming on the application layer reward and the balance factor for controlling the degree of influence of the startup time on the application layer reward are set by those skilled in the art based on experience.

[0125] The application layer reward takes into account the video quality, lag and startup time, and combines the three indicators in the form of a product. The higher the video quality, the fewer lags and the shorter the startup time, the greater the application layer reward, which means the user's viewing experience is better.

[0126] Step S232: Count the time interval delay from the sending to the receiving of the video frame at the transport layer, and calculate its difference from the target delay delay. target , calculate the impact factor of transmission delay based on the difference, count the proportion of video frames lost during transmission, calculate the impact factor of packet loss rate, calculate the product of the normalized value of video quality, the impact factor of transmission delay, and the impact factor of packet loss rate, and obtain the transport layer reward;

[0127] The calculation formula for the transport layer reward is: in, is the influencing factor of transmission delay, is the influencing factor of packet loss rate, It is a balancing factor that controls the effect of transmission delay on reward, ∈ is a constant;

[0128] The value of the balance factor and the constant ∈ that control the degree of influence of transmission delay on the reward is set by those skilled in the art based on experience.

[0129] The transport layer reward takes into account video quality, latency, and packet loss rate, combining the three indicators in the form of a product. The higher the video quality, the lower the latency, and the lower the packet loss rate, the greater the transport layer reward, indicating better transmission performance.

[0130] Step S233: Obtain network bandwidth utilization at the network layer, based on the network bandwidth utilization util and the target utilization util target Calculate the factors affecting network bandwidth utilization and obtain the queue delay in the network delay , based on the queuing delay and the target queuing delay Calculate the impact factor of queuing delay, calculate the impact factor of resource consumption based on the resource consumption of the transmission process, calculate the product of the impact factor of network bandwidth utilization, the impact factor of queuing delay, and the impact factor of resource consumption, and obtain the network layer reward;

[0131] The calculation formula for the network layer reward is: Among them, η is the balance factor that controls the impact of queuing delay on rewards, ∈' is a constant used to control the denominator to be non-zero, is the influencing factor of network bandwidth utilization, is the influencing factor of queuing delay, is the influencing factor of resource consumption;

[0132] The balance factor for controlling the influence of queuing delay on the reward and the value of the constant for controlling the denominator to be not 0 are set by those skilled in the art based on experience.

[0133] Step S234: Calculate a comprehensive reward function based on the application layer reward, the transport layer reward, and the network layer reward;

[0134] The calculation formula of the reward function is: Among them, α and β are balance factors; the balance factor is used to control the contribution of the network layer reward to the comprehensive reward function, and the value is set by those skilled in the art based on experience;

[0135] Among them, r app × trans Represents the comprehensive performance of the application layer and the transport layer. The larger the two sub-rewards are, the better the user experience and transmission quality of video transmission. Represents the influence factor of network layer performance on the total reward function. When r net When it is greater than β, the function value is close to 1, indicating that the network layer performance contributes more to the total reward; when r net When it is less than β, the function value is close to e α×β , indicating that the network layer performance contributes less to the total reward function.

[0136] The reward function r reflects the idea of ​​cross-layer optimization. When the network layer performance is good, the reward function r is more determined by the application layer reward and the transmission layer reward, which means that when the network conditions permit, priority should be given to improving user experience and transmission quality; when the network layer performance is poor, the reward function r is limited by the network layer performance, which means that when the network conditions are not ideal, priority should be given to improving the efficiency and quality of network transmission. This nonlinear combination method can balance the performance of different levels and prompt the intelligent agent to comprehensively consider various factors in cross-layer optimization, so as to make more comprehensive and reasonable decisions.

[0137] Step S240: Designing an agent system, wherein the agent includes an application layer agent, a transport layer agent, and a network layer agent, and using a deep reinforcement learning model to build a transmission strategy selection model;

[0138] The intelligent agents achieve cross-layer information sharing and joint decision-making through communication and collaboration. Specifically, the application layer intelligent agent feeds back video quality parameters to the transport layer intelligent agent, and the network layer intelligent agent feeds back congestion information to the application layer intelligent agent.

[0139] The training process of the transmission strategy selection model is:

[0140] Step S241: Collect historical data of the application layer, transport layer and network layer, including state transition data, action selection and performance feedback data;

[0141] The historical data comes from an actual video transmission system or is generated through a simulation environment.

[0142] Step S242: For each agent, initialize the parameters of its strategy network and value network;

[0143] The policy network is used to generate the probability distribution of actions, and its parameters include the application layer policy network parameters θ app , transport layer strategy network parameters θ trans and network layer policy network parameters θ net ;

[0144] The value network is used to estimate the value of the state-action pair, and its parameters include the application layer value network parameter φ app , transport layer value network parameter φ trans and network layer value network parameter φ net ;

[0145] Step S243: Initialize the experience replay cache of the agent; the experience replay cache is used to store state transition data during the training process;

[0146] Step S244: for each agent, according to the current state s and the policy network π(a|s;θ), an action a is generated, the generated action is executed, the agent interacts with the environment, and an immediate reward function r and the state s′ of the next time step are obtained;

[0147] Step S245: Store the state transition data (s, a, r, s′) in the experience replay cache, and randomly extract a batch of state transition data (s i ,a i ,r i ,s′ i ), for each agent, calculate the temporal difference error, use the temporal difference error to update the parameter φ of the value network, and minimize the mean square error loss;

[0148] Among them, s i 、a i 、r i , s′ i is the state, action, reward function and state of the next time step in the state transition data of the randomly selected time step i;

[0149] The timing difference error δ i The calculation formula is: Among them, Q(s i ,a i ; φ) represents the state-action value estimated by the value network, γ δ is the discount factor.

[0150] The discount factor ranges from 0 to 1 and is used to weigh the importance of current rewards and future rewards;

[0151] The calculation formula of the mean square error loss L(φ) is: Among them, N and i are the number of samples;

[0152] Step S246: Use the policy gradient algorithm to update the parameters θ of the policy network to maximize the expected return;

[0153] The expected return The calculation formula is: Among them, ρ π represents the steady-state distribution of states;

[0154] Step S247: repeat the above steps S245 to S246 until the preset number of training rounds is reached to obtain a trained transmission strategy selection model;

[0155] The value of the preset number of training rounds is set by those skilled in the art based on experience.

[0156] Step S250: At each time step, according to the states and policy networks corresponding to each layer observed by the current agent, generate actions of the corresponding layer, combine the actions generated by the agent into a transmission strategy and output it;

[0157] Step S300: define a global utility function, calculate the global utility value when the compression strategy and the transmission strategy are assumed to be executed, and design an optimization objective function, wherein the optimization objective function inputs the compression strategy and the transmission strategy and outputs the optimized compression strategy and the transmission strategy;

[0158] like Figure 3 As shown, the specific method of defining the global utility function, calculating the global utility value when assuming the compression strategy and the transmission strategy are executed, and designing the optimization objective function is:

[0159] Step S310: Define a global utility function U(s) based on video quality parameters, transmission delay, bandwidth utilization, compression ratio, and resource consumption. c ,s t );

[0160] The calculation formula of the global utility function is: Among them, PSNR (s c ) is the compression strategy s c Video quality under delay(s t ) is the transmission strategy s t The transmission delay under 0 is the transmission delay threshold, uti(s t ) is the transmission strategy s t Bandwidth utilization under c ) is the compression strategy s c The compression ratio under 0 For the optimal compression ratio, energy consumption (s c ,s t ) indicates that in compression strategy s c and transmission strategies t The total resource consumption under the condition, α1, β1, γ1 are balance factors, σ is the standard deviation of the normal distribution, which is used to control the balance between video quality and compression ratio, and ∈1 is a constant to avoid the denominator being 0;

[0161] The transmission delay threshold, the optimal compression ratio, the balance factor, the standard deviation of the normal distribution, and the value of the constant that avoids the denominator being 0 are set by those skilled in the art based on experience.

[0162] The compression ratio is the ratio of the original video data size to the compressed video size;

[0163] Step S320: Calculate the global utility function value when the compression strategy and the transmission strategy are assumed to be executed, wherein the global utility function value includes the global utility function value of the compression strategy. and the global utility function value of executing the transmission strategy And the global utility function value when the compression strategy and the transmission strategy are executed simultaneously

[0164] Step S330: Design an optimization objective function, take the current compression strategy and transmission strategy as input, take maximizing video quality, minimizing transmission delay, maximizing bandwidth utilization, and minimizing total resource consumption as optimization goals, and output the optimized compression strategy s c ′ and transmission strategy s t ′;

[0165] Step S400: setting conditional judgment logic, and selecting a final strategy based on the compression strategy and transmission strategy before and after optimization;

[0166] The specific method of setting the conditional judgment logic is:

[0167] Step S410: The conditional judgment logic includes:

[0168] if and where θ 1 is the first utility threshold, then the compression strategy before optimization is directly executed and transmission strategy U(s c ′,s t ′) is the optimized compression strategy s c ′ and optimized transmission strategy s t ′’s global utility function value;

[0169] if and where θ 2 is the second utility threshold, then only the transmission strategy is optimized and the compression strategy is executed and optimized transmission strategies t ′;

[0170] if and Then only the compression strategy is optimized and the optimized compression strategy s is executed c ′ and transmission strategy

[0171] Otherwise, the compression strategy and transmission strategy are optimized at the same time, and the optimized compression strategy s is executed. c ′ and transmission strategy s t ′;

[0172] The method for adjusting the first utility threshold and the second utility threshold is: respectively obtaining the average global utility function values ​​of the compression strategy and the transmission strategy before optimization, the compression strategy before optimization and the transmission strategy after optimization, and the compression strategy after optimization and the transmission strategy before optimization in the historical time at the current moment Based on the current first utility threshold θ 1 and the second utility threshold θ 2 , combined with the average global utility function value, the first utility threshold and the second utility threshold are adjusted to obtain the adjusted first utility threshold θ 1 ′ and the second utility threshold θ 2 ′;

[0173] The calculation formula of the adjusted first utility threshold is:

[0174] The calculation formula of the adjusted second utility threshold is:

[0175] Wherein, α2 is a smoothing coefficient; the value of α2 is set by those skilled in the art based on experience.

[0176] Step S420: Select the compression strategy and transmission strategy to be executed based on the conditional judgment logic output to obtain the final strategy;

[0177] The above steps realize direct execution of the pre-optimization strategy when the global utility values ​​of the compression strategy and the transmission strategy are very high, thus avoiding unnecessary optimization overhead; when the optimization potential of the compression strategy or the transmission strategy is large, targeted optimization is performed to improve the optimization efficiency; when the global utility value of the strategy before optimization is low and both have optimization space, the two strategies are optimized simultaneously to achieve global optimality, and by adaptively adjusting the threshold, a dynamic balance between decision efficiency and optimization performance is achieved under different network environments and video contents.

[0178] Furthermore, the final strategy obtained is sent to the edge node, which is responsible for the execution and implementation of the final strategy;

[0179] Specifically, the final strategy obtained is sent to the edge node, and the edge node is responsible for the execution and implementation of the final strategy, including: the central server encapsulates the final decision into a policy update message, sends the policy update message to each edge node through a communication channel, and the edge node receives the policy update message and parses out specific compression strategy parameters and transmission strategy parameters;

[0180] The edge node stores the received compression policy parameters and transmission policy parameters in a local policy database;

[0181] When the edge node receives a video request from a user, it obtains the final policy from the policy mapping table based on the original video data of the video request and the user's network environment parameters, and compresses and transmits the video in real time;

[0182] Optionally, a video transcoding service is set up on the edge node to perform real-time transcoding and adaptation of the original video data based on the user's network environment parameters, thereby reducing the computing burden of the terminal device.

[0183] Optionally, a video cache service is deployed on the edge node to store popular or frequently accessed video content; when a user requests the video content, it can be directly obtained from the edge node, reducing the transmission delay and bandwidth consumption from the remote server.

[0184] Example 2

[0185] like Figure 4 As shown, the deep learning video multi-level adaptive compression cross-layer transmission optimization system provided by this application includes:

[0186] The compression strategy generation module is used to design a compression level selection model based on deep learning, output the optimal compression level based on the original video data and network environment parameters, and obtain the compression strategy corresponding to the optimal compression level;

[0187] A transmission strategy generation module is used to design a transmission strategy selection model and generate a transmission strategy based on video quality parameters and network environment parameters;

[0188] A global optimization calculation module, used to define a global utility function, calculate a global utility value when assuming that a compression strategy and a transmission strategy are executed, and design an optimization objective function, wherein the optimization objective function inputs the compression strategy and the transmission strategy and outputs an optimized compression strategy and the transmission strategy;

[0189] The final strategy output module is used to set the conditional judgment logic and select the final strategy based on the compression strategy and transmission strategy before and after optimization.

[0190] The methods, apparatuses, and devices of the present application may be implemented in many ways. For example, the methods, apparatuses, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0191] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0192] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A deep learning video multi-level adaptive compression cross-layer transmission optimization method, characterized in that: include: Design a compression level selection model based on deep learning, output the optimal compression level based on the original video data and network environment parameters, and obtain the compression strategy corresponding to the optimal compression level; Design a transmission strategy selection model to generate transmission strategies based on video quality parameters and network environment parameters; Define a global utility function, calculate the global utility value when assuming that the compression strategy and the transmission strategy are executed, and design an optimization objective function, wherein the optimization objective function inputs the compression strategy and the transmission strategy and outputs the optimized compression strategy and the transmission strategy; Set the conditional judgment logic and select the final strategy based on the compression strategy and transmission strategy before and after optimization.

2. The deep learning video multi-level adaptive compression cross-layer transmission optimization method according to claim 1 is characterized in that: The design is based on a deep learning-based compression level selection model. Based on the original video data and network environment parameters, the specific method of outputting the optimal compression level is: Constructing a compression level selection model based on deep learning, wherein the compression level selection model adopts a combined structure of a convolutional neural network and a long short-term memory network; Obtain real-time raw video data and network environment parameters; The convolutional neural network extracts features from the video frames of the original video data to obtain the feature vector of each frame. The feature vectors of all frames constitute a frame feature sequence. The frame feature sequence extracted by the convolutional neural network is input into the long short-term memory network to extract the video dynamic features of the frame feature sequence. Combine the video dynamic features with the network environment parameters to form a comprehensive feature vector; Using a decision maker in a compression level selection model to map the comprehensive feature vector to a probability distribution of compression levels, and outputting the compression level with the largest probability value as the optimal compression level of the original video data; The decision maker is a fully connected layer and a softmax activation function.

3. The deep learning video multi-level adaptive compression cross-layer transmission optimization method according to claim 2 is characterized in that: The specific method of obtaining the compression strategy corresponding to the optimal compression level is: Define compression level and compression parameters, the compression parameters include quantization parameter QP, intra-frame prediction mode Intra, inter-frame prediction mode Inter, entropy coding mode Entropy, and design a mapping table between compression level and compression parameters based on the compression parameters; The optimal compression level is used as an index to find the corresponding compression parameters in the mapping table, and the compression parameter combination is used as the compression strategy.

4. The deep learning video multi-level adaptive compression cross-layer transmission optimization method as claimed in claim 3 is characterized in that: The specific method of designing a transmission strategy selection model and generating a transmission strategy based on video quality parameters and network environment parameters is as follows: A state space is defined based on video quality parameters and network environment parameters, and the state includes application layer state s app , transport layer status s trans , network layer status s net ; Define an action space, the action includes application layer actions a app , transport layer action a trans 、Network layer action a net ; Define the reward function r, based on the application layer reward r app , transport layer reward r trans 、Network layer reward r net Design comprehensive reward functions; Design an intelligent agent system, wherein the intelligent agent includes an application layer agent, a transport layer agent and a network layer agent, and use a deep reinforcement learning model to build a transmission strategy selection model; At each time step, the action of the corresponding layer is generated according to the state and policy network corresponding to each layer observed by the current agent, and the actions generated by the agent are combined into a transmission strategy and output.

5. The deep learning video multi-level adaptive compression cross-layer transmission optimization method according to claim 4 is characterized in that: The design method of the reward function r is: The peak signal-to-noise ratio (PSNR) of the video is used as a measure of video quality at the application layer, based on the peak signal-to-noise ratio (PSNR) and the maximum peak signal-to-noise ratio (PSNR) at each time step. max Calculate the normalized value of video quality and count the number of times the video playback stalls count Stall duration duration Calculate the video fluency impact factor and obtain the video startup time. Calculate the video startup time impact factor and calculate the product of the video startup time impact factor, the normalized value of the video quality and the video fluency impact factor to obtain the application layer reward r. app ; At the transport layer, count the time interval delay between the sending and receiving of video frames and calculate its delay relative to the target delay. target The influence factor of transmission delay is calculated based on the difference, the proportion of video frames lost during transmission is counted, the influence factor of packet loss rate is calculated, and the product of the normalized value of video quality, the influence factor of transmission delay and the influence factor of packet loss rate is calculated to obtain the transport layer reward r trans ; Obtain the network bandwidth utilization at the network layer, calculate the influencing factor of the network bandwidth utilization based on the network bandwidth utilization and the target utilization, obtain the queuing delay in the network, calculate the influencing factor of the queuing delay based on the queuing delay and the target queuing delay, calculate the influencing factor of resource consumption according to the resource consumption of the transmission process, calculate the product of the influencing factor of the network bandwidth utilization, the influencing factor of the queuing delay, and the influencing factor of the resource consumption, and obtain the network layer reward r net ; A comprehensive reward function is calculated based on application layer rewards, transport layer rewards, and network layer rewards.

6. The deep learning video multi-level adaptive compression cross-layer transmission optimization method according to claim 5, characterized in that: The specific method of defining the global utility function, calculating the global utility value when assuming the compression strategy and the transmission strategy are implemented, and designing the optimization objective function is as follows: The global utility function U(s) is defined based on video quality parameters, transmission delay, bandwidth utilization, compression ratio and resource consumption. c ,s t ); Calculate the global utility function value when assuming that the compression strategy and the transmission strategy are executed, wherein the global utility function value includes the global utility function value of executing the compression strategy and the global utility function value of executing the transmission strategy And the global utility function value when the compression strategy and the transmission strategy are executed simultaneously Design an optimization objective function, take the current compression strategy and transmission strategy as input, maximize video quality, minimize transmission delay, maximize bandwidth utilization, and minimize total resource consumption as optimization goals, and output the optimized compression strategy s c ′ and transmission strategy s t ′.

7. The deep learning video multi-level adaptive compression cross-layer transmission optimization method according to claim 6 is characterized in that: The specific method of setting the conditional judgment logic is: The conditional judgment logic includes: if and Where θ1 is the first utility threshold, then the compression strategy before optimization is directly executed and transmission strategy To execute the optimized compression strategy s c ′ and optimized transmission strategy s t ′’s global utility function value; if and Where θ2 is the second utility threshold, then only the transmission strategy is optimized and the compression strategy is executed and optimized transmission strategies t ′; if and Then only the compression strategy is optimized and the optimized compression strategy s is executed c ′ and transmission strategy Otherwise, the compression strategy and transmission strategy are optimized at the same time, and the optimized compression strategy s is executed. c ′ and transmission strategy s t ′; The compression strategy and transmission strategy to be executed are selected based on the conditional judgment logic output to obtain the final strategy.

8. The deep learning video multi-level adaptive compression cross-layer transmission optimization method according to claim 7, characterized in that: The method for adjusting the first utility threshold and the second utility threshold is: respectively obtaining the average global utility function values ​​of the compression strategy and the transmission strategy before optimization, the compression strategy before optimization and the transmission strategy after optimization, and the compression strategy after optimization and the transmission strategy before optimization in the historical time at the current moment Based on the current first utility threshold θ1 and second utility threshold θ2, combined with the average global utility function value, the first utility threshold and the second utility threshold are adjusted to obtain the adjusted first utility threshold θ1′ and second utility threshold θ2′.

9. A deep learning video multi-level adaptive compression cross-layer transmission optimization system, which is used to implement the deep learning video multi-level adaptive compression cross-layer transmission optimization method according to any one of claims 1 to 8, characterized in that: include: The compression strategy generation module is used to design a compression level selection model based on deep learning, output the optimal compression level based on the original video data and network environment parameters, and obtain the compression strategy corresponding to the optimal compression level; A transmission strategy generation module is used to design a transmission strategy selection model and generate a transmission strategy based on video quality parameters and network environment parameters; A global optimization calculation module, used to define a global utility function, calculate a global utility value when assuming that a compression strategy and a transmission strategy are executed, and design an optimization objective function, wherein the optimization objective function inputs the compression strategy and the transmission strategy and outputs an optimized compression strategy and the transmission strategy; The final strategy output module is used to set the conditional judgment logic and select the final strategy based on the compression strategy and transmission strategy before and after optimization.

Images