Real-time super-resolution video stream transmission optimization method based on end-cloud cooperation

Abstract

The application is based on the real-time super-resolution video stream transmission optimization method of end-cloud cooperation, comprising: S1: collecting environment-agent state observation value as training data set D; S2: first initializing the reinforcement learning network model Q, then randomly sampling from the training data set D for iterative training, combining the real-time video stream user experience quality QoE model as the reward function r i Optimal code rate decision is obtained in a real application scene; S3: according to the optimal code rate decision, the corresponding video block file is requested to the client to be super-resolution reconstruction buffer, whether the video block file in the buffer needs to be reconstructed and the super-resolution reconstruction degree are judged through the super-resolution reconstruction scheduling mechanism, then the to-be-reconstructed video block file in the buffer is transmitted into the offline trained super-resolution reconstruction network model, the video quality super-resolution reconstruction process is carried out according to the reconstruction degree, finally the reconstructed video file is re-placed into the playing buffer, and waits to be rendered on the screen.

Description

Technical Field

[0001] This invention relates to the field of network transmission technology, and in particular to a method for optimizing real-time super-resolution video stream transmission based on end-to-cloud collaboration. Background Technology

[0002] With the rapid development of the internet information age and the upgrading of network infrastructure, real-time video streaming applications have become increasingly widespread, and users have higher expectations for the quality of video streaming services. However, current video streaming services still suffer from issues such as unstable network conditions and short client buffers, leading to a decline in user viewing experience. Stable and efficient real-time video streaming transmission technology is therefore crucial for improving the quality of real-time video streaming services.

[0003] Current real-time video streaming solutions mainly focus on designing adaptive bitrate control algorithms, but they still have many shortcomings: First, the rules are designed and formulated manually, which often leads to a small buffer size in the video player when real-time video streams have low latency requirements, making it difficult to respond to dynamic and complex network environments in a timely manner; Second, they focus more on controlling the video bitrate itself and ignore the utilization of edge computing resources.

[0004] Therefore, how to solve these problems from the underlying technical level and design a stable and efficient real-time video streaming optimization method is an urgent problem for researchers in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time super-resolution video stream transmission optimization method based on end-cloud collaboration, which realizes dynamic adaptation in complex network environments, makes full use of end-side computing resources, and can solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The real-time super-resolution video stream transmission optimization method based on edge-cloud collaboration includes the following steps:

[0008] S1: Collect environmental-agent state observations as training dataset D, including client playback state data and network environment state data;

[0009] S2: First, initialize the reinforcement learning network model Q, then randomly sample from the training dataset D described in step S1 for iterative training, combining it with the real-time video stream user experience quality (QoE) model as the reward function r. i The optimal network model is obtained through continuous reinforcement learning and interactive trial and error training, thereby obtaining the optimal bitrate decision in real application scenarios.

[0010] S3: Based on the optimal bitrate decision obtained in S2, request the corresponding video block file to the client's super-resolution reconstruction buffer. Use the super-resolution reconstruction scheduling mechanism to determine whether the video block file in the buffer needs to be reconstructed and the degree of super-resolution reconstruction. Then, pass the video block file to be reconstructed in the buffer into the offline trained super-resolution reconstruction network model. Perform the video quality super-resolution reconstruction process according to the pre-calculated reconstruction degree, i.e., the super-resolution ratio. Finally, put the reconstructed video file back into the playback buffer, waiting to be rendered and displayed on the screen.

[0011] A further improvement of the present invention is that, in step S1, the client playback status data includes the current buffer occupancy rate b. i Historical video segment download time list Requested video chunk bitrate list The bitrate d of the previous video chunk being downloaded i-1 .

[0012] A further improvement of the present invention is that, in step S1, the network environment status data includes the current real-time latency l i List of historical video chunk throughput

[0013] A further improvement of the present invention lies in the reward function r in step S2. i In other words, the Quality of User Experience (QoE) model is represented as:

[0014]

[0015] In the formula, n is the video block index, N is the total number of video blocks, and R... n Let q(R) be the bitrate of video block n. n To map video bitrate to the quality perceived by the user, T n L represents the buffering time caused by downloading video chunks n at a selected bitrate. n For the current real-time delay, S n α is the number of video chunks currently skipped during download, β is the video quality reward factor, β is the stuttering penalty factor, η is the real-time latency penalty factor, σ is the skip download penalty factor, and μ is the bitrate switching penalty factor.

[0016] A further improvement of the present invention is that S2 specifically includes the following steps:

[0017] Step S201: First, reset the gradient values ​​of the parameters of the Q network and initialize the model parameters of the Q network;

[0018] Step S202: Randomly sample the current state from the training dataset D described in step S1. The Q-network is iteratively initialized using the initialization data, then the video stream session process is initiated, the Q-network is used for data processing, and a corresponding bitrate probability distribution π is output. θ (s i ,a i )∈[0,1], thereby obtaining the code rate decision a i Then it interacts with the current player environment to obtain the reward value r for the current action. i and move to the next state.

[0019] Step S203: After the session ends, use the accumulated gradient value to update the parameters of the local network model, and then repeat step S202 continuously until the reward function value r of the network model is reached. i No more additions;

[0020] Step S204: Based on the optimal network model obtained during training, the optimal bitrate decision is obtained in the real application scenario.

[0021] A further improvement of the present invention is that S3 specifically includes the following steps:

[0022] Step S301: Based on the optimal bitrate decision obtained in step 2, the client requests the corresponding video chunk file to the client super-resolution reconstruction buffer and waits for the super-resolution reconstructor to determine whether to perform reconstruction work.

[0023] Step S302: By inputting the computing power information, buffer status, and resolution information of the video to be reconstructed from the client device, the super-resolution reconstruction controller will predict the super-resolution reconstruction time and select the reconstruction resolution decision value; then, it will compare the reconstruction time with the current computing power value of the device.

[0024] Step S303: Based on the target resolution decision result of step S302, perform super-resolution reconstruction using a real-time super-resolution network model, and put the super-resolution reconstructed video blocks into the playback buffer to complete the entire super-resolution reconstruction process.

[0025] A further improvement of the present invention is that, in step S302, the prediction of super-resolution reconstruction time is based on the offline calculation of all inference times of a basic device.

[0026] A further improvement of the present invention is that, in step S302, the current client device reconstruction time ψ i for:

[0027]

[0028] In the formula, ψ i Indicates the current client device rebuild time. Indicates the standard rebuild time of the reference equipment. Indicates the initial video block resolution. This represents the target reconstruction resolution, where μ is the current computing power of the client device. This refers to the computing power of the benchmark testing equipment.

[0029] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0030] This invention discloses a real-time super-resolution video stream transmission optimization method based on edge-cloud collaboration. The method includes collecting environment-agent (Env-Agent) state observations as a training dataset D, i.e., client playback state data and network environment state data; the network randomly samples from the training dataset D and obtains an optimal bitrate decision network model based on interaction data using deep reinforcement learning, thereby achieving the optimal bitrate decision in real-world application scenarios; then, based on the obtained optimal bitrate decision, it requests the corresponding video chunk files to the client's super-resolution reconstruction buffer, performs super-resolution reconstruction of the video chunk files through a super-resolution reconstruction scheduling mechanism, and finally renders the video onto the screen. This invention's real-time super-resolution video stream transmission optimization method based on edge-cloud collaboration achieves dynamic adaptation to complex network environments and fully utilizes edge-side computing resources. Attached Figure Description

[0031] Figure 1 A complete system architecture diagram for transmitting real-time video streams in a real network environment (the core of which adopts this method);

[0032] Figure 2 This provides a complete optimization method for real-time super-resolution video stream transmission based on edge-cloud collaboration.

[0033] Figure 3 For deep reinforcement learning A3C network models;

[0034] Figure 4 This provides a framework for real-time super-resolution reconstruction methods. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0036] like Figure 1As shown, this example demonstrates the process of transmitting a real-time video stream in a simulated real-world network environment. In this process, the server acts as the storage and forwarding center for the video source files. A Media Presentation Description (MPD) file is used to request video bitrate selection. Each video chunk is encoded and listed at multiple bitrate levels. The MPD file, an XML document, provides an index for available video chunks on the server, facilitating direct network request downloads. The network layer, as the basic network infrastructure, handles the interaction between the server and client. In the real world, there are different types of network scenarios, such as wired networks, wireless LANs, and mobile cellular networks. The method uses a simulated network environment, simulating various network throughput variations, jitter, latency, and packet loss by inputting different types of network trace files. The client, as the main video presentation end and method control platform, includes a video player, adaptive bitrate controller, super-resolution reconstructor, and status logger. It is responsible for requesting and downloading video files, super-resolution reconstruction of video buffer files, and finally displaying the video content on the screen. The multiple components on the client are interconnected and play a crucial role in the normal operation and method deployment of the final real-time video streaming application research platform. The server layer, network layer, and client layer jointly support the construction and operation of the edge-cloud collaborative real-time super-resolution video streaming transmission platform. Each component is responsible for its own task throughout the entire lifecycle of the real-time video streaming session and collaborates to complete information transmission, state transition, and system goal optimization.

[0037] The following example illustrates a real-time super-resolution video stream transmission optimization method based on edge-cloud collaboration. Assume the task scenario is as follows: Figure 1 As shown, this can be deployed in production using an HTTP server or an Nginx server, while the client can be based on the popular open-source real-time video streaming platform dash.js. The steps include:

[0038] Step 1: Collect environment-agent (Env-Agent) state observations as training dataset D, including client playback state data and network environment state data. Furthermore, the client playback state data specifically includes the current buffer occupancy rate b. i Historical video segment download time list Requested video chunk bitrate list The bitrate d of the previous video chunk downloaded i-1 The network environment status data specifically includes the current real-time latency. i Historical video block throughput list A dedicated data acquisition and analysis module can be set up here. Using this module, on the one hand, status information can be fed back to the reinforcement learning network model in step 2 in real time; on the other hand, performance analysis can be performed after the entire real-time video stream session ends, allowing for adjustments to method parameters and logic control accordingly.

[0039] Step 2: As Figure 2 As shown in the left half of the middle section, the reinforcement learning network model Q is first initialized, and then iteratively trained by randomly sampling from the training dataset D described in step S1, combined with the real-time video stream user experience quality QoE model as the reward function r. i The optimal network model is obtained through continuous reinforcement learning and interactive trial and error training, thereby achieving the optimal bitrate decision in real application scenarios.

[0040] The reward function r in step S2 i In other words, the Quality of User Experience (QoE) model is crucial in real-time video streaming applications because it relates to the objective quantitative evaluation of video transmission quality and has a direct impact on the optimization analysis and adjustment of methods and strategies for real-time video streaming. In QoE modeling, the key elements affecting the transmission quality of real-time video streams mainly include the following: (1) the overall quality of downloaded video chunks, i.e., the corresponding quality of the bitrate of the requested video chunks during the real-time video streaming session; (2) stuttering time statistics, the longer the stuttering time, the greater the negative impact on the viewing user; (3) real-time latency, calculated using the timestamp value from the generation of the original video file to the final video playback; (4) skipping video chunk downloads, i.e., using a mechanism to abandon request downloads to ensure the real-time latency target; (5) video bitrate switching statistics, used to measure the smoothness of video playback. By combining the above different QoE indicators, the final overall QoE model is calculated as follows:

[0041]

[0042] In the formula, n is the video block index, N is the total number of video blocks, and R... n Let q(R) be the bitrate of video block n. n To map video bitrate to the quality perceived by the user, T n L represents the stuttering time (in seconds) caused by downloading video chunks n at a selected bitrate. n S represents the current real-time latency (s). n α is the number of video chunks currently skipped during download, β is the video quality reward factor, β is the stuttering penalty factor, η is the real-time latency penalty factor, σ is the skip download penalty factor, and μ is the bitrate switching penalty factor.

[0043] The result of the neural network Q is as follows Figure 3As shown, it employs an A3C network structure, including an Actor network and a Critic network, for generating behavioral policies and providing reward evaluation guidance. The A3C network structure primarily accelerates the convergence process of the network model through policy learning. Its policy training method uses the policy gradient method. Furthermore, to prevent the reinforcement learning network model from prematurely converging to a suboptimal deterministic policy, a policy cross-entropy function is added to further explore the objective function of the entire model, thereby ensuring that the model can be trained to the optimal policy.

[0044] The specific training process for the Q-neural network model is as follows:

[0045] Step 21: First, reset the gradient values ​​of the Q network parameters and initialize the model parameters of the Q network.

[0046] Step 22: Randomly sample the current state from the training dataset D described in step S1. The Q-network is iteratively initialized using the initialization data, then the video stream session process is initiated, the Q-network is used for data processing, and a corresponding bitrate probability distribution π is output. θ (s i ,a i )∈[0,1], thereby obtaining the code rate decision a i Then it interacts with the current player environment to obtain the reward value r for the current action. i and move to the next state.

[0047] Step 23: After the session ends, use the accumulated gradient value to update the parameters of the local network model, and then repeat step S202 until the reward function value r of the network model is reached. i No more additions will be made.

[0048] Step S204: Based on the optimal network model obtained during training, the optimal bitrate decision is obtained in the real application scenario.

[0049] Step 3: As Figure 2 As shown in the right half, based on the optimal bitrate decision obtained in S2, the corresponding video block file is requested to the client's super-resolution reconstruction buffer. The super-resolution reconstruction scheduling mechanism determines whether the video block file in the buffer needs to be reconstructed and the degree of super-resolution reconstruction. Then, the video block file to be reconstructed in the buffer is passed to the offline trained super-resolution reconstruction network model. The video quality super-resolution reconstruction process is carried out according to the pre-calculated reconstruction degree, i.e., the super-resolution ratio. Finally, the reconstructed video file is put back into the playback buffer, waiting to be rendered and displayed on the screen.

[0050] Specifically, such as Figure 4The diagram illustrates the overall framework of this method, comprising a buffer partitioner, a super-resolution reconstruction controller, and a real-time super-resolution network reconstructor. These three modules work collaboratively to complete the entire super-resolution reconstruction decision-making process serially. Specifically, the client status and device information are passed to the super-resolution reconstruction controller to obtain the super-resolution reconstruction resolution decision information. Then, the real-time super-resolution network reconstruction is executed, placing the final high-quality video frames into the playback buffer. Furthermore, for the new video chunk request download process, parallel execution is employed; that is, the video chunk download process and the buffer video super-resolution reconstruction process are executed simultaneously, preventing further increases in the latency of the real-time video stream. The specific process is as follows:

[0051] Step 31: Based on the optimal bitrate decision obtained in Step 2, the client requests the corresponding video chunk file to the client's super-resolution reconstruction buffer, waiting for the super-resolution reconstructor to determine whether to perform reconstruction. Simultaneously, the video chunk to be reconstructed is extracted from the reconstruction buffer, super-resolution reconstruction is performed, and after reconstruction, it is directly placed into the playback buffer. To ensure that the normal playback of the real-time video stream is not affected, the size of the playback buffer needs to be limited. Based on real-world testing and experience, two video chunk durations are used as the playback buffer size, i.e., a safe and controllable playback area, to avoid real-time video stream stuttering events, thereby providing a good and reliable user viewing environment.

[0052] Step 32: By inputting the client device's computing power information, buffer status, and the resolution of the video to be reconstructed, the super-resolution reconstruction controller predicts the super-resolution reconstruction time, thereby selecting an appropriate reconstruction resolution decision value. For the super-resolution reconstruction control task, the main purpose is to ensure that the real-time super-resolution reconstruction time is strictly less than the size of the playback buffer; otherwise, the playback buffer may be exhausted before the super-resolution reconstruction process is complete, leading to video stuttering and affecting the user's overall viewing experience. Therefore, the core RA hub of this module is to accurately predict the super-resolution reconstruction time, thereby selecting an appropriate reconstruction resolution decision value. Specifically, the super-resolution reconstruction time is predicted by offline calculation of all inference times of a base device as a benchmark, then compared with the current device's computing power value to obtain its reconstruction time. The current client device reconstruction time ψ... i for:

[0053]

[0054] In the formula, ψ i Indicates the current client device rebuild time. Indicates the standard rebuild time of the reference equipment. Indicates the initial video block resolution. This represents the target reconstruction resolution, where μ is the current computing power of the client device. The computing power of the benchmark testing equipment;

[0055] Step 33: Based on the target resolution decision result in step S302, perform super-resolution reconstruction using a real-time super-resolution network model. Place the reconstructed video blocks into the playback buffer to complete the entire super-resolution reconstruction process. Finally, place the reconstructed video file into the playback buffer, waiting to be rendered and displayed on the screen.

[0056] Here, the real-time super-resolution network model in step 33 can adopt the FSRCNN network model. It is a lightweight super-resolution network model, which is redesigned based on the SRCNN network model and has better reconstruction time and quality performance. The network model mainly includes 5 parts: (1) Feature extraction: the original low-resolution image is used as input, and no two- or three-dimensional interpolation is needed to enlarge it to the target resolution; (2) Compression: shrinking layers are used to reduce the dimension of the input features, thereby greatly reducing the number of network parameters; (3) Nonlinear mapping: multiple 3*3 network layers are used to replace a single 5*5 network layer, so that a good trade-off between performance and network size can be achieved; (4) Enlargement: if high-resolution images are generated directly from low-dimensional features, the final reconstruction quality is very poor. Therefore, an expansion layer is used to increase the feature dimension and ensure the high-quality effect after super-resolution reconstruction; (5) Deconvolution: used for upsampling operations and aggregation of previous feature maps, and finally outputs a high-resolution reconstructed image.

[0057] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art who makes equivalent substitutions or modifications to the technical solution and inventive concept of the present invention within the scope of the technology disclosed in the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for real-time super-resolution video streaming optimization based on end-cloud collaboration, characterized in that, Comprising the following steps: S1: Collecting environment-agent state observation values as training data set D, including client playback state data and network environment state data; S2: first initialize the reinforcement learning network model Q, then randomly sample from the training data set D described in step S1 for iterative training, and combine the real-time video stream user experience quality QoE model as the reward function r i In the continuous reinforcement learning interaction trial and error training process, the optimal network model is obtained, so as to obtain the optimal code rate decision in the real application scene; S3: According to the optimal code rate decision obtained in S2, request the corresponding video block file into the client super-resolution reconstruction buffer, judge whether the video block file in the buffer needs to be reconstructed and the degree of super-resolution reconstruction through the super-resolution reconstruction scheduling mechanism, then input the video block file to be reconstructed in the buffer into the offline trained super-resolution reconstruction network model, according to the pre-computed reconstruction degree, that is, the super-resolution ratio, to perform the video quality super-resolution reconstruction process, and finally put the reconstructed video file back into the playback buffer, waiting to be rendered on the screen; Specifically comprising the following steps: Step S301: According to the optimal code rate decision obtained in step 2, the client requests the corresponding video block file into the client super-resolution reconstruction buffer, and waits for the super-resolution reconstruction device to judge whether to perform the reconstruction work; Step S302: By inputting the computing power information of the client device, the buffer state and the resolution information of the video to be reconstructed, the super-resolution reconstruction controller can predict the super-resolution reconstruction time, so as to select the reconstruction resolution decision value; Then compare the current device computing power value to obtain its reconstruction time; Step S303: According to the target resolution decision result of step S302, use the real-time super-resolution network model to perform super-resolution reconstruction, and put the video block after super-resolution reconstruction into the playback buffer, thus completing the entire super-resolution reconstruction process.

2. The method of claim 1, wherein the method further comprises: In step S1, the client playback status data includes the current buffer occupancy b i , a list of historical video chunk download times , a list of requested video chunk bitrates , and the bitrate d at which the previous video chunk was downloaded i-1 .

3. The method of claim 1, wherein the method further comprises: In step S1, the network environment state data includes the current real-time delay l i and a list of historical video chunk throughput rates 4. The end-cloud collaboration based real-time super-resolution video streaming optimization method of claim 1, wherein, The reward function r in step S2 i That is, the user experience quality QoE model is represented as: where n is the video chunk index, N is the total number of video chunks, R n is the code rate of video chunk n, q(R n ) is the mapping of video code rate to the perceived quality of users, T n is the stall time caused by downloading video chunk n at the selected code rate, L n is the current real-time delay, S n is the current number of skipped video chunks for download, α is the video quality reward factor, β is the stall penalty factor, η is the real-time delay penalty factor, σ is the skipped download penalty factor, and μ is the code rate switching penalty factor.

5. The end-cloud collaboration based real-time super-resolution video streaming optimization method according to claim 1, characterized in that, S2 specifically comprises the following steps: Step S201: First, reset the parameter gradient value of the Q network, and initialize the model parameters of the Q network; Step S202: randomly sampling a current state from the training data set D described in step S1 The Q network is iteratively initialized as initialization data, and then a video stream session process is started, data is processed by the Q network, and a corresponding code rate probability distribution π is output θ (s i , a i )∈[0,1], so as to obtain a code rate decision a i ; then interact with the current player environment to obtain the reward value r i of the current action behavior, and move to the next state Step S203: When the session ends, use the accumulated gradient value to update the parameters of the local network model, and then repeatedly step S202 until the reward function value r i No longer increase; Step S204: According to the optimal network model obtained in the training process, the optimal code rate decision in the real application scenario is obtained.

6. The end-cloud collaboration based real-time super-resolution video streaming optimization method of claim 1, wherein, In step S302, how to predict the super-resolution reconstruction time is to calculate all inference times of a basic device as a benchmark.

7. The end-cloud collaboration based real-time super-resolution video streaming optimization method of claim 1, wherein, In step S302, the current client device reconstructs the time ψ i is: wherein ψ i denotes the current client device reconstruction time, denotes the standard reconstruction time of the reference device, denotes the initial video tile resolution, denotes the target reconstruction resolution, μ is the current client device computing power, is the computing power of the reference test device.

Images