Multi-channel video super-resolution server processing method and processing system
Through TPU hardware acceleration and multi-agent reinforcement learning algorithm, dynamically adjusting video super-resolution processing parameters, solving the problem of insufficient computing resources in multiple high-definition video processing, improving video quality and efficiency, reducing user waiting time and server pressure .
Patent Information
- Application Number
- CN202411136745.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-29
AI Technical Summary
When processing multi-channel HD video, existing video super-resolution technology is difficult to take into account dynamic computing resources, resulting in a decrease in latency and user experience, and the versatility and cost of hardware accelerators limit large-scale applications.
采用基于TPU的可伸缩超分辨率处理方法,结合多智能体强化学习算法,动态调整视频超分顺序、重建分辨率级别和VSR模型配置,根据时变的网络带宽和计算资源状态,优化视频处理 Parameters and policies.
It significantly improves video processing efficiency and quality, meets users' demand for high-resolution video, reduces user waiting time and server concurrency pressure, and achieves optimal video quality and transmission efficiency.
Smart Images

Figure CN119052533A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of video processing and artificial intelligence, and more specifically, to a multi-channel video super-resolution server processing method and a processing system. Background Art
[0002] Ultra-high-definition video refers to the highest standard of video technology at this stage, the ITU-RBT.2020 standard, which was released by the International Telecommunication Union (ITU) in August 2012. It includes two specifications: 4K and 8K. The pixel of each frame of 4K ultra-high-definition video technology is 3840*2160, and the pixel of each frame of 8K ultra-high-definition video technology is 7680*4320. Compared with high-definition video, ultra-high-definition video not only has higher pixel parameters, but also has higher frame rate, bit depth, dynamic range and wider color gamut and other technical characteristics. It has finer image details, stronger information carrying capacity and wider application prospects, which can bring subversive, more appealing and immersive on-site experience to the audience.
[0003] With the rapid growth of video content consumption, users' demand for high-quality video experience is increasing. Whether it is online video streaming, real-time video conferencing, or high-definition video calls, users expect to obtain high-resolution video content with the shortest waiting time. However, existing video super-resolution technology (VSR) faces many challenges when dealing with real-time multi-channel video processing. First, traditional super-resolution algorithms usually rely on a large amount of computing resources. As a single server that processes multiple channels of video at the same time, its computing resources have a very high time-varying nature. Second, with the increase in the number of users, the server needs to process a large number of high-definition video requests at the same time. Although hardware accelerators such as GPUs can partially solve the computing bottleneck, they still cannot meet the real-time processing requirements of high concurrency and multiple channels of video. When processing multiple channels of high-definition video streams, existing technical solutions often find it difficult to take into account dynamic computing resources, resulting in unnecessary delays when processing multiple channels of video. Third, in actual applications, the diversity and complexity of the network environment increase the difficulty of video transmission. Factors such as network bandwidth, delay, and jitter will affect the transmission quality of the video and the user experience. Under the dynamic changes of the network environment, the existing technology cannot flexibly adjust the parameters and strategies of super-resolution processing, resulting in a decline in user experience. Fourth, although some hardware accelerators (such as GPU, FPGA, etc.) have improved the efficiency of super-resolution processing to a certain extent, their versatility and cost issues limit their large-scale applications.
[0004] Therefore, the present application provides a scalable super-resolution processing method and system based on TPU acceleration to solve the above problems. Summary of the invention
[0005] The purpose of this application is to provide a multi-channel video super-resolution server processing method and processing system to solve the problem that the existing server cannot adjust the parameters and strategies of super-resolution processing according to the time-varying network bandwidth and computing resources when performing multi-channel video super-resolution processing; this application combines TPU hardware acceleration and optimization algorithms to achieve efficient real-time multi-channel video processing and super-resolution processing, so as to improve the user's video experience, reduce waiting time, and reduce the concurrent pressure of the server.
[0006] The present application first provides a multi-channel video super-resolution server processing method, including: obtaining multiple low-resolution videos sent by a client, and caching the multiple low-resolution videos in a download buffer; obtaining client network bandwidth information and server computing resource information in real time, constructing an optimization problem with the goal of maximizing user experience quality, and solving the optimization problem based on multi-agent reinforcement learning to obtain decision information, the decision information including: video super-resolution order, video reconstruction resolution level and VSR model configuration; configuring a scalable super-resolution model in TPU hardware according to the decision information, loading low-resolution videos in sequence from the download buffer for super-resolution processing, and obtaining high-resolution videos; importing the high-resolution videos into a distribution buffer, and distributing the high-resolution videos according to client requests.
[0007] By adopting the above technical solution, by combining the hardware acceleration characteristics of the domestic TPU and the optimized algorithm, the efficiency and quality of the server's video super-resolution processing can be significantly improved to meet users' demand for high-resolution video; the server can dynamically adjust the parameters and strategies of super-resolution processing according to the changes in the client's network bandwidth status and its own TPU computing resource status to achieve optimal video quality and transmission efficiency, while reducing user waiting time and server concurrency pressure; in addition, considering the time-varying network bandwidth status and computing resource status of the server, the neural network size of the super-resolution processing model is designed to be scalable and dynamically determined, and through an adaptive algorithm based on reinforcement learning (DRL), with the goal of maximizing user experience quality, the optimal video super-resolution order, video reconstruction resolution level and VSR model configuration are jointly selected.
[0008] In one possible implementation, the scalable super-resolution model includes: an input layer, a scalable neural network layer and an output layer, the scalable neural network layer includes a plurality of residual convolution blocks determined by a decision information configuration, and the plurality of residual convolution blocks are connected to the output layer through a plurality of output paths.
[0009] In a possible implementation, an optimization problem is constructed with the goal of maximizing the user experience quality, including: dividing multiple low-resolution videos into N video blocks based on multi-agents, and determining the video super-resolution order, video reconstruction resolution level and VSR model configuration of each video block; calculating the computing power required by the server for super-resolution processing based on decision information, and calculating the computing power available for super-resolution processing based on server computing resources; calculating the overall user satisfaction based on decision information calculation, client network bandwidth and user weight, calculating the quality jitter based on the difference in bit rate between each two adjacent video blocks after super-resolution processing, and calculating the experience quality of a single user based on overall user satisfaction, quality jitter and penalty factor; calculating the TPU core utilization and memory usage of each video block based on decision information, and calculating the TPU server concurrency pressure based on the TPU core utilization, memory usage and TPU resource utilization efficiency of each video block; constructing an optimization objective function based on the experience quality of a single user and the TPU server concurrency pressure, and constructing a constraint function based on the computing power required by the server for super-resolution processing, the computing power available for super-resolution processing by the server and a probability constraint threshold to form an optimization problem.
[0010] In a possible implementation, the optimization objective function and constraint function are:
[0011] maxE{Q Total =E{QoE-Q TPU}
[0012] stP(μ δ <μ' δ )≤ε
[0013] Among them, Q Total QoE is the quality of experience for a single user. TPU is the concurrent pressure of TPU server, μ δ is the computing power available for super-resolution processing on the server, μ' δ is the computing power required by the server for super-resolution processing, and ε is the probability constraint threshold.
[0014] In one possible implementation, decision information is obtained by solving an optimization problem based on multi-agent reinforcement learning, including: treating the solution of the optimization problem as a Markov decision problem; determining the states that the agent can observe, including: video block states, network bandwidth states, computing resource states, and global system states; determining the actions that the agent can take in each state, including: video super-resolution order, video reconstruction resolution level, and VSR model configuration; defining a reward function based on the system state and the actions taken to guide the agent to learn behaviors that improve the quality of user experience; solving the Markov decision problem based on a deep network approximation and training a deep neural network model; inputting the real-time collected state information into the deep neural network model, and outputting the optimal decision information: video super-resolution order, video reconstruction resolution level, and VSR model configuration.
[0015] The present application also provides a multi-channel video super-resolution server processing system, including: a data acquisition unit, used to acquire multiple low-resolution videos sent by a client, and cache the multiple low-resolution videos in a download buffer; a decision generation unit, used to acquire client network bandwidth information and server computing resource information in real time, construct an optimization problem with the goal of maximizing user experience quality, and solve the optimization problem based on multi-agent reinforcement learning to obtain decision information, the decision information includes: video super-resolution order, video reconstruction resolution level and VSR model configuration; a super-resolution processing unit, used to configure a scalable super-resolution model in TPU hardware according to the decision information, load low-resolution videos in sequence from the download buffer for super-resolution processing, and obtain high-resolution videos; a data output unit, used to import high-resolution videos into a distribution buffer, and distribute high-resolution videos according to client requests.
[0016] In one possible implementation, in the super-resolution processing unit, the scalable super-resolution model includes: an input layer, a scalable neural network layer and an output layer, the scalable neural network layer includes a plurality of residual convolution blocks determined by the decision information configuration, and the plurality of residual convolution blocks are connected to the output layer through a plurality of output paths.
[0017] In a possible implementation, the decision generation unit includes an optimization problem construction unit, which includes: an agent configuration unit, which is used to divide multiple low-resolution videos into N video blocks based on multiple agents, and determine the video super-resolution order, video reconstruction resolution level and VSR model configuration of each video block; a computing power analysis unit, which is used to calculate the computing power required by the server for super-resolution processing based on decision information, and calculate the computing power available for super-resolution processing of the server based on server computing resources; an experience quality analysis unit, which is used to calculate the overall user satisfaction based on the decision information calculation, the client network bandwidth and the user weight, and calculate the overall user satisfaction based on the average user satisfaction of each two adjacent videos after super-resolution processing. The quality jitter is calculated by the difference in the bit rates of the video blocks, and the experience quality of a single user is calculated based on the overall user satisfaction, quality jitter and penalty factor; a concurrent pressure analysis unit is used to calculate the TPU core utilization and memory usage of each video block based on the decision information, and calculate the concurrent pressure of the TPU server based on the TPU core utilization, memory usage and TPU resource utilization efficiency of each video block; an optimization problem output unit is used to construct an optimization objective function based on the experience quality of a single user and the concurrent pressure of the TPU server, and to construct a constraint function based on the computing power required by the server for super-resolution processing, the computing power that the server can use for super-resolution processing and the probability constraint threshold to form an optimization problem.
[0018] In a possible implementation, in the optimization problem output unit, the optimization objective function and constraint function are:
[0019] maxE{Q Total =E{QoE-Q TPU}
[0020] stP(μ δ <μ' δ )≤ε
[0021] Among them, Q Total QoE is the quality of experience for a single user. TPU is the concurrent pressure of TPU server, μ δ is the computing power available for super-resolution processing on the server, μ' δ is the computing power required by the server for super-resolution processing, and ε is the probability constraint threshold.
[0022] In one possible implementation, the decision generation unit includes an optimization problem solving unit, and the optimization problem solving unit includes: an optimization problem conversion unit, which is used to regard the optimization problem solving as a Markov decision problem; a state determination unit, which is used to determine the state that the agent can observe, including: video block state, network bandwidth state, computing resource state and global system state; an action determination unit, which is used to determine the actions that the agent can take in each state, including: video super-resolution order, video reconstruction resolution level and VSR model configuration; a reward training unit, which is used to define a reward function to guide the agent to learn behaviors that improve the quality of user experience based on the system state and the actions taken; a model training unit, which is used to approximate the Markov decision problem based on a deep network and train a deep neural network model; a decision output unit, which is used to input the state information collected in real time into the deep neural network model and output the optimal decision information: video super-resolution order, video reconstruction resolution level and VSR model configuration.
[0023] Compared with the prior art, the present application has the following beneficial effects: through the hardware acceleration characteristics of TPU and the algorithm for resource allocation optimization, the video processing efficiency and quality are significantly improved to meet the user's demand for high-resolution video; the server can dynamically adjust the parameters and strategies of super-resolution processing according to changes in network and device status to achieve optimal video quality and transmission efficiency, while reducing user waiting time and server concurrency pressure; the scalability of model size includes two aspects: 1) the number of residual convolution blocks in the neural network can be dynamically configured, and 2) the output path of the early exit mechanism can be dynamically determined; in addition, in order to better capture the dynamic network and computing resource status of the server, this solution also uses a DRL-based adaptive algorithm to jointly select the optimal super-resolution order of each video, video reconstruction resolution level and VSR model configuration. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:
[0025] Figure 1 A flowchart of a multi-channel video super-resolution server processing method;
[0026] Figure 2 It is a schematic diagram of the structure of the scalable super-resolution model;
[0027] Figure 3 Schematic diagram of the structure of the multi-channel video super-resolution server processing system DETAILED DESCRIPTION
[0028] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with examples and drawings. The illustrative implementation scheme of the present application and its description are only used to explain the present application and are not intended to limit the present application.
[0029] See also Figure 1 As shown, Figure 1 The flowchart of the multi-channel video super-resolution server processing method includes: obtaining multiple low-resolution videos sent by the client, caching the multiple low-resolution videos in the download buffer; obtaining the client network bandwidth information and server computing resource information in real time, constructing an optimization problem with the goal of maximizing the user experience quality, solving the optimization problem based on multi-agent reinforcement learning to obtain decision information, and the decision information includes: video super-resolution order, video reconstruction resolution level and VSR model configuration; configuring the scalable super-resolution model in the TPU hardware according to the decision information, loading the low-resolution videos in sequence from the download buffer for super-resolution processing, and obtaining high-resolution videos; importing the high-resolution videos into the distribution buffer, and distributing the high-resolution videos according to the client request.
[0030] Specifically, the multi-channel video super-resolution server has the video super-resolution capability of adaptive bandwidth and computing resources, and may include a super-resolution module, a monitor module, a multi-agent decision module, and a video storage and distribution module. The client only needs to transmit the low-resolution video to the server to obtain the high-resolution video after high-resolution processing (reconstruction). Among them, the super-resolution module is responsible for receiving the low-resolution video, making decisions on the video reconstruction resolution level and the VSR model configuration through the multi-agent decision module, so that the TPU processor completes the reconstruction of the low-resolution video, outputs the high-resolution video to the buffer, and finally sends the high-resolution video to the client, thereby completing the entire server super-resolution process. Monitor module is responsible for monitoring the client's end network bandwidth information and server computing resource information, and transmitting it to the multi-agent decision module in real time. Multi-agent decision module is responsible for super-resolution order S of each video channel according to the different network bandwidths and time-varying computing resource information collected by the monitor module. i , video reconstruction resolution level R i and VSR model configuration M i Video storage and distribution module: mainly includes download buffer and distribution buffer, collects low-resolution videos transmitted by multiple clients, caches them in the download buffer, reduces the impact of network fluctuations on super-resolution inference speed, and imports the reconstructed high-resolution videos into the distribution buffer, and distributes them to the corresponding clients.
[0031] It should be noted that the improvement of this solution is that by combining the hardware acceleration characteristics of the domestic TPU and the optimized algorithm, the efficiency and quality of the server's video super-resolution processing can be significantly improved to meet the user's demand for high-resolution video. The server can dynamically adjust the parameters and strategies of super-resolution processing according to the changes in the client's network bandwidth status and its own TPU computing resource status to achieve the best video quality and transmission efficiency, while reducing user waiting time and server concurrency pressure. In addition, considering the time-varying network bandwidth status and computing resource status of the server, the neural network size of the super-resolution processing model is designed to be scalable and dynamically determined, and through an adaptive algorithm based on reinforcement learning (DRL), with the goal of maximizing the quality of user experience, the optimal video super-resolution order, video reconstruction resolution level and VSR model configuration are jointly selected.
[0032] In one possible implementation, see Figure 2 The scalable super-resolution model includes: an input layer, a scalable neural network layer and an output layer, the scalable neural network layer includes a plurality of residual convolution blocks determined by the decision information configuration, and the plurality of residual convolution blocks are connected to the output layer through a plurality of output paths.
[0033] Specifically, we provide scalable super-resolution models of various model sizes for computing resources at different times. At the same time, we configure multiple early-out output paths in each scalable super-resolution model for the real-time computing capabilities of edge devices to complete video super-resolution processing at different running speeds. The scalable neural network layer of the scalable super-resolution model is mainly composed of multiple residual convolution blocks. The scalability of the scalable super-resolution model is mainly reflected in 1) the number of residual convolution blocks in the neural network can be dynamically configured, and 2) the output path of the early-out mechanism can be dynamically determined.
[0034] About dynamically configuring the number of residual convolution blocks in the neural network. Configuring different numbers of residual convolution blocks in the scalable neural network layer will result in different model levels. The more residual convolution blocks there are, the higher the model level is, and the better the video quality after super-resolution processing (VSR) reconstruction is, but the corresponding reconstruction time is longer. VSR can pre-configure a set of scalable neural network layers for each possible resolution increase, so that the scalable neural network layer that best matches the computing resources can be selected during super-resolution processing.
[0035] About dynamically determining the output path of the early exit mechanism. Different output paths can be selected in the scalable neural network layer for early exit output. In order to further adapt to the time-varying computing power of the server, multiple early exit output paths are set in the scalable super-resolution model, that is, a direct connection from the intermediate block (residual convolution block) to the output layer can be added to the scalable super-resolution model, that is, the establishment of the early exit mechanism. The more residual convolution blocks added to the output path, the better the video quality after VSR reconstruction, but the corresponding reconstruction time is longer. Even if the number of residual convolution blocks in the output path is small, the user experience quality is still greatly improved compared to directly watching low-resolution videos.
[0036] In a possible implementation, an optimization problem is constructed with the goal of maximizing the user experience quality, including: dividing multiple low-resolution videos into N video blocks based on multi-agents, and determining the video super-resolution order, video reconstruction resolution level and VSR model configuration of each video block; calculating the computing power required by the server for super-resolution processing based on decision information, and calculating the computing power available for super-resolution processing based on server computing resources; calculating the overall user satisfaction based on decision information calculation, client network bandwidth and user weight, calculating the quality jitter based on the difference in bit rate between each two adjacent video blocks after super-resolution processing, and calculating the experience quality of a single user based on overall user satisfaction, quality jitter and penalty factor; calculating the TPU core utilization and memory usage of each video block based on decision information, and calculating the TPU server concurrency pressure based on the TPU core utilization, memory usage and TPU resource utilization efficiency of each video block; constructing an optimization objective function based on the experience quality of a single user and the TPU server concurrency pressure, and constructing a constraint function based on the computing power required by the server for super-resolution processing, the computing power available for super-resolution processing by the server and a probability constraint threshold to form an optimization problem.
[0037] Specifically, the performance of the server is mainly reflected in the super-resolution processing of each video and the transmission process of each reconstructed video from the server to the client. In order to ensure smooth video streaming services, the download buffer and distribution buffer should meet a series of conditions to avoid video interruption. Under the premise that the smoothness of the video streaming service is guaranteed, an optimization model is established to improve the quality of video transmission, reduce server concurrency pressure and reduce user waiting time. The decision variables are the super-resolution order of each video, the resolution level of the reconstructed video and the configuration of the VSR model. The following is a detailed introduction on how to construct the optimization problem.
[0038] A1: Start time of super-resolution processing. The server obtains the low-resolution video sent from the client and caches it in the download buffer. Then, the low-resolution video in the download buffer is reconstructed according to the super-resolution order decided by the multi-agent, and the high-resolution video block is output to the distribution buffer. For the low-resolution videos of n users in the download buffer, the multi-agent divides them into N video blocks according to their own decision results, and determines the video reconstruction resolution level R of each video block i = {1, 2, ..., N} i =(r i , r i ′), r i , r i ′∈R, the magnification of the i-th video block is (such as 2x, 4x, etc.).
[0039] set up represents all reconstruction pairs. Indicates the start time of reconstructing the i-th low-resolution video block.
[0040] The start time of reconstructing the i-th low-resolution video block can be defined as:
[0041]
[0042] in, represents the reconstruction start time of the i-th low-resolution video block; represents the reconstruction start time of the (i-1)th low-resolution video block; represents the reconstruction processing time of reconstructing the (i-1)th low-resolution video block into a high-resolution video block; represents the cache start time of the i-th high-resolution video block cached to the distribution buffer, and D represents all reconstruction pairs.
[0043] A2: Constraints and time of super-resolution processing under computational resource constraints. Since the available computational resources fluctuate over time, the reconstruction time is not constant. represents the baseline reconstruction time, which is used to estimate the reconstruction time from the low-resolution r δ to high resolution r′ δ The time length of T is the time required to reconstruct a video block δ.
[0044] The computing power μ that the server can use to reconstruct the video chunk δ δ It can be defined as:
[0045]
[0046] Assume that the computing power required by the server to reconstruct the video block δ is μ′ δIn order to ensure that the super-resolution module can perform super-resolution reconstruction as much as possible, the following constraints must be met:
[0047] P(μ δ <μ′ δ )≤ε1#(3)
[0048] Among them, ε1 is the probability constraint threshold.
[0049] The reconstruction processing time of the i-th low-resolution video block can be defined as:
[0050]
[0051] in, represents the reconstruction processing time of the i-th low-resolution video block; u i represents the computing power available on the server to reconstruct the i-th low-resolution video block; Indicates the baseline reconstruction time.
[0052] A3: Overall user satisfaction function Q U . Design an optimization function to maximize the overall user satisfaction. Optimization is achieved by setting different user weights for different users. The larger the user video, the more time it takes to reconstruct the inference, so the larger the weight assigned. The lower the user bandwidth, the smaller the magnification, and the smaller the weight assigned.
[0053] Defining user satisfaction U i As a trade-off between video quality and waiting time for each user, the following is the overall user satisfaction function Q U :
[0054]
[0055] Among them, w i is the user's weight (importance or priority), Z i is the magnification of the i-th video. wait (S i , R i , M i ) is the waiting time of the i-th video block, including the processing time and the time to transmit from the distribution buffer to the user end, S i is the super-resolution order of the i-th video block, R i is the magnification of the i-th video block, M i is the super-resolution model configuration of the i-th video block. i is the bandwidth of the i-th video block user. wait It can be expressed as the processing time T process With the transmission time T transmitThe sum of n users’ videos is divided into N video blocks.
[0056] Quality jitter function Q v The quality jitter between two adjacent video blocks can be defined as:
[0057]
[0058] Among them, B′ i represents the bit rate of the i-th video block after reconstruction; B′ i-1 N represents the bit rate of the i-1th video block after reconstruction; c Indicates the total number of video blocks received by the client during the entire system operation.
[0059] When the video is jittery, a penalty value will be obtained. The longer the delay, the greater the penalty value. The quality of experience (QoE) of a single user can be expressed as:
[0060] QoE = Q U -λQ v
[0061] A4: TPU server concurrent pressure function Q TPU Design a function to optimize and dynamically adjust the model size: Dynamically adjust the size and complexity of the scalable super-resolution model according to the current TPU load. For high load, you can reduce the number of residual convolution blocks and the number of channels of residual convolution blocks in the model, or choose a lower resolution output path.
[0062] Task Scheduling: Through intelligent scheduling, tasks with high processing requirements are staggered to avoid excessive occupation of TPU core and memory bandwidth in the same time period.
[0063] Priority allocation: Dynamically allocate TPU resources based on the importance of video streams and user bandwidth. Prioritize high-priority video streams and use simplified model configurations for low-priority video streams.
[0064]
[0065] Where N represents the total number of video blocks that need to be processed in the system, P i represents the concurrent pressure of the i-th video block on the TPU server, U Core (S i , R i , M i ), the i-th video block in the current model configuration M i TPU core utilization under Memory (S i , R i, M i ) is the memory usage of the i-th video block under the current configuration, Efficiency TPU It is the utilization efficiency of TPU resources, which measures the overall utilization efficiency of resources under normal circumstances. i is the super-resolution order of the i-th video block, R i is the magnification of the i-th video block, M i is the super-resolution model configuration of the i-th video block.
[0066] A5: Set the objective function. Establish an optimization model that can improve video transmission quality, reduce server concurrency pressure, and reduce user waiting time. The decision variables are the video super-resolution order, the reconstructed video resolution level, and the configuration of the VSR model. The objective function mainly includes A3 and A4, and the constraints mainly include A2:
[0067] In summary, the optimization problem can be expressed as the objective function and constraint function:
[0068] maxE{Q Total =E{QoE-Q TPU}
[0069] stP(μ δ <μ' δ )≤ε
[0070] Among them, Q Total QoE is the quality of experience for a single user. TPU is the concurrent pressure of the TPU server, μδ is the computing power of the server available for super-resolution processing, and μ' δ is the computing power required by the server for super-resolution processing, and ε is the probability constraint threshold.
[0071] In one possible implementation, decision information is obtained by solving an optimization problem based on multi-agent reinforcement learning, including: treating the solution of the optimization problem as a Markov decision problem; determining the states that the agent can observe, including: video block states, network bandwidth states, computing resource states, and global system states; determining the actions that the agent can take in each state, including: video super-resolution order, video reconstruction resolution level, and VSR model configuration; defining a reward function based on the system state and the actions taken to guide the agent to learn behaviors that improve the quality of user experience; solving the Markov decision problem based on a deep network approximation and training a deep neural network model; inputting the real-time collected state information into the deep neural network model, and outputting the optimal decision information: video super-resolution order, video reconstruction resolution level, and VSR model configuration.
[0072] Specifically, the reinforcement learning method is used to solve the optimization problem. The multi-agent supporting reinforcement learning is trained until the algorithm converges. The trained reinforcement learning algorithm is used to make decisions on the super-resolution order of each video, the video reconstruction resolution level, and the VSR model configuration. The following is a detailed introduction to the multi-agent reinforcement learning method for solving the optimization problem.
[0073] B1: The above optimization problem solving process is a Markov decision problem, and reinforcement learning does not require any data to be given in advance, but obtains learning information and updates model parameters by receiving rewards (feedback) from the environment for actions. Therefore, reinforcement learning can be used to solve the above optimization problem.
[0074] B2: State space S. It is the input used by the DRL agent to perceive the environment. It describes all the key information of the system at any time so that multiple agents can make decisions. It mainly includes 4 states: 1) Video stream state: S i is the super-resolution order of the i-th video block, R i is the magnification of the i-th video block, Z i is the current magnification (super-resolution multiple) of the i-th video stream, which affects the video clarity and computing load. i The current accumulated waiting time of the i-th video block (the time from request to completion) affects the user experience. 2) The user’s network status, B i The bandwidth of the user of the i-th video block determines the transmission rate and transmission delay of the network. 3) TPU resource status, U Core Current model configuration M i TPU core utilization under Memory Memory usage under the current configuration, Efficiency TPU It is the utilization efficiency of TPU resources, which measures the overall utilization efficiency of resources under normal circumstances. 4) Global system status, N is the number of video blocks processed in parallel in the current system, reflecting the load of the system, w i The priority weight of the i-th video block is used to determine the processing order and resource allocation, P TPU The current total concurrency pressure of the system, which measures the combined load of all video blocks on TPU resources.
[0075] B3: Action space. For each video block, the DRL agent must select the target video resolution for reconstruction based on the current state and select the most appropriate VSR model based on the currently available computing resources. The size of the VSR model and the target video resolution will affect the consumption of computing resources on the edge node. Therefore, the DRL agent in the VSR needs to make decisions including 1) the video reconstruction resolution level R i =(r i , r′ i), 2) Configuration M of SVSR model i (including model level m i and output path l i ), 3) The super-resolution order S of the i-th video block i The action vector of VSR when it starts to reconstruct the i-th video block can be expressed as
[0076] B4: Reward function. The DRL agent makes adaptive decisions based on the real-time system status collected. After the system performs an action, it will give the DRL agent an immediate reward. In order to enable users to obtain a better video service experience and achieve the goal of maximizing user QoE. The reward function is defined as
[0077] R i =B′ i -λ|B′ i -B′ i-1 |
[0078] Among them, B′ i represents the resolution of the i-th video block after reconstruction; |B′ i -B′ i-1 | represents the quality jitter between the i-th video block and the i-1-th video block; λ is the weight factor.
[0079] B5: Deep neural network model and model training. The Bellman equation for deep reinforcement learning can be written as:
[0080]
[0081] For Markov decision processes (MDPs) with a large number of states and actions, deep neural networks Q(s i , a i ;θ) to approximate Q(s i , a i ), θ is a parameter.
[0082] For the deep neural network Q(s i , a i ;θ) follows the DQN (Deep Q-Network) algorithm. DQN contains two neural networks. An evaluation network (eval_net) and a target network (target_net). The evaluation network is used to evaluate each action a in the current state. i The Q value of the network is used to select action a i , select the action a with the largest Q value i After that, the environment feeds back the reward and the next state s i+1 , that is (s i , ai , R i ,s i+1 ) and then store it. Input s to the target network i+1 , that is, the next environmental state, so that each action can be obtained through the network. i The Q value of the state s i Next action a i The reward R i , and the Bellman equation to calculate the target, and get Q(s i , a i ).
[0083] The DQN algorithm combines neural networks with Q-learing and uses neural networks to approximate the function Q(s i , a i ), the input is the state s i , the output is for each action a i The corresponding Q value is then used to select the action to be performed in the corresponding state according to the size of the Q value to complete the control.
[0084] Status i Input, get the Q value q(s) corresponding to all actions i , a i );Select the action a′ with the largest Q value i and execute; after execution, the environment changes and the reward R of the environment can be obtained i ; Use reward R i Update Q(s i , a′ i ), using the new Q(s i , a′ i )Update network parameters.
[0085] B6: After the model is trained, it can collect real-time status information as input and output the optimal control strategy, including: video reconstruction resolution level R i =(r i , r′ i ), the configuration M of the SVSR model i (including model level m i and output path l i ), the super-resolution order S of the i-th video block i .
[0086] It can be understood that the multi-channel video super-resolution server processing method provided by this solution significantly improves the video processing efficiency and quality through the hardware acceleration characteristics of TPU and the resource allocation optimization algorithm, meeting the user's demand for high-resolution video. The server can dynamically adjust the parameters and strategies of super-resolution processing according to changes in network and device status to achieve optimal video quality and transmission efficiency, while reducing user waiting time and server concurrency pressure. The scalability of model size includes two aspects: 1) the number of residual convolution blocks in the neural network can be dynamically configured, and 2) the output path of the early exit mechanism can be dynamically determined. In addition, in order to better capture the dynamic network and computing resource status of the server, this solution also uses a DRL-based adaptive algorithm to jointly select the optimal super-resolution order of each video, video reconstruction resolution level, and VSR model configuration.
[0087] See also Figure 3 As shown, Figure 3 The structure diagram of the multi-channel video super-resolution server processing system is shown in FIG. The system includes: a data acquisition unit, which is used to acquire multiple low-resolution videos sent by the client and cache the multiple low-resolution videos in the download buffer; a decision generation unit, which is used to acquire the client network bandwidth information and server computing resource information in real time, construct an optimization problem with the goal of maximizing the user experience quality, and solve the optimization problem based on multi-agent reinforcement learning to obtain decision information, which includes: video super-resolution order, video reconstruction resolution level and VSR model configuration; a super-resolution processing unit, which is used to configure the scalable super-resolution model in the TPU hardware according to the decision information, load the low-resolution videos in sequence from the download buffer for super-resolution processing, and obtain high-resolution videos; a data output unit, which is used to import the high-resolution videos into the distribution buffer and distribute the high-resolution videos according to the client request.
[0088] In one possible implementation, in the super-resolution processing unit, the scalable super-resolution model includes: an input layer, a scalable neural network layer and an output layer, the scalable neural network layer includes a plurality of residual convolution blocks determined by the decision information configuration, and the plurality of residual convolution blocks are connected to the output layer through a plurality of output paths.
[0089] In a possible implementation, the decision generation unit includes an optimization problem construction unit, which includes: an agent configuration unit, which is used to divide multiple low-resolution videos into N video blocks based on multiple agents, and determine the video super-resolution order, video reconstruction resolution level and VSR model configuration of each video block; a computing power analysis unit, which is used to calculate the computing power required by the server for super-resolution processing based on decision information, and calculate the computing power available for super-resolution processing of the server based on server computing resources; an experience quality analysis unit, which is used to calculate the overall user satisfaction based on the decision information calculation, the client network bandwidth and the user weight, and calculate the overall user satisfaction based on the average user satisfaction of each two adjacent videos after super-resolution processing. The quality jitter is calculated by the difference in the bit rates of the video blocks, and the experience quality of a single user is calculated based on the overall user satisfaction, quality jitter and penalty factor; a concurrent pressure analysis unit is used to calculate the TPU core utilization and memory usage of each video block based on the decision information, and calculate the concurrent pressure of the TPU server based on the TPU core utilization, memory usage and TPU resource utilization efficiency of each video block; an optimization problem output unit is used to construct an optimization objective function based on the experience quality of a single user and the concurrent pressure of the TPU server, and to construct a constraint function based on the computing power required by the server for super-resolution processing, the computing power that the server can use for super-resolution processing and the probability constraint threshold to form an optimization problem.
[0090] Furthermore, in the optimization problem output unit, the optimization objective function and constraint function are:
[0091] maxE{Q Total =E{QoE-Q TPU}
[0092] stP(μ δ <μ' δ )≤ε
[0093] Among them, Q Total QoE is the quality of experience for a single user. TPU is the concurrent pressure of TPU server, μ δ is the computing power available for super-resolution processing on the server, μ' δ is the computing power required by the server for super-resolution processing, and ε is the probability constraint threshold.
[0094] In one possible implementation, the decision generation unit includes an optimization problem solving unit, and the optimization problem solving unit includes: an optimization problem conversion unit, which is used to regard the optimization problem solving as a Markov decision problem; a state determination unit, which is used to determine the state that the agent can observe, including: video block state, network bandwidth state, computing resource state and global system state; an action determination unit, which is used to determine the actions that the agent can take in each state, including: video super-resolution order, video reconstruction resolution level and VSR model configuration; a reward training unit, which is used to define a reward function to guide the agent to learn behaviors that improve the quality of user experience based on the system state and the actions taken; a model training unit, which is used to approximate the Markov decision problem based on a deep network and train a deep neural network model; a decision output unit, which is used to input the state information collected in real time into the deep neural network model and output the optimal decision information: video super-resolution order, video reconstruction resolution level and VSR model configuration.
[0095] It can be understood that the system provided in this embodiment is used to implement the above method embodiments, corresponds one-to-one with the above method embodiments, and has the same effect, so it will not be elaborated herein.
[0096] It should be noted that the multi-channel video super-resolution server processing method and system provided by the present application dynamically adapts to time-varying bandwidth and computing resources through the configuration of the real-time decision super-resolution model to ensure the stability of super-resolution. At the same time, multi-agent decision-making aims to maximize the quality of user experience and jointly selects the optimal super-resolution order of each video, video reconstruction resolution level and VSR model configuration. And hardware acceleration is performed through the TPU chip. The TPU (tensor processing unit) chip, with its efficient computing power and low power consumption characteristics, can not only converge to the optimal solution faster when processing the same task, but also consumes only 25% of the power of the GPU chip.
[0097] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A multi-channel video super-resolution server processing method, characterized in that: include: Obtain multiple low-resolution videos sent by the client, and cache the multiple low-resolution videos in a download buffer; Obtain client network bandwidth information and server computing resource information in real time, construct an optimization problem with the goal of maximizing user experience quality, and solve the optimization problem based on multi-agent reinforcement learning to obtain decision information, which includes: video super-resolution order, video reconstruction resolution level, and VSR model configuration; configuring a scalable super-resolution model in the TPU hardware according to the decision information, sequentially loading low-resolution videos from the download buffer for super-resolution processing, and obtaining high-resolution videos; Import high-resolution video into the distribution buffer and distribute high-resolution video based on client requests.
2. A multi-channel video super-resolution server processing method according to claim 1, characterized in that: The scalable super-resolution model includes: an input layer, a scalable neural network layer and an output layer, the scalable neural network layer includes a plurality of residual convolution blocks determined by a decision information configuration, and the plurality of residual convolution blocks are connected to the output layer via a plurality of output paths.
3. A multi-channel video super-resolution server processing method according to claim 1, characterized in that: Construct an optimization problem with the goal of maximizing the quality of user experience, including: Based on multi-agent, multi-channel low-resolution videos are divided into N video blocks, and the video super-resolution order, video reconstruction resolution level and VSR model configuration of each video block are determined; calculating the computing power required by the server for super-resolution processing based on the decision information, and calculating the computing power available for the server for super-resolution processing based on the server computing resources; The overall user satisfaction is calculated based on the decision information, client network bandwidth and user weight. The quality jitter is calculated based on the bit rate difference between every two adjacent video blocks after super-resolution processing. The experience quality of a single user is calculated based on the overall user satisfaction, quality jitter and penalty factor. Calculate the TPU core utilization and memory usage of each video block based on the decision information, and calculate the TPU server concurrency pressure based on the TPU core utilization, memory usage and TPU resource utilization efficiency of each video block; An optimization objective function is constructed based on the experience quality of a single user and the concurrent pressure of the TPU server. A constraint function is constructed based on the computing power required by the server for super-resolution processing, the computing power available for super-resolution processing, and the probability constraint threshold to form an optimization problem.
4. A multi-channel video super-resolution server processing method according to claim 3, characterized in that: The optimization objective function and constraint function are: max{Q Total }=E{QoE-Q TPU } stP(μ δ <m' δ )≤e Among them, Q Total QoE is the quality of experience for a single user. TPU is the concurrent pressure of TPU server, μ δ is the computing power available for super-resolution processing on the server, μ' δ is the computing power required by the server for super-resolution processing, and ε is the probability constraint threshold.
5. A multi-channel video super-resolution server processing method according to claim 1, characterized in that: The optimization problem is solved based on multi-agent reinforcement learning to obtain decision information, including: Treat the optimization problem as a Markov decision problem; Determine the states that the agent can observe, including: video block state, network bandwidth state, computing resource state, and global system state; Determine the actions that the agent can take in each state, including: video super-resolution order, video reconstruction resolution level, and VSR model configuration; Based on the system state and the actions taken, define a reward function to guide the agent to learn behaviors that improve the quality of user experience; Solve the Markov decision problem based on deep network approximation and train deep neural network models; The real-time collected state information is input into the deep neural network model, and the optimal decision information is output: video super-resolution order, video reconstruction resolution level and VSR model configuration.
6. A multi-channel video super-resolution server processing system, characterized in that: include: A data acquisition unit, used for acquiring multiple low-resolution videos sent by a client, and caching the multiple low-resolution videos into a download buffer; A decision generation unit is used to obtain client network bandwidth information and server computing resource information in real time, construct an optimization problem with the goal of maximizing user experience quality, and solve the optimization problem based on multi-agent reinforcement learning to obtain decision information, wherein the decision information includes: video super-resolution order, video reconstruction resolution level, and VSR model configuration; A super-resolution processing unit, configured to configure a scalable super-resolution model in the TPU hardware according to the decision information, and sequentially load low-resolution videos from the download buffer for super-resolution processing to obtain high-resolution videos; The data output unit is used to import the high-resolution video into the distribution buffer and distribute the high-resolution video according to the client request.
7. A multi-channel video super-resolution server processing system according to claim 6, characterized in that: In the super-resolution processing unit, the scalable super-resolution model includes: an input layer, a scalable neural network layer and an output layer, the scalable neural network layer includes a plurality of residual convolution blocks determined by the decision information configuration, and the plurality of residual convolution blocks are connected to the output layer through a plurality of output paths.
8. The multi-channel video super-resolution server processing system according to claim 6, characterized in that: The decision generation unit includes an optimization problem construction unit, and the optimization problem construction unit includes: An agent configuration unit, used for dividing multiple low-resolution videos into N video blocks based on multiple agents, and determining a video super-resolution order, a video reconstruction resolution level and a VSR model configuration for each video block; A computing capacity analysis unit, used to calculate the computing capacity required by the server for super-resolution processing based on the decision information, and to calculate the computing capacity available for super-resolution processing by the server based on the server computing resources; An experience quality analysis unit, used to calculate overall user satisfaction based on decision information calculation, client network bandwidth and user weight, calculate quality jitter based on the difference in bit rates of every two adjacent video blocks after super-resolution processing, and calculate the experience quality of a single user based on overall user satisfaction, quality jitter and penalty factor; A concurrent pressure analysis unit, used to calculate the TPU core utilization and memory usage of each video block based on the decision information, and calculate the concurrent pressure of the TPU server based on the TPU core utilization, memory usage and TPU resource utilization efficiency of each video block; The optimization problem output unit is used to construct an optimization objective function based on the experience quality of a single user and the concurrent pressure of the TPU server, and to construct a constraint function based on the computing power required by the server for super-resolution processing, the computing power available for super-resolution processing by the server, and the probability constraint threshold to form an optimization problem.
9. A multi-channel video super-resolution server processing system according to claim 8, characterized in that: In the optimization problem output unit, the optimization objective function and constraint function are: max{Q Total }=E{QoE-Q TPU } stP(μ δ <m' δ )≤e Among them, Q Total QoE is the quality of experience for a single user. TPU is the concurrent pressure of TPU server, μ δ is the computing power available for super-resolution processing on the server, μ' δ is the computing power required by the server for super-resolution processing, and ε is the probability constraint threshold.
10. The multi-channel video super-resolution server processing system according to claim 6, characterized in that: The decision making unit includes an optimization problem solving unit, and the optimization problem solving unit includes: An optimization problem conversion unit, used to regard the solution of the optimization problem as a Markov decision problem; A state determination unit, used to determine the states that the agent can observe, including: video block states, network bandwidth states, computing resource states, and global system states; The action determination unit is used to determine the actions that the agent can take in each state, including: video super-resolution order, video reconstruction resolution level, and VSR model configuration; The reward training unit is used to define the reward function to guide the agent to learn behaviors that improve the quality of user experience based on the system state and the actions taken; Model training unit, used to solve Markov decision problems based on deep network approximation and train deep neural network models; The decision output unit is used to input the real-time collected status information into the deep neural network model and output the optimal decision information: video super-resolution order, video reconstruction resolution level and VSR model configuration.
Citation Information
Cited By
Single-channel high-resolution output circuit and data processing method thereof
CN119722464A
Single-channel high-resolution output circuit and data processing method thereof
CN119722464B
Data collaborative operation method and system of edge computing server
CN121262192A
