A video interpolation task scheduling method based on deep reinforcement learning

Abstract

The embodiment of the present invention provides a method for scheduling video interpolation tasks based on deep reinforcement learning, which belongs to the field of data processing technology, and specifically includes: step 1, decoding video data to obtain different video frame groups and dividing interpolation algorithm types to obtain video frame group feature information vectors; step 2, obtaining computing power equipment performance index vectors; step 3, based on the video frame group feature information vectors and computing power equipment performance index vectors, using reinforcement learning to allocate computing power equipment to different video frame groups; step 4, obtaining interpolation results from each computing power device, inserting the interpolation results between video frames, obtaining a complete interpolation video group, encoding it, and obtaining interpolation video data. Through the scheme of the present invention, the efficiency, accuracy and adaptability of task scheduling are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a video interpolation task scheduling method based on deep reinforcement learning. Background Art

[0002] At present, the video interpolation algorithm is a technology used to improve the frame rate and smoothness of a video by generating an intermediate frame between two adjacent frames. The video interpolation algorithm can be applied to video compression, video transcoding, video editing, video enhancement and other fields, and has a wide range of application value.

[0003] In real-time video interpolation tasks, it is usually necessary to find a balance between the time and quality of video interpolation. Currently, there are still the following major problems in video interpolation task scheduling:

[0004] (1) There is a lack of adaptive analysis of video content and scenes, and it is impossible to select appropriate video interpolation algorithms and parameters based on the characteristics and requirements of the video.

[0005] (2) There is a lack of dynamic scheduling of video interpolation tasks and computing resources, and it is impossible to adjust the allocation of video interpolation tasks in a timely manner according to the progress and quality of video interpolation, as well as the status and performance of computing resources.

[0006] It can be seen that there is an urgent need for a video interpolation task scheduling method based on deep reinforcement learning with high scheduling efficiency, accuracy and adaptability. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides a video interpolation task scheduling method based on deep reinforcement learning, which at least partially solves the problems of poor scheduling efficiency, accuracy and adaptability in the prior art.

[0008] The embodiment of the present invention provides a video frame insertion task scheduling method based on deep reinforcement learning, including:

[0009] Step 1, decoding the video data to obtain different video frame groups and classifying them by interpolation algorithm type to obtain a video frame group feature information vector;

[0010] Step 2, obtaining the performance index vector of the computing power equipment;

[0011] Step 3: Based on the feature information vector of the video frame group and the performance index vector of the computing power device, reinforcement learning is used to allocate computing power devices to different video frame groups;

[0012] Step 4: Obtain interpolation results from each computing device, insert the interpolation results between video frames to obtain a complete interpolation video group, perform encoding operations on it, and obtain interpolation video data.

[0013] According to a specific implementation of the embodiment of the present invention, step 1 specifically includes:

[0014] Step 1.1, get video data , is the total number of video frames in the video data, and every two consecutive video frames are divided into a video frame group to obtain a video frame group ,in ;

[0015] Step 1.2, by measuring the motion intensity between two frames, we get The amount of inter-frame motion for a video frame group:

[0016]

[0017] in, and Respectively represent The first video frame group Frame and frame, and are the height and width of the video frame, are the three color channels of the video frame, , Indicates Frame at coordinates The The pixel value of each channel;

[0018] Step 1.3, calculate the Expected payload of video frame groups :

[0019]

[0020] in, is the resolution of the video frame, expressed as , is the inter-frame motion, is the quantified value of the computational complexity of the interpolation algorithm, Respectively represent the weight coefficients of different items;

[0021] Step 1.4, The video frame groups correspond to different types Video frame interpolation algorithm ;

[0022] Step 1.5, Definition is the video frame group feature information vector, which contains the features of each video frame group:

[0023]

[0024] in, is the inter-frame motion, for the expected load.

[0025] According to a specific implementation of the embodiment of the present invention, step 2 specifically includes:

[0026] Step 2.1, evaluate the computing power of the computing equipment:

[0027]

[0028] in, It is a computing device Maximum computing power without load, It's time The computing power equipment load factor, It is a computing device Load;

[0029] Step 2.2: Calculate storage space based on current used space and available space :

[0030]

[0031] in, It is a computing device Total storage space, It's time Storage space used;

[0032] Step 2.3, calculate network bandwidth :

[0033]

[0034] in, It is a computing device The maximum network bandwidth when there is no congestion, It's time The network congestion factor, It's time The number of network connections or data transfer volume;

[0035] Step 2.4: Calculate the energy consumption based on the device power consumption and task execution time :

[0036]

[0037] in, It's time The energy consumption coefficient, It is a computing device Power consumption under standard conditions, It's time The task execution time;

[0038] Step 2.5, for each computing device , define a performance indicator vector that varies over time ,in Indicates time:

[0039]

[0040] in, They represent the computing power, storage space, network bandwidth and energy consumption at time t respectively.

[0041] According to a specific implementation of the embodiment of the present invention, step 3 specifically includes:

[0042] Step 3.1: Define the state based on the video frame group feature information vector and the computing device performance index vector ;

[0043] Step 3.2, define the action space for:

[0044] ;

[0045] in, is the number of video frame groups, is the number of available computing devices, each action Indicates that the The video frame groups are assigned to the The task scheduling problem of computing devices;

[0046] Step 3.3, express the task scheduling problem as:

[0047]

[0048]

[0049]

[0050]

[0051] in, is the actual processing time, is the expected processing time, is a positive adjustment parameter used to control the effect of time difference on reward. is the video frame after interpolation. SSIM is used to measure the structural similarity between two images. The closer the SSIM value is to 1, the higher the quality. Indicates that at a given time, the task Offloading to computing devices On, and use Indicates the task Has been successfully assigned to the computing device On, constraint Indicates that each task can only be scheduled to one computing device, and each task can only be scheduled once. Indicates that the computing power, storage space, network bandwidth, and energy consumption allocated to each task do not exceed the maximum value of the computing device. It is The maximum computing power of each computing device, It is The maximum storage space of a computing device, It is The maximum network bandwidth of each computing device, It is The maximum set energy consumption of each computing device;

[0052] Step 3.4, design reward function for:

[0053] ;

[0054] in, and is the weight coefficient, is the actual processing time, is the expected processing time, is a positive adjustment parameter used to control the effect of time difference on reward. is the video frame after interpolation;

[0055] Step 3.5, in the state Next action The expected return is used as the Q value and updated using the update formula, which is

[0056]

[0057] in, is the learning rate, is the discount factor, is the state at time step t, is the action at time step t, is the time step The state of is the next state Take an action from the set of all possible actions, is the next state The best action to choose The corresponding Q value;

[0058]

[0059] Step 3.6, derive strategy from Q value :

[0060] .

[0061] According to a specific implementation of the embodiment of the present invention, step 4 specifically includes:

[0062] Step 4.1, obtaining the intermediate frame and the corresponding video frame group sequence number from the computing device, and inserting the obtained intermediate frame between the video frames according to the video frame group sequence number to form a video interpolation frame group;

[0063] Step 4.2, encoding the video interpolation frame group to form interpolation frame video data.

[0064] The video interpolation task scheduling scheme based on deep reinforcement learning in the embodiment of the present invention includes: step 1, decoding the video data to obtain different video frame groups and dividing the interpolation algorithm types to obtain the video frame group feature information vector; step 2, obtaining the computing power device performance index vector; step 3, based on the video frame group feature information vector and the computing power device performance index vector, using reinforcement learning to allocate computing power devices to different video frame groups; step 4, obtaining the interpolation result from each computing power device, inserting the interpolation result between the video frames to obtain a complete interpolation video group, encoding it, and obtaining interpolation video data.

[0065] The beneficial effects of the embodiments of the present invention are as follows: through the scheme of the present invention, appropriate video interpolation algorithms and parameters are selected according to the characteristics and requirements of the video, and the allocation of video interpolation tasks is adjusted in a timely manner according to the progress and quality of the video interpolation, as well as the status and performance of computing resources, thereby improving the efficiency, accuracy and adaptability of task scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 A flowchart of a video interpolation task scheduling method based on deep reinforcement learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0069] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0070] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present invention, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0071] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0072] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0073] An embodiment of the present invention provides a video frame insertion task scheduling method based on deep reinforcement learning, which can be applied to video frame processing in scenarios such as video compression or video editing.

[0074] See also Figure 1 , is a flow chart of a method for scheduling video frame insertion tasks based on deep reinforcement learning provided by an embodiment of the present invention. Figure 1As shown, the method mainly comprises the following steps:

[0075] Step 1, decoding the video data to obtain different video frame groups and classifying them by interpolation algorithm type to obtain a video frame group feature information vector;

[0076] In specific implementation, the original video data can be decoded to obtain video frame groups, the inter-frame motion intensity of each video frame group can be calculated, and then the amount of interpolation calculation for each video frame group can be evaluated. The feature vector of each video frame group can be obtained. The specific processing process can be as follows:

[0077] Video data is acquired, the video data is decoded, a plurality of video frames are acquired, and a group of the to-be-processed video frame groups and video frame serial numbers corresponding to the to-be-processed video frame groups are generated according to every two consecutive video frames.

[0078] 1.1 Get video data , is the total number of video frames in the video data. Every two consecutive video frames are divided into a video frame group, and the video frame group is obtained. ,in .

[0079] 1.2 By measuring the motion intensity between two frames, we can get Inter-frame motion evaluation of video frame groups:

[0080]

[0081] in, and Respectively represent The first video frame group Frame and frame. and are the height and width of the video frame respectively. are the three color channels of the video frame, . Indicates Frame at coordinates The The pixel value of each channel.

[0082] 1.3 Calculate the Expected payload of video frame groups :

[0083]

[0084] in, is the resolution of the video frame, expressed as . is the amount of motion between frames. is the quantified value of the computational complexity of the interpolation algorithm, They represent the weight coefficients of different items respectively.

[0085] 1.4 The video frame groups correspond to different types Video frame interpolation algorithm For example, the video interpolation algorithm may include traditional linear interpolation, motion compensation interpolation based on optical flow method, and deep learning-based method. The size of the type is divided into smaller Corresponding to traditional linear interpolation, the intermediate size Corresponding to motion compensation interpolation based on optical flow method, the larger Corresponding to the deep learning based method.

[0086] 1.5 Definitions is a vector containing the features of each video frame group:

[0087]

[0088] in, is the inter-frame motion, is the expected load, The video frame interpolation algorithm used.

[0089] Step 2, obtaining the performance index vector of the computing power equipment;

[0090] In specific implementation, the computing power, storage space, network bandwidth, energy consumption and other factors of each computing device can be considered to more accurately describe the status of the computing device and provide more abundant information for the reinforcement learning model to make effective task scheduling decisions. The specific process can be as follows:

[0091] 2.1 Computing Power It will change over time due to factors such as the load of the device. The following formula is used to measure the computing power of the device:

[0092]

[0093] in, Yes Equipment Maximum computing power without load, It's time The equipment load factor, Yes Equipment of load.

[0094] 2.2 Storage Space It can be calculated based on the currently used and available space. The change in storage space is expressed as follows:

[0095]

[0096] in, Yes Equipment Total storage space, It's time Storage space used.

[0097] 2.3 Network Bandwidth It may change due to network congestion or other external factors. The change of network bandwidth is described as follows:

[0098]

[0099] in, Yes Equipment The maximum network bandwidth when there is no congestion, It's time The network congestion factor, It's time The number of network connections or data transfer volume.

[0100] 2.4 Energy consumption It can be estimated based on the power consumption of the device and the task execution time. The energy consumption calculation formula can be expressed as:

[0101]

[0102] in, It's time The energy consumption coefficient, Yes Equipment Power consumption under standard conditions, It's time The task execution time.

[0103] 2.5 Performance indicators of computing equipment

[0104] For each computing device , defining a time-varying performance vector ,in Indicates time:

[0105]

[0106] in, They represent the computing power, storage space, network bandwidth and energy consumption at time t respectively.

[0107] Step 3: Based on the feature information vector of the video frame group and the performance index vector of the computing power device, reinforcement learning is used to allocate computing power devices to different video frame groups;

[0108] In specific implementation, deep reinforcement learning is used to allocate video frame group tasks to ensure that each video frame group can minimize time consumption and maximize interpolation quality. The specific process can be as follows:

[0109] 3.1 Status It can be represented as a vector that contains all the above features:

[0110]

[0111] in, is a vector containing the characteristics of a group of video frames, Every computing device exist A vector of performance indicators over time.

[0112] 3.2 Action involves assigning video frame groups to computing devices, action space Defined as:

[0113]

[0114] in, is the number of video frame groups, is the number of available computing devices. Each action Indicates that the The video frame groups are assigned to the A computing device.

[0115] 3.3 In order to make the best use of available computing resources, an important optimization problem is proposed, namely the delay minimization problem. Its goal is to minimize the total delay in the task scheduling process. This task scheduling problem is expressed as:

[0116]

[0117]

[0118]

[0119]

[0120] in, is the actual processing time, is the expected processing time, is a positive adjustment parameter used to control the effect of time difference on reward. is the video frame after interpolation. SSIM is used to measure the structural similarity between two images. The closer the SSIM value is to 1, the higher the quality. Indicates that at a given time, the task Offloading to computing devices On, and use Indicates the task Has been successfully assigned to the computing device On. Constraint Indicates that each task can only be scheduled to one computing device, and each task can only be scheduled once. Constraint The computing power, storage space, network bandwidth, and energy consumption allocated to each task are limited to ensure that the maximum value of the computing power device is not exceeded. It is The maximum computing power of each computing device, It is The maximum storage space of a computing device, It is The maximum network bandwidth of each computing device, It is The maximum set energy consumption of a computing device.

[0121] 3.4 The reward function should reflect the balance between interpolation time and quality. Designed for:

[0122]

[0123] in, and is the weight coefficient, is the actual processing time, is the expected processing time, is a positive tuning parameter that controls the extent to which time differences affect rewards. The video frame after interpolation.

[0124] 3.5Q value Indicates that an action is performed in the state The expected return. The update formula of Q value is:

[0125]

[0126] in, is the learning rate, which determines how quickly new information overwrites old information. is the discount factor, which determines the present value of future rewards. is the state at time step t, is the action at time step t, is the time step The state of is the next state Take an action from the set of all possible actions, is the next state The best action to choose The corresponding Q value.

[0127] 3.6 Derive the strategy from the Q value, and then improve the total return by optimizing the strategy. Strategy It can be derived from the Q-values ​​so that in each state the action with the highest Q-value is chosen:

[0128] .

[0129] Step 4: Obtain interpolation results from each computing device, insert the interpolation results between video frames to obtain a complete interpolation video group, perform encoding operations on it, and obtain interpolation video data.

[0130] In specific implementation, after computing power devices are assigned to different video frame groups, the interpolation results can be obtained from each computing power device, and then the intermediate frames and the corresponding video frame group numbers are obtained from the computing power devices. According to the video frame group numbers, the obtained intermediate frames are inserted between the video frames to form a video interpolation frame group, and then the obtained video interpolation frame group is encoded to form interpolation video data.

[0131] The video interpolation task scheduling method based on deep reinforcement learning provided in this embodiment can realize accurate task allocation according to the feature vector of each video frame group and the performance index of the computing power device through deep reinforcement learning, which means that each video frame group is assigned to the most suitable device for processing, thereby minimizing time consumption; by calculating the inter-frame motion intensity and evaluating the size of the interpolation calculation amount of each video frame group, the use efficiency of the computing power device can be effectively optimized. This optimization can help save computing resources and energy consumption; deep reinforcement learning not only focuses on minimizing time consumption, but also maximizes the quality of interpolation. This means that the smoothness and visual quality of the interpolated video may be significantly improved, making the final interpolated video more in line with user expectations and needs; evaluate the computing power, storage space, network bandwidth, energy consumption and other performance indicators of the computing power device to ensure that comprehensive factors are considered in task allocation, thereby improving the overall efficiency and performance of the system; the entire method uses automation technology to achieve a smooth processing flow from decoding the video to inserting the interpolation result. This automation not only improves the processing speed, but also reduces the possibility of human error. In summary, this method effectively improves the efficiency, accuracy, and adaptability of the video interpolation process by accurately allocating tasks, optimizing the use of computing resources, improving interpolation quality, and comprehensively considering performance indicators.

[0132] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof.

[0133] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A video interpolation task scheduling method based on deep reinforcement learning, characterized in that: include: Step 1, decoding the video data to obtain different video frame groups and classifying them by interpolation algorithm type to obtain a video frame group feature information vector; The step 1 specifically includes: Step 1.1, get video data , is the total number of video frames in the video data, and every two consecutive video frames are divided into a video frame group to obtain a video frame group ,in ; Step 1.2, by measuring the motion intensity between two frames, we get The amount of inter-frame motion for a video frame group: in, and Respectively represent The first video frame group Frame and frame, and are the height and width of the video frame, are the three color channels of the video frame, , Indicates Frame at coordinates The The pixel value of each channel; Step 1.3, calculate the Expected payload of video frame groups : in, is the resolution of the video frame, expressed as , is the inter-frame motion, is the quantified value of the computational complexity of the interpolation algorithm, Respectively represent the weight coefficients of different items; Step 1.4, The video frame groups correspond to different types Video frame interpolation algorithm ; Step 1.5, Definition is the video frame group feature information vector, which contains the features of each video frame group: in, is the inter-frame motion, is the expected load; Step 2, obtaining the performance index vector of the computing power equipment; Step 3: Based on the feature information vector of the video frame group and the performance index vector of the computing power device, reinforcement learning is used to allocate computing power devices to different video frame groups; Step 4: Obtain interpolation results from each computing device, insert the interpolation results between video frames to obtain a complete interpolation video group, perform encoding operations on it, and obtain interpolation video data.

2. The method according to claim 1, characterized in that , the step 2 specifically includes: Step 2.1, evaluate the computing power of the computing equipment: in, It is a computing device Maximum computing power without load, It's time The computing power equipment load factor, It is a computing device Load; Step 2.2: Calculate storage space based on current used space and available space : in, It is a computing device Total storage space, It's time Storage space used; Step 2.3, calculate network bandwidth : in, It is a computing device The maximum network bandwidth when there is no congestion, It's time The network congestion factor, It's time The number of network connections or data transfer volume; Step 2.4: Calculate the energy consumption based on the device power consumption and task execution time : in, It's time The energy consumption coefficient, It is a computing device Power consumption under standard conditions, It's time The task execution time; Step 2.5, for each computing device , define a performance indicator vector that varies over time ,in Indicates time: in, They represent the computing power, storage space, network bandwidth and energy consumption at time t respectively.

3. The method according to claim 2, characterized in that , the step 3 specifically includes: Step 3.1: Define the state based on the video frame group feature information vector and the computing device performance index vector ; Step 3.2, define the action space for: ； in, is the number of video frame groups, is the number of available computing devices, each action Indicates that the The video frame groups are assigned to the The task scheduling problem of computing devices; Step 3.3, express the task scheduling problem as: in, is the actual processing time, is the expected processing time, is a positive adjustment parameter used to control the effect of time difference on reward. is the video frame after interpolation. SSIM is used to measure the structural similarity between two images. The closer the SSIM value is to 1, the higher the quality. Indicates that at a given time, the task Offloading to computing devices On, and use Indicates the task Has been successfully assigned to the computing device On, constraint Indicates that each task can only be scheduled to one computing device, and each task can only be scheduled once. Indicates that the computing power, storage space, network bandwidth, and energy consumption allocated to each task do not exceed the maximum value of the computing device. It is The maximum computing power of each computing device, It is The maximum storage space of a computing device, It is The maximum network bandwidth of each computing device, It is The maximum set energy consumption of each computing device; Step 3.4, design reward function for: in, and is the weight coefficient, is the actual processing time, is the expected processing time, is a positive adjustment parameter used to control the effect of time difference on reward. is the video frame after interpolation; Step 3.5, in the state Next action The expected return is used as the Q value and updated using the update formula, which is in, is the learning rate, is the discount factor, is the state at time step t, is the action at time step t, is the time step t The state of is the next state An action from the set of all possible actions, is the next state The best action to choose The corresponding Q value; Step 3.6, derive strategy from Q value : is the next state 。 4. The method according to claim 3, characterized in that , the step 4 specifically includes: Step 4.1, obtaining the intermediate frame and the corresponding video frame group sequence number from the computing device, and inserting the obtained intermediate frame between the video frames according to the video frame group sequence number to form a video interpolation frame group; Step 4.2, encoding the video interpolation frame group to form interpolation frame video data.

Images