A distributed transcoding scheduling method and system

Abstract

The present application relates to transcoding technical field, specifically related to a kind of distributed transcoding scheduling method and system, comprising, step S1, read the highest priority in transcoding task to be transcoded file, and calculate current time window reference value;Step S2, obtain the video duration that can be converted by current time window reference value calculation, in each transcoding task select the total duration less than or equal to the video duration that can be converted step S1 read to be transcoded file;Step S3, the to-be-transcoded file after selection is sent to transcoding engine;Step S4, to be transcoded file is sliced and transcoded;Step S5, return step S1 to the to-be-transcoded file of transcoding task remaining is transcoded operation;The present application allocates the transcoding resource corresponding to the service level of each tenant, solves the resource contention problem in multi-tenant transcoding application, so that system maximizes efficiency while meeting the quality of service under multi-tenant environment.

Description

Technical Field

[0001] This invention relates to the field of transcoding technology, specifically to a transcoding scheduling method and system. Background Technology

[0002] In multi-tenant applications, traditional distributed transcoding scheduling systems cannot solve the problem of resource contention among tenants. In order to complete their transcoding tasks as soon as possible, tenants need to increase the priority of their tasks. When all tasks are given the highest priority, the allocation of transcoding resources among tenants becomes completely first-come, first-served, which cannot effectively guarantee the service level of each tenant. Summary of the Invention

[0003] The purpose of this invention is to provide a distributed transcoding scheduling method to solve the above-mentioned technical problems;

[0004] The present invention also aims to provide a distributed transcoding scheduling system to solve the above-mentioned technical problems.

[0005] The technical problem solved by this invention can be achieved by the following technical solutions:

[0006] A distributed transcoding scheduling method, comprising,

[0007] Step S1: Read the highest priority file to be transcoded in each transcoding task, divide the duration of each file to be transcoded by the transcoding resource multiplier corresponding to the transcoding task, and select the maximum value from the obtained results as the current time window reference value.

[0008] Step S2: Calculate the convertible video duration using the current time window reference value, and select the file to be transcoded read in step S1 whose total duration is less than or equal to the convertible video duration in each transcoding task;

[0009] Step S3: The selected files to be transcoded are sent to the transcoding engine as the video set to be transcoded in the current time window;

[0010] Step S4: Slice the file to be transcoded according to the maximum available channels, and transcode the sliced ​​file line by line in a set order;

[0011] Step S5: After all the files to be transcoded within the current time window have been transcoded, return to step S1 to perform transcoding operations on the remaining files to be transcoded in the transcoding task, until all the files to be transcoded have been transcoded.

[0012] Preferably, in step S1, the transcoding task has files to be transcoded with pre-set priorities, and each priority corresponds to zero or one or more files to be transcoded.

[0013] Preferably, in step S2, the video duration obtained by multiplying the current time window reference value by the transcoding resource multiplier of the transcoding task is used as the transcoding video duration corresponding to the transcoding task.

[0014] Preferably, step S2 includes,

[0015] Step S21: Select the file to be transcoded with the highest priority from the transcoding tasks in sequence;

[0016] Step S22: After each selection, determine whether the total duration of the selected files to be transcoded exceeds the duration of the convertible video. If so, abandon the selection of the most recently selected file to be transcoded, lower the priority of the most recently selected file to be transcoded by one, and continue to execute step S21; otherwise, directly continue to execute step S21 until all files to be transcoded have been traversed.

[0017] Step S23: Read the files to be transcoded for the remaining transcoding tasks according to steps S21 and S22.

[0018] Preferably, the method further includes readjusting the priority of the files to be transcoded within the transcoding task based on usage requirements, and extracting the files to be transcoded based on the adjusted priority in step S2.

[0019] Preferably, step S2 further includes selecting the files to be upgraded from the transcoding task in an intervention manner, and performing transcoding operations on the upgraded files to be upgraded first.

[0020] Preferably, in step S4, the required transcoding time of the files to be transcoded is sorted, and the sliced ​​files to be transcoded are transcoded in order from the file with the shortest transcoding time to the file with the longest transcoding time.

[0021] Preferably, the transcoding task is a task list containing a plurality of the files to be transcoded.

[0022] A distributed transcoding scheduling system for implementing the aforementioned distributed transcoding scheduling method includes,

[0023] A scheduling engine, in which the transcoding task is imported, the scheduling engine is used to select the file to be transcoded based on the priority of the file to be transcoded and based on the time window reference value;

[0024] N distributed transcoding engines are connected to the scheduling engine. Each transcoding engine receives the file to be transcoded and the transcoding task instruction issued by the scheduling engine, and performs slice transcoding on the file to be transcoded based on the transcoding task instruction.

[0025] Where N is a positive integer greater than or equal to 2.

[0026] Preferably, the status of the transcoding engine is monitored before the scheduling engine issues the transcoding task instruction each time, and the transcoding task instruction is sent to the transcoding engine with the fewest idle tasks or the fewest task queues based on the monitoring results for transcoding.

[0027] The beneficial effects of the present invention are as follows: By adopting the above technical solution, the present invention solves the resource contention problem in multi-tenant transcoding applications by allocating transcoding resources to each tenant in accordance with their service level, thereby maximizing the efficiency of the system while meeting the quality of service in a multi-tenant environment. Attached Figure Description

[0028] Figure 1 This is a schematic diagram illustrating the steps of the distributed transcoding scheduling method in an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of step S2 in an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of queue adjustment in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of transcoding task processing in an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0035] A distributed transcoding scheduling method, such as Figure 1 As shown, including,

[0036] Step S1: Read the highest priority file to be transcoded in each transcoding task, divide the duration of each file to be transcoded by the transcoding resource multiplier corresponding to the transcoding task, and select the maximum value from the obtained results as the current time window reference value.

[0037] Step S2: Calculate the convertible video duration using the current time window reference value, and select the file to be transcoded read in step S1 whose total duration is less than or equal to the convertible video duration in each transcoding task;

[0038] Step S3: Send the selected files to be transcoded to the transcoding engine as the video set to be transcoded in the current time window;

[0039] Step S4: Slice the file to be transcoded according to the maximum available channels, and transcode the sliced ​​file line by line in the set order.

[0040] Step S5: After all files to be transcoded within the current time window have been transcoded, return to step S1 to perform transcoding operations on the remaining files to be transcoded in the transcoding task, until all files to be transcoded have been transcoded.

[0041] Specifically, this invention solves the resource contention problem in multi-tenant transcoding applications by allocating transcoding resources to each tenant according to their service level, thereby maximizing system efficiency while meeting service quality requirements in a multi-tenant environment.

[0042] In a preferred embodiment, in step S1, the transcoding task has files to be transcoded with pre-set priorities, and each priority corresponds to zero or one or more files to be transcoded.

[0043] Specifically, for each transcoding task issued by a tenant, the priority within the task can be customized when the tenant initiates the transcoding task.

[0044] In a preferred embodiment, in step S2, the video duration obtained by multiplying the current time window reference value by the transcoding resource multiplier of the transcoding task is used as the transcoding video duration corresponding to the transcoding task.

[0045] Specifically, the current time window baseline value is multiplied by the transcoding resource multiplier of each tenant to calculate the video duration that can be transcoded for each tenant in the current time window. According to the priority order set by each tenant, the current window is selected for each tenant to select the video to be transcoded. The total duration of the selected video is less than or equal to the video duration that each tenant can transcode in the current time window calculated in the previous step.

[0046] In a preferred embodiment, such as Figure 2 As shown, step S2 includes,

[0047] Step S21: Select the highest priority file to be transcoded from the transcoding tasks in sequence;

[0048] Step S22: After each selection, determine whether the total duration of the selected files to be transcoded exceeds the duration of the video to be transcoded. If so, abandon the selection of the most recently selected file to be transcoded, reduce the priority of the most recently selected file to be transcoded by one, and continue to execute step S21; otherwise, directly continue to execute step S21 until all files to be transcoded have been traversed.

[0049] Step S23: Read the files to be transcoded for the remaining transcoding tasks according to steps S21 and S22.

[0050] In a preferred embodiment, the method further includes readjusting the priority of the files to be transcoded within the transcoding task based on usage requirements, and extracting the files to be transcoded based on the adjusted priority in step S2.

[0051] In a preferred embodiment, step S2 further includes selecting files to be upgraded from the transcoding task in an intervention manner, and performing transcoding operations on the upgraded files first.

[0052] Specifically, this invention provides queue adjustment functionality, such as... Figure 3 As shown, t1, t2, t3 to t7 represent transcoding times. The transcoding scheduling method provided in this invention has a two-level task priority adjustment function.

[0053] The first-level priority adjustment function sorts the priority of each tenant's tasks. Each tenant can adjust the priority queue of its own transcoding tasks according to its own business situation. The second level is the global priority adjustment function. Administrators can manually select the tasks that need to be promoted from the list of all transcoding tasks, and the tasks will be completed in advance to the next time.

[0054] Ideally, by adjusting queue priorities, the transcoding needs of tasks in emergency situations can be met quickly.

[0055] In a preferred embodiment, in step S4, the required transcoding time of the files to be transcoded is sorted, and the sliced ​​files to be transcoded are transcoded in order from the file with the shortest transcoding time to the file with the longest transcoding time.

[0056] In a preferred embodiment, the transcoding task is a task list with a number of files to be transcoded.

[0057] A distributed transcoding scheduling system, used to implement the distributed transcoding scheduling method in any of the embodiments, includes,

[0058] A scheduling engine is used to import transcoding tasks. The scheduling engine is used to select transcoding files based on their priority and time window reference values.

[0059] N groups of distributed transcoding engines are connected to the scheduling engine. The transcoding engine receives the files to be transcoded and transcoding task instructions issued by the scheduling engine, and performs slice transcoding on the files to be transcoded based on the transcoding task instructions.

[0060] Where N is a positive integer greater than or equal to 2.

[0061] In a preferred embodiment, the status of the transcoding engine is monitored before the scheduling engine issues a transcoding task instruction each time, and the transcoding task instruction is sent to the transcoding engine with the least idle or least task queue for transcoding based on the monitoring result; preferably, this can improve the disaster recovery capability of the system.

[0062] In one specific embodiment, it is assumed that all tenants use only the same transcoding service, for example: MXF_720p_50M->MP4_720p_8;

[0063] When the transcoding speed of this service is set to 1x, the transcoding resource used is 1 unit;

[0064] Assuming that the file is sliced, transcoded and merged across N channels, the efficiency is improved by a factor of N compared to a single channel.

[0065] Based on the above assumptions, when a tenant requires N times the transcoding efficiency, the transcoding resources used are N units. Table 1 shows the assumed service level for each tenant, and Table 2 shows the platform's transcoding resource volume. The over-resolution rate is calculated to be 20%.

[0066]

[0067] Table 1: Assumed service levels for each tenant

[0068]

[0069] Table 2: Platform Transcoding Resource Quantity

[0070] The over-scoring rate is: (12-10) / 10×100%=20%.

[0071] Assuming that each tenant has a batch of transcoding tasks between time t1 and t4, the number of tasks, duration and other information are shown in Table 3.

[0072]

[0073]

[0074] Table 3

[0075] Based on the aforementioned scheduling algorithm, the transcoding task processing status of the transcoding engine from time t1 to t4 is as follows: Figure 4 As shown, this embodiment uses time t1 for explanation. Based on step S2, it can be seen that in step S1, the highest priority file to be transcoded in each transcoding task is read, which corresponds to the file to be transcoded at time t1. The duration of each file to be transcoded is divided by the transcoding resource multiplier corresponding to the transcoding task, where:

[0076] Tenant A's calculation result is: 80 / 5 = 16;

[0077] Tenant B's calculation results are: 20 / 2 = 10; 10 / 2 = 5; 50 / 2 = 25;

[0078] The calculation result for tenant C is: 40 / 3 = 13.33;

[0079] The maximum value obtained is selected as the base value for the current time window; therefore, the base value for the current time window is 25.

[0080] In step S2, the video duration obtained by multiplying the current time window base value (25) by the transcoding resource multiplier of the transcoding task is used as the transcoding video duration corresponding to the transcoding task.

[0081] therefore:

[0082] Tenant A's convertible video duration is: 25 × 5 = 125;

[0083] Tenant B's convertible video duration is: 25 × 2 = 50;

[0084] Tenant C's convertible video duration is: 25 × 3 = 75;

[0085] When selecting the file to be transcoded in step S2, the files to be transcoded for tenants A and C are both shorter than their convertible video length. When performing step S2 for tenant B, selecting the first two files to be transcoded will not exceed the convertible video length. When the third file to be transcoded is selected, its length reaches 80 minutes, which exceeds the convertible video length of tenant B (50 minutes). Therefore, the selection of the third file to be transcoded is abandoned and its priority is reduced. It will be transcoded in subsequent transcoding operations.

[0086] In step S4, the file to be transcoded is sliced ​​according to the maximum available channel, and the sliced ​​file to be transcoded is transcoded line by line in the order of the shortest transcoding time to the longest transcoding time.

[0087] At this point, the system load rate = 460 / 46 / 10 * 100% = 100%.

[0088] Assuming all tasks are initiated before the preceding tasks are completed, and the system only has resources to process them after waiting for half the execution time of the previous group of tasks, the overall task efficiency for each tenant, including queuing time, is as follows:

[0089] Tenant A: (80+20+100) / (15+8.5+18.5)=4.76;

[0090] Tenant B: (10+20+50+10+20) / (1+3+24+5.5+8.5)=2.62;

[0091] Tenant C: (40+20+10) / (7+9.5+4.5)=3.33;

[0092] Tenant D: (30+10) / (11.5+4.5)=2.5;

[0093] Tenant E: (20+10+10) / (11.5+18+6.5)=1.11;

[0094] In reality, if the system load rate is not always 100%, meaning there are periods when the system is idle, and new tasks arrive without waiting for previous tasks to finish, then the overall task efficiency will be further improved.

[0095] In summary, this invention solves the resource contention problem in multi-tenant transcoding applications by allocating transcoding resources to each tenant according to their service level, enabling the system to maximize efficiency while meeting service quality requirements in a multi-tenant environment. At the same time, the system provides a queue adjustment function, which allows for manual intervention in the transcoding queue when necessary.

[0096] The above description is merely a preferred embodiment of the present invention and does not limit the implementation and protection scope of the present invention. Those skilled in the art should realize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. A distributed transcoding scheduling method, characterized in that, include, Step S1: Read the highest priority file to be transcoded in each transcoding task, divide the duration of each file to be transcoded by the transcoding resource multiplier corresponding to the transcoding task, and select the maximum value from the obtained results as the current time window reference value. Step S2: Calculate the convertible video duration using the current time window reference value, and select the file to be transcoded read in step S1 whose total duration is less than or equal to the convertible video duration in each transcoding task; Step S3: The selected files to be transcoded are sent to the transcoding engine as the video set to be transcoded in the current time window; Step S4: Slice the file to be transcoded according to the maximum available channels, and transcode the sliced ​​file line by line in a set order; Step S5: After all the files to be transcoded within the current time window have been transcoded, return to step S1 to perform transcoding operations on the remaining files to be transcoded in the transcoding task until all the files to be transcoded have been transcoded. The transcoding resource multiplier represents the number of units of equivalent platform transcoding resources required to execute the corresponding transcoding task.

2. The distributed transcoding scheduling method according to claim 1, characterized in that, In step S1, the transcoding task contains files to be transcoded with pre-set priorities, and each priority corresponds to zero or one or more files to be transcoded.

3. The distributed transcoding scheduling method according to claim 1, characterized in that, In step S2, the video duration obtained by multiplying the current time window reference value by the transcoding resource multiplier of the transcoding task is used as the transcoding video duration corresponding to the transcoding task.

4. The distributed transcoding scheduling method according to claim 1, characterized in that, Step S2 includes, Step S21: Select the file to be transcoded with the highest priority from the transcoding tasks in sequence; Step S22: After each selection, determine whether the total duration of the selected files to be transcoded exceeds the duration of the convertible video. If so, abandon the selection of the most recently selected file to be transcoded, lower the priority of the most recently selected file to be transcoded by one, and continue to execute step S21; otherwise, directly continue to execute step S21 until all files to be transcoded have been traversed. Step S23: Read the files to be transcoded for the remaining transcoding tasks according to steps S21 and S22.

5. The distributed transcoding scheduling method according to claim 1, characterized in that, It also includes readjusting the priority of the files to be transcoded within the transcoding task based on usage requirements, and extracting the files to be transcoded based on the adjusted priority in step S2.

6. The distributed transcoding scheduling method according to claim 1, characterized in that, Step S2 further includes selecting the files to be upgraded from the transcoding task in an intervention manner, and performing transcoding operations on the upgraded files to be upgraded first.

7. The distributed transcoding scheduling method according to claim 1, characterized in that, In step S4, the required transcoding time of the files to be transcoded is sorted, and the sliced ​​files to be transcoded are transcoded in order from the file with the shortest transcoding time to the file with the longest transcoding time.

8. The distributed transcoding scheduling method according to claim 1, characterized in that, The transcoding task is a task list containing several files to be transcoded.

9. A distributed transcoding scheduling system, used to implement the distributed transcoding scheduling method according to any one of claims 1-8, characterized in that, include, A scheduling engine, in which the transcoding task is imported, the scheduling engine is used to select the file to be transcoded based on the priority of the file to be transcoded and based on the time window reference value; N distributed transcoding engines are connected to the scheduling engine. Each transcoding engine receives the file to be transcoded and the transcoding task instruction issued by the scheduling engine, and performs slice transcoding on the file to be transcoded based on the transcoding task instruction. Where N is a positive integer greater than or equal to 2.

10. The distributed transcoding scheduling system according to claim 9, characterized in that, Before each transcoding task instruction is issued by the scheduling engine, the status of the transcoding engine is monitored, and based on the monitoring results, the transcoding task instruction is sent to the transcoding engine with the least idle time or the least number of tasks in the queue for transcoding.

Images