A transcoding task scheduling method, system, electronic device and storage medium

Abstract

Embodiments of the present application disclose a transcoding task scheduling method and system, an electronic device and a storage medium. The technical solution provided by the embodiments of the present application comprises the following steps: obtaining the overall load information of a processing node, and the task load information of a real-time transcoding task and an offline transcoding task being executed; determining the task state of the processing node according to the overall load information and the task load information, wherein the task state comprises a predefined first task state and a second task state; suspending the offline transcoding task when it is determined that the processing node is in the first task state, and restarting the offline transcoding task when it is determined that the processing node is in the second task state. By using the above technical solution, the computing resources that are not needed by the real-time transcoding task can be scheduled for use by the offline transcoding task, thereby avoiding the idle of the computing resources, improving the utilization rate of the computing resources for the transcoding task, improving the processing efficiency of the transcoding task, and optimizing the processing effect of the transcoding task.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a transcoding task scheduling method, system, electronic device, and storage medium. Background Art

[0002] Currently, in audio and video media application scenarios, the service backend often needs to transcode the original videos uploaded by users, and output videos of different resolutions through transcoding for use under different requirements. According to the different formats of the original videos, the service backend will create real-time transcoding tasks or offline transcoding tasks to perform video transcoding. In order to schedule and manage different types of transcoding tasks, a container management service cluster is used to deploy transcoding tasks in containers. When it is necessary to run a transcoding task, a container with a corresponding resource quota is created on the cluster based on the computing resource requirements of the transcoding task, and the cluster's resource scheduling platform selects a suitable machine to run the container. When computing resources are tight, higher-priority transcoding tasks can be prioritized to obtain computing resources and run, thereby achieving the effect of mixing different types of transcoding tasks on the same computing cluster.

[0003] However, existing container management service clusters typically allocate relatively fixed computing resources to each transcoding task when scheduling and managing them. However, in transcoding scenarios, real-time transcoding tasks utilize computing resources dynamically, with significant variations in resource usage at different time points. This results in underutilization of fixed computing resources, resulting in relatively low resource utilization. Summary of the Invention

[0004] The embodiments of the present application provide a transcoding task scheduling method, system, electronic device and storage medium, which can improve the utilization rate of transcoding task computing resources and solve the technical problem of insufficient computing resource utilization in video transcoding task processing scenarios.

[0005] In a first aspect, an embodiment of the present application provides a transcoding task scheduling method, comprising:

[0006] Obtain the full load information of the processing node, as well as the task load information of the running real-time transcoding tasks and offline transcoding tasks;

[0007] Determine a task state of the processing node according to the full load information of the entire machine and the task load information, where the task state includes a predefined first task state and a second task state;

[0008] If it is determined that the processing node is in the first task state, the offline transcoding task is suspended, and if it is determined that the processing node is in the second task state, the offline transcoding task is restarted.

[0009] In a second aspect, an embodiment of the present application provides a transcoding task scheduling system, including:

[0010] An information acquisition module configured to obtain full machine load information of a processing node, as well as task load information of running real-time transcoding tasks and offline transcoding tasks;

[0011] A state determination module is configured to determine a task state of the processing node according to the full load information of the entire machine and the task load information, where the task state includes a predefined first task state and a second task state;

[0012] The scheduling module is configured to suspend the offline transcoding task when it is determined that the processing node is in a first task state, and restart the offline transcoding task when it is determined that the processing node is in a second task state.

[0013] In a third aspect, an embodiment of the present application provides a transcoding task scheduling device, including:

[0014] memory and one or more processors;

[0015] The memory is configured to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the transcoding task scheduling method as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are configured to execute the transcoding task scheduling method as described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes instructions, which, when executed on a computer or processor, causes the computer or processor to execute the transcoding task scheduling method as described in the first aspect.

[0019] The embodiment of the application obtains the whole machine full load information of a processing node, and task load information of a real-time transcoding task and an offline transcoding task being run; determines the task state of the processing node according to the whole machine full load information and the task load information, the task state including a predefined first task state and a second task state; suspends the offline transcoding task in the case of determining that the processing node is in the first task state, and restarts the offline transcoding task in the case of determining that the processing node is in the second task state. By using the above technical means, the task state of the processing node is determined according to the load condition of the processing node, so that the transcoding task scheduling is performed according to different task states, the computing resources not needed by the real-time transcoding task are scheduled to the offline transcoding task for use, the idle computing resources are avoided, the utilization rate of the computing resources of the transcoding task is improved, the processing efficiency of the transcoding task is improved, and the processing effect of the transcoding task is optimized. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flowchart of a transcoding task scheduling method provided by the embodiment of the application;

[0021] Figure 2 is a K8S-based transcoding task processing cluster schematic diagram in the embodiment of the application;

[0022] Figure 3 is an interaction flowchart of a K8S-based transcoding task processing cluster in the embodiment of the application;

[0023] Figure 4 is a task state determination flowchart of a processing node in the embodiment of the application;

[0024] Figure 5 is a structure schematic diagram of a transcoding task scheduling device provided by the embodiment of the application;

[0025] Figure 6 is a structure schematic diagram of a transcoding task scheduling device provided by the embodiment of the application. DETAILED DESCRIPTION

[0026] In order to make the purposes, technical solutions and advantages of the present application clearer, specific embodiments of the present application are further described in detail below in combination with the drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only parts related to the present application are shown in the drawings, but not all. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The processes can be terminated when the operations are completed, but can also have additional steps not included in the drawings. The processes can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0027] The transcoding task scheduling method provided by the present application aims to determine the task state of the processing node according to the load condition of the processing node, and then schedule the transcoding task according to different task states, so that the computing resources not needed by the real-time transcoding task are scheduled for the offline transcoding task, so as to improve the utilization rate of the computing resources of the processing node.

[0028] For the traditional transcoding platform in managing the transcoding task, it generally uses the container hosting service of K8S (kubernetes, container service cluster) to deploy and run the transcoding task in the container. When the transcoding task needs to be run, according to the demand of the transcoding task for the computing resources, the container with corresponding resource quota is created on K8S, the number of copies of the container is managed, and the appropriate machine is selected by the system background of K8S to run the container, that is, the scheduling of the computing resources is generally left to K8S to handle. When there is a conflict between different types of containers due to resource shortage, the priority of different containers can be configured to ensure that the transcoding task (container) with higher priority gets the computing resources and runs. Thus, the effect of mixing different types of transcoding tasks into the same computing cluster is achieved.

[0029] Managing computing resources through Kubernetes (K8S) can meet the basic requirement of colocating different types of transcoding tasks on a single machine and within the same computing cluster, ensuring that all transcoding tasks run as smoothly as possible. However, as a general-purpose container hosting service cluster, K8S cannot manage resource allocation to optimize computing resource utilization based on the specific characteristics of transcoding scenarios. Its computing resource management is relatively static. In K8S, a container (i.e., a transcoding task process) must pre-configure a resource quota before deploying it, such as the number of CPU cores and memory required. Based on the specified quota, a fixed amount of computing resources is allocated to the container on the corresponding processing node. In video transcoding scenarios, however, real-time transcoding tasks utilize computing resources dynamically. Depending on changes in bitrate, resolution, bitrate, and business requirements, the computing resource usage of the same real-time transcoding task can vary significantly over time. To address this resource jitter, processing nodes must configure a larger resource quota for real-time transcoding tasks, which the container can exclusively use. Due to changes in computing resource usage, the reserved resource quota cannot be fully used most of the time, and the actual computing resource utilization is not high.

[0030] Furthermore, when computing resources are tight, resource conflicts between containers are prone to occur. For example, a newly requested transcoding task container cannot find a suitable processing node to deploy in the existing cluster. At this time, higher-priority real-time transcoding tasks will be forced to occupy computing resources based on pre-set priorities. Offline transcoding tasks with lower priorities that are already deployed and running on processing nodes will be evicted from the processing nodes to ensure that higher-priority tasks can run first. When evicting offline transcoding tasks, this is often done by directly killing the process. This approach is not friendly to offline transcoding tasks and can easily lead to waste of previous running results.

[0031] Based on actual operational results, while ensuring the normal operation of both real-time and offline tasks, the overall CPU utilization of the container service cluster can only reach around 50%, leaving significant room for improvement. Based on this, a transcoding task scheduling method according to an embodiment of the present application is provided to address the technical problem of insufficient computing resource utilization in video transcoding task processing scenarios.

[0032] Figure 1A flowchart of a transcoding task scheduling method provided in an embodiment of the present application is provided. The transcoding task scheduling method provided in this embodiment can be executed by a transcoding task scheduling device. The transcoding task scheduling device can be implemented by software and / or hardware. The transcoding task scheduling device can be composed of two or more physical entities or a single physical entity. Generally speaking, the transcoding task scheduling device can be a processing device such as each processing node, server host, etc. of a K8S-based transcoding task processing cluster.

[0033] The following description will be made by taking the transcoding task scheduling device as an example to describe the method for executing the transcoding task scheduling method. Figure 1 , the transcoding task scheduling method specifically includes:

[0034] S110: Obtain the full load information of the processing node and the task load information of the running real-time transcoding tasks and offline transcoding tasks.

[0035] The transcoding task scheduling method of the embodiment of the present application adopts a method of dynamically managing the computing resources of the transcoding task. On the basis of the resource division of K8S, it dynamically adjusts the operation and status of the offline transcoding task according to the actual resource usage of the real-time transcoding task. When the real-time transcoding task is relatively idle, the offline transcoding task is allowed to use the currently idle computing resources (referring to the machine resources required for the normal operation of the transcoding task, generally including CPU, memory, GPU, network and other hardware resources). When the real-time transcoding task occupies more computing resources, the offline transcoding task is promptly allowed to give up the computing resources, thereby achieving the effect of dynamically using idle resources and thus improving resource utilization.

[0036] In audio and video media application scenarios, service providers often need to transcode videos uploaded by users, transcoding the original videos uploaded by users into videos of different resolutions to improve the viewing experience of viewers in different network environments. Or they can perform secondary processing on the videos, such as image enhancement, to enrich the diversity of video applications. These transcoding operations on videos are all managed in the form of tasks. According to whether the original video format is a real-time stream or a file, transcoding tasks are divided into real-time transcoding tasks (for real-time video streams) and offline transcoding tasks (for video files).

[0037] Generally speaking, offline transcoding tasks do not have processing time limits, while real-time transcoding tasks do. For example, transcoding tasks for live video streams need to be processed as real-time transcoding tasks so that users can watch live videos with the corresponding resolution and bit rate according to time requirements. Therefore, real-time transcoding tasks generally cannot be interrupted during processing, and their processing priority is higher than offline transcoding tasks. Based on this feature, when the real-time transcoding task is relatively idle, the offline transcoding task can use the currently idle computing resources. When the real-time transcoding task occupies more computing resources, the offline transcoding task can be promptly allowed to give up computing resources to fully utilize the computing resources of the processing node.

[0038] Specifically, refer to Figure 2 , providing a structural diagram of the K8S-based transcoding task processing cluster. Among them, the K8S-based transcoding task processing cluster includes a central node, a K8S background and multiple processing nodes. The central node is used to allocate transcoding tasks and assign transcoding tasks to corresponding processing nodes. The K8S background is used to build a transcoding task process on the corresponding processing node to process the transcoding tasks allocated by the central node. In addition, the transcoding task scheduling method of the embodiment of the present application is executed by the agent process on the processing node to improve the utilization rate of the computing resources of the transcoding task.

[0039] like Figure 3 As shown in the figure, each newly generated transcoding task is assigned to the corresponding processing node through the central node. The processing node then executes the transcoding task processing operations based on the task processing process created by the K8S backend. During operation, the processing node schedules both real-time transcoding tasks and offline transcoding tasks running on its own processing node. The processing node also reports its load status to the K8S backend in real time. Based on the load status of each processing node, the K8S backend notifies the central node to no longer deploy offline transcoding tasks for processing nodes with heavy loads, to prevent further exacerbation of the heavy load and the impact on the processing node's transcoding task processing efficiency.

[0040] Each processing node will have an agent process configured to execute the transcoding task scheduling method of the embodiment of the present application. The agent process will collect the current resource utilization of the processing node and the resource usage of each transcoding task process in real time. Based on the task load information of the real-time transcoding task and offline transcoding task currently running on the processing node, the start and stop of offline tasks will be dynamically managed. The agent process needs to communicate with different task processes to ensure timely control of the synchronous start and stop of offline transcoding tasks, and quickly release computing resources to real-time transcoding tasks when resources are tight.

[0041] Among them, the agent process on the processing node obtains the full load information of the processing node and the task load information of the running real-time transcoding tasks and offline transcoding tasks, and determines the task status of the current processing node based on the obtained relevant load information, so as to schedule transcoding tasks according to the different task states of the processing node.

[0042] For the transcoding task process on the processing node, after it is started, a subdirectory will be created on the transcoding task directory of the processing node, and the transcoding task process corresponding to the subdirectory will be registered. The transcoding task process is the process that processes the transcoding task of the corresponding subdirectory.

[0043] Specifically, when the processing node runs a real-time transcoding task or an offline transcoding task, a sub-directory of the real-time transcoding task or the offline transcoding task is created in the transcoding task directory. The sub-directory contains a priority number for identifying the processing priority of the real-time transcoding task or the offline transcoding task. The processing priority of the real-time transcoding task is greater than the processing priority of the offline transcoding task. The priority number of the offline transcoding task is selected within the specified range.

[0044] For example, after starting a transcoding task, a subdirectory containing a corresponding priority number will be created in the transcoding task directory / var / run / cs_task. For example, when starting a real-time transcoding task, its corresponding real-time transcoding task process mounts the subdirectory / var / run / cs_tasks / 0 to the transcoding task directory / var / run / cs_task, creates a unique file in this directory (the file name is pid) and keeps it open. Priority numbers range from 0 to N, 0 represents the highest priority, corresponding to real-time transcoding tasks, and the larger the priority number, the lower the priority. N represents the lowest priority, corresponding to offline transcoding tasks. Prior to this, by setting a specified range, such as MN, M is greater than 0, indicating that in the transcoding task directory, the transcoding tasks corresponding to the subdirectories with priority numbers from M to N are all offline transcoding tasks. This makes it easier for processing nodes to distinguish between real-time transcoding tasks and offline transcoding tasks.

[0045] The proxy process uses file monitoring to obtain real-time information about the task load of its own processing node. The processing node's proxy process monitors the transcoding task directory, monitoring the files within it and the processes that open them. This process obtains process information for different transcoding tasks and determines the load usage of each transcoding task. It then obtains the processing node's overall machine load information, using these two pieces of information to determine the processing node's task status. It should be understood that overall machine load information represents the number of CPU cores occupied by the processing node when it is fully loaded, assuming 100% CPU utilization.

[0046] By adopting a distributed, autonomous transcoding task management approach, whereby each processing node is configured with an agent process to manage its own transcoding tasks, resource utilization is ensured. Compared to centralized management, this approach allows for faster and more accurate response to changes in computing resources, reduces reliance on the system backend, and optimizes transcoding task scheduling.

[0047] S120 , determining a task state of the processing node according to the full load information of the entire machine and the task load information, where the task state includes a predefined first task state and a second task state.

[0048] Furthermore, based on the acquired full machine load information and task load information, the embodiment of the present application determines the current task state of the application itself and executes the corresponding task scheduling strategy according to the different task states. Prior to this, the first task state and the second task state are pre-defined. The first task state indicates that the load resources of the current processing node are tight, and the second task state indicates that the load resources of the current processing node are idle. In order to determine the task state of the processing node, the embodiment of the present application pre-sets the corresponding task state determination process according to the full machine load information and the task load information to accurately distinguish the different task states of the processing node.

[0049] Among them, reference Figure 4 ,The task status determination process of the processing node includes:

[0050] S1201, determining a current real-time load reference value of a processing node based on task load information of a real-time transcoding task and an offline transcoding task;

[0051] S1202: Compare the full load information of the entire machine with the real-time load reference value, and determine the task state of the processing node according to the comparison result.

[0052] When processing nodes schedule transcoding tasks based on their real-time load, resource utilization fluctuations must be considered when assessing the real-time load of the processing node, as the computing resource usage of real-time transcoding tasks fluctuates in real time. Obviously, real-time transcoding tasks often experience significant fluctuations in CPU usage. During these fluctuations, if computing resources are idle, offline transcoding tasks are immediately started, and if computing resources become scarce, they are immediately paused. This results in frequent starting and stopping of offline transcoding tasks, impacting their success rate and efficiency.

[0053] Based on this, the embodiment of the present application sets a real-time load reference value, which is determined based on the task load information and takes into account the fluctuations in the task load information obtained at different acquisition time nodes in the past. The task status of the processing node is determined by comparing the real-time load reference value with the full load information of the entire machine. This can avoid the situation where the task status is frequently switched when the task status is determined based on the implementation load reference value, resulting in frequent starting and stopping of offline transcoding tasks.

[0054] When determining the real-time load reference value, the initial value of the real-time load reference value is set to the full load information of the entire machine; when the task load information collected for a set number of consecutive times is less than the real-time load reference value, the set value of the real-time load reference value is lowered; when the task load information is greater than or equal to the real-time load reference value, the task load information is updated to the load reference value.

[0055] For example, the real-time load reference value represents the reference value of the number of CPU cores occupied by the current processing node, denoted as R. In the initial state, the processing node is considered to be in a fully loaded state, that is, the CPU utilization rate is 100%. The full load information (that is, the number of CPU cores occupied in the fully loaded state) is denoted as T, and then R=T in the initial state. Furthermore, each time the processing node collects task load information, the real-time load reference value is updated according to the collected task load information. If the number of CPU cores occupied by the task load information is greater than the number of cores in the real-time load reference value, the number of CPU cores occupied by the task load information is set to the value of the current real-time load reference value. The processing node collects task load information once every set time (such as N seconds) and compares it with the task load information collected last time. If the number of CPU cores occupied by the task load information collected for five consecutive times is less than the number of CPU cores occupied by the real-time load reference value, the number of CPU cores occupied by the real-time load reference value is reduced by 1. In this way, an appropriate real-time load reference value is used to represent the load of the current processing node to cope with the fluctuation of computing resource usage by real-time transcoding tasks.

[0056] Furthermore, when comparing the full load information of the entire machine with the real-time load reference value to determine the task state of the processing node, the full load information of the entire machine and the real-time load reference value are compared to obtain the load difference; when the load difference is less than the preset cache, it is determined that the processing node is in the first task state; when the load difference is greater than the sum of the preset cache and the load occupied space of the offline transcoding task, it is determined that the processing node is in the second task state.

[0057] The processing node determines the number of CPU cores occupied by the running offline transcoding task, i.e., the load occupancy space of the offline transcoding task, denoted as L. The number of CPU cores occupied by the offline transcoding task can be fixed, i.e., the agent thread allocates a certain amount of computing resources to the offline transcoding task, and then the offline transcoding task uses all the computing resources to execute the transcoding task. In addition, the processing node also sets a preset buffer, denoted as Buffer, to ensure that there is a reserved idle CPU when the offline transcoding task is running.

[0058] Based on the above determined full load information T, real-time load reference value R, load occupancy space L of the offline transcoding task, and preset buffer Buffer. When T-R<Buffer is satisfied, it means that the current idle computing resources are less, the processing node resource is tight, and it is in the first predefined task state; when T-R>L+Buffer is satisfied, it means that the current idle computing resources are more, and it is in the second predefined task state. Through the above task state determination strategy, the task state of the processing node can be accurately determined according to the load condition of the processing node, and the problem of frequent start and stop of the offline transcoding task caused by computing resource jitter can be effectively avoided.

[0059] Optionally, the offline transcoding task is run based on a preset load upper limit, and the load occupancy space of the offline transcoding task is less than the load upper limit. When the offline transcoding task is deployed, a load upper limit can be set to avoid the load occupancy of the offline transcoding task exceeding the load upper limit. A suitable load upper limit can ensure that the offline transcoding task has enough CPU resources to use, while limiting the upper limit of the CPU resources that can be used. When deploying the transcoding task, the real-time transcoding task and the offline transcoding task can be mixed and deployed in the same processing node as much as possible, so as to avoid deploying multiple offline transcoding tasks on the same machine, and to use the idle computing resources of the real-time transcoding task by the offline transcoding task, so as to improve the utilization rate of computing resources.

[0060] S130, in a case where it is determined that the processing node is in the first task state, suspending the offline transcoding task, and in a case where it is determined that the processing node is in the second task state, restarting the offline transcoding task.

[0061] Finally, based on the determined task state of the processing node, a corresponding task scheduling strategy is executed according to different task states. It can be understood that, in a case where the processing node is in the first task state, since the current processing node load resource is tight, the offline transcoding task needs to be suspended to give the computing resources to the real-time transcoding task. When the processing node is in the second task state, since the current processing node load resource is idle, the suspended offline transcoding task can be restarted to fully utilize the computing resources left by the real-time transcoding task.

[0062] When computing resources are limited, the proxy process pauses the offline transcoding task to free up computing resources. Compared to killing the process, the embodiment of the present application adopts a more user-friendly approach of pausing the offline transcoding task. This allows the offline transcoding task to retain the completed transcoding work, thus avoiding wasting the previously completed transcoding work and improving the processing efficiency of the offline transcoding task.

[0063] Specifically, due to the pre-configured priority code number of each offline transcoding task, the priority number takes a value from a specified range. When pausing an offline transcoding task, the offline transcoding task with the lowest priority will be suspended one by one according to the priority number of each offline transcoding task. By querying the transcoding task directory, it is determined that the transcoding tasks with priority numbers in the specified range among the running transcoding tasks are offline transcoding tasks. Furthermore, according to the priority numbers of the offline transcoding tasks, the offline transcoding tasks are suspended one by one from large to small, so as to release computing resources for real-time transcoding tasks. The agent process of the processing node judges the task status of the processing node every set period (N seconds), and stops the suspension operation of the offline transcoding task until the processing node releases the first task status.

[0064] On the other hand, when restarting a paused transcoding task, the highest-priority offline transcoding task is started one by one according to the priority number of each paused offline transcoding task. Offline transcoding tasks are restarted one by one according to their priority numbers, from smallest to largest, to reuse the computing resources released by the real-time transcoding task. The processing node's agent process checks the processing node's task state at set intervals (N seconds) and stops starting offline transcoding tasks until the processing node is released from the second task state.

[0065] For example, when scheduling offline transcoding tasks, a processing node uses a proxy process to send signals to the offline transcoding task process to notify the task status and update the processing node's load status through the K8S interface. Initially, after the offline transcoding task process starts, it waits for a SIGUSR2 signal from the proxy process to notify it to begin normal operation. When the processing node is under heavy load, the proxy process sends a SIGUSR1 signal to the currently running offline transcoding task process, instructing it to pause the executing offline transcoding task. After the offline transcoding task is paused for 5 seconds, if it is found that the offline transcoding process is still occupying the CPU, the process is suspended via a SIGSTOP signal. This prevents some offline transcoding task processes from continuing to occupy the processing node's computing resources due to being unable to handle the SIGUSR1 signal (e.g., lack of signal processing logic). When the processing node enters a heavy load state, the proxy node updates the processing node's load status to the K8S backend through the K8S interface, preventing the K8S backend's offline task processes from being deployed to the current processing node, thereby avoiding further computing resource occupation. Afterwards, when the machine load is idle, the proxy process notifies the offline transcoding task process via the SIGUSR2 signal to resume normal operation. It then updates the processing node's load status via the K8S interface, resuming normal scheduling and deployment of the offline transcoding task process. It should be noted that, given that the proxy process may also restart, the offline transcoding task process must be idempotent in its handling of the SIGUSR1 and SIGUSR2 signals. This means that the logic must remain correct even if SIGUSR1 or SIGUSR2 signals are continuously received.

[0066] In the above, by obtaining the full load information of the processing node and the task load information of the running real-time transcoding tasks and offline transcoding tasks; determining the task state of the processing node based on the full load information and task load information, the task state includes a predefined first task state and a second task state; if it is determined that the processing node is in the first task state, pausing the offline transcoding task, and if it is determined that the processing node is in the second task state, restarting the offline transcoding task. Using the above technical means, the task state of the processing node is determined according to the load of the processing node, so as to schedule the transcoding tasks according to different task states, and schedule the computing resources that are not needed for the real-time transcoding task to the offline transcoding task, so as to avoid idle computing resources, improve the utilization rate of the computing resources of the transcoding task, improve the processing efficiency of the transcoding task, and optimize the processing effect of the transcoding task.

[0067] Based on the above embodiments, Figure 5 This is a schematic diagram of the structure of a transcoding task scheduling system provided by this application. Figure 5The transcoding task scheduling system provided in this embodiment specifically includes: an information acquisition module 21, a state determination module 22 and a scheduling module 23.

[0068] The information acquisition module 21 is configured to acquire the full load information of the processing node, as well as the task load information of the running real-time transcoding tasks and offline transcoding tasks;

[0069] The state determination module 22 is configured to determine the task state of the processing node according to the full load information of the entire machine and the task load information, where the task state includes a predefined first task state and a second task state;

[0070] The scheduling module 23 is configured to suspend the offline transcoding task if it is determined that the processing node is in the first task state, and restart the offline transcoding task if it is determined that the processing node is in the second task state.

[0071] Specifically, offline transcoding tasks are processed based on a pre-set load limit, and the load space occupied by offline transcoding tasks is less than the load limit. In addition, when running real-time transcoding tasks or offline transcoding tasks, a subdirectory of the real-time transcoding task or offline transcoding task is created in the transcoding task directory. The subdirectory contains a priority number used to identify the processing priority of the real-time transcoding task or offline transcoding task. The processing priority of the real-time transcoding task is greater than the processing priority of the offline transcoding task. The priority number of the offline transcoding task is selected within a specified range.

[0072] Specifically, the status determination module 22 is configured to determine the current real-time load reference value of the processing node based on the task load information of the real-time transcoding task and the offline transcoding task; compare the full load information of the entire machine with the real-time load reference value, and determine the task status of the processing node based on the comparison result.

[0073] Among them, the initial value of the real-time load reference value is set to the full load information of the entire machine; when the task load information collected for a set number of consecutive times is less than the real-time load reference value, the set value of the real-time load reference value is lowered; when the task load information is greater than or equal to the real-time load reference value, the task load information is updated to the load reference value.

[0074] The load difference is obtained by comparing the full load information of the entire machine with the real-time load reference value; when the load difference is less than the preset cache, the processing node is determined to be in the first task state; when the load difference is greater than the sum of the preset cache and the load occupied space of the offline transcoding task, the processing node is determined to be in the second task state.

[0075] Specifically, the scheduling module 23 is configured to sequentially pause the offline transcoding tasks with the lowest priority according to the priority numbers of the offline transcoding tasks; and sequentially start the offline transcoding tasks with the highest priority according to the priority numbers of the paused offline transcoding tasks.

[0076] In the above, by obtaining the full load information of the processing node and the task load information of the running real-time transcoding tasks and offline transcoding tasks; determining the task state of the processing node based on the full load information and task load information, the task state includes a predefined first task state and a second task state; if it is determined that the processing node is in the first task state, pausing the offline transcoding task, and if it is determined that the processing node is in the second task state, restarting the offline transcoding task. Using the above technical means, the task state of the processing node is determined according to the load of the processing node, so as to schedule the transcoding tasks according to different task states, and schedule the computing resources that are not needed for the real-time transcoding task to the offline transcoding task, so as to avoid idle computing resources, improve the utilization rate of the computing resources of the transcoding task, improve the processing efficiency of the transcoding task, and optimize the processing effect of the transcoding task.

[0077] The transcoding task scheduling system provided in the embodiment of the present application can be configured to execute the transcoding task scheduling method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0078] Based on the above practical example, the embodiment of the present application also provides a transcoding task scheduling device, referring to Figure 6 , the transcoding task scheduling device includes: a processor 31, a memory 32, a communication module 33, an input device 34 and an output device 35. The memory 32, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the transcoding task scheduling method described in any embodiment of the present application (for example, the information acquisition module, state determination module and scheduling module in the transcoding task scheduling system). The communication module 33 is configured to perform data transmission. The processor executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory, that is, realizes the above-mentioned transcoding task scheduling method. The input device 34 can be configured to receive input digital or character information, and generate key signal input related to the user settings and function control of the device. The output device 35 may include a display device such as a display screen. The above-mentioned transcoding task scheduling device can be configured to execute the transcoding task scheduling method provided in the above-mentioned embodiment, and has corresponding functions and beneficial effects.

[0079] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide a storage medium containing computer executable instructions, which are configured to perform a transcoding task scheduling method when executed by a computer processor. The storage medium can be any various types of memory device or storage device. Of course, the computer executable instructions of the storage medium provided by the embodiments of the present application are not limited to the transcoding task scheduling method described above, but can also perform the related operations in the transcoding task scheduling method provided by any embodiments of the present application.

[0080] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide a computer program product. The technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a plurality of instructions for causing a computer device, a mobile terminal or a processor therein to perform all or part of the steps of the transcoding task scheduling method described in the embodiments of the present application.

Claims

1. A transcoding task scheduling method, characterized in that: include: Obtain the full load information of the processing node, as well as the task load information of the running real-time transcoding tasks and offline transcoding tasks; Determine a current real-time load reference value of the processing node according to task load information of the real-time transcoding task and the offline transcoding task; Comparing the full load information of the entire machine with the real-time load reference value to obtain a load difference; and determining that the processing node is in the first task state when the load difference is less than a preset buffer; When the load difference is greater than the sum of the load occupied space of the preset cache and the load of the offline transcoding task, determining that the processing node is in the second task state; If it is determined that the processing node is in the first task state, the offline transcoding task is suspended; and if it is determined that the processing node is in the second task state, the offline transcoding task is restarted.

2. The transcoding task scheduling method according to claim 1, wherein: The determining of the current real-time load reference value of the processing node according to the task load information of the real-time transcoding task and the offline transcoding task includes: Setting the initial value of the real-time load reference value to the full load information of the whole machine; When the task load information collected is less than the real-time load reference value for a set number of consecutive times, lowering the set value of the real-time load reference value; In a case where the task load information is greater than or equal to the real-time load reference value, the task load information is updated to the load reference value.

3. The transcoding task scheduling method according to claim 1, wherein: Also includes: The offline transcoding task is run based on a preset load upper limit, and the load space occupied by the offline transcoding task is less than the load upper limit.

4. The transcoding task scheduling method according to claim 1, wherein: Before obtaining the full load information of the processing node and the task load information of the running real-time transcoding tasks and offline transcoding tasks, the following is also included: When the real-time transcoding task or the offline transcoding task is run, a subdirectory of the real-time transcoding task or the offline transcoding task is created in the transcoding task directory. The subdirectory contains a priority number for identifying the processing priority of the real-time transcoding task or the offline transcoding task. The processing priority of the real-time transcoding task is greater than the processing priority of the offline transcoding task. The priority number of the offline transcoding task is selected within a specified range.

5. The transcoding task scheduling method according to claim 4, wherein: The pausing the offline transcoding task includes: According to the priority numbers of the offline transcoding tasks, pausing the offline transcoding tasks with the lowest priority one by one in sequence; The restarting of the offline transcoding task includes: According to the priority numbers of the paused offline transcoding tasks, the offline transcoding tasks with the highest priority are started one by one in sequence.

6. A transcoding task scheduling system, characterized in that: include: An information acquisition module configured to obtain full machine load information of a processing node, as well as task load information of running real-time transcoding tasks and offline transcoding tasks; a state determination module configured to determine a current real-time load reference value of a processing node according to task load information of the real-time transcoding task and the offline transcoding task; Comparing the full load information of the entire machine with the real-time load reference value to obtain a load difference; and determining that the processing node is in the first task state when the load difference is less than a preset buffer; When the load difference is greater than the sum of the load occupied space of the preset cache and the load of the offline transcoding task, determining that the processing node is in the second task state; The scheduling module is configured to suspend the offline transcoding task if it is determined that the processing node is in the first task state, and restart the offline transcoding task if it is determined that the processing node is in the second task state.

7. A transcoding task scheduling device, characterized in that: include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the transcoding task scheduling method according to any one of claims 1 to 5.

8. A storage medium containing computer-executable instructions, characterized in that: When executed by a computer processor, the computer executable instructions are configured to execute the transcoding task scheduling method according to any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the transcoding task scheduling method according to any one of claims 1 to 5.

Images