A dynamic task expansion scheduling system and method

Abstract

The application relates to computer application technology and discloses a dynamic expansion task scheduling system and method, which comprises a task center module, a task plug-in module, a task client module and a pre-configuration module; the pre-configuration module is used for setting pre-delivered tasks; the task plug-in module is used for defining a standard interface of the task center module and loading the task center module; the task client is used for generating task information and then handing over the task information to the task center module for scheduling and execution; the task client module and the task plug-in module correspond to the task center module one by one and are used for the flow of the scheduling system, are connected with the task client module and interact with the task plug-in module. Through the designed dynamic expansion task scheduling system, different types of tasks can be gathered, a certain type of task can be added or removed without affecting the scheduling and execution of other tasks, and each type of task can support custom scheduling rules.

Description

Technical Field

[0001] This invention relates to computer application technology, and in particular to a scheduling system and method for dynamically expanding tasks. Background Technology

[0002] Task scheduling refers to the allocation of different types of tasks to different resources for execution. The scheduling system is responsible for the unified scheduling and execution of all types of tasks. Since scheduling and execution are tightly coupled, if a new task type is added, the scheduling system needs to be interrupted for iterative upgrades, which will affect other tasks and cannot achieve dynamic expansion.

[0003] The core of a scheduling system is its scheduling rules. Currently, the industry uses rules defined by the scheduling system itself, which cannot be customized for a specific task type.

[0004] Every enterprise or organization has various online tasks to perform, such as video transcoding, facial recognition, and nucleic acid testing. Currently, there is no good solution to aggregate these tasks onto a single platform for scheduling and execution.

[0005] For example, the existing technology CN202011623543.0, the task scheduling applied for by our company, cannot effectively expand tasks dynamically, and therefore cannot effectively meet the needs of adding or removing certain types of tasks when other task scheduling and execution are not desired. Summary of the Invention

[0006] This invention addresses the problem that existing task scheduling systems cannot effectively dynamically expand tasks or satisfy the need to add or remove tasks when other tasks are not desired. It provides a scheduling system for dynamically expanding tasks.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] A dynamically expandable task scheduling system for video task scheduling includes a task center module, a task plugin module, a task client module, and a pre-configuration module.

[0009] The pre-configuration module is used to set up pre-issued tasks;

[0010] The task plugin module is used to implement the definition of the standard interface of the task center module and load it into the task center module;

[0011] The task client is used to generate task information, which is then handed over to the task center module for scheduling and execution; and there is a one-to-one correspondence between the task client module and the task plugin module.

[0012] The task center module is used to schedule the flow of the system, interfaces with the task client module, and interacts with the task plugin module.

[0013] The dynamically expandable task scheduling system can aggregate different types of tasks and add or remove certain types of tasks without affecting the scheduling and execution of other tasks. It also supports custom scheduling rules for each type of task.

[0014] As a preferred embodiment: the task center module includes an abstract module and a matching module; the abstract module is used to define the standard interface and is implemented through the task plugin module; the definition of the standard interface includes scheduling rule definition, scheduling rule matching, task execution, and functional definition;

[0015] The matching module is used to filter scheduling tasks and locate the resource information of executable tasks; the matching module includes route matching, cluster matching and resource matching.

[0016] As a preferred option, the pre-configured module includes a routing configuration module, a cluster configuration module, and a resource configuration module.

[0017] As a preferred option: route configuration includes task type settings, route rule settings, and route value settings; task type settings are used to set the task type, which is obtained from the task plugin module; route rule settings include common rule settings and scheduling rule settings; common rule settings are defined through the task center module; scheduling rule settings are obtained through the task plugin module.

[0018] The route value is set to the route value corresponding to the routing rule.

[0019] As a preferred configuration, the cluster configuration module includes route settings, feature settings, and task limit settings. The route settings are used to configure the route information associated with the assigned route. The feature settings are used to configure the custom features of the task plugin module. The task limit settings are used to limit the number of tasks that can run on each resource in the cluster.

[0020] As a preferred approach: route matching is used for the task center module to connect with the task client module; cluster matching is used to filter eligible clusters under eligible routes; and resource matching is used for processing before task distribution.

[0021] To address the aforementioned technical problems, this invention also provides a method for scheduling dynamically extended tasks, used in the scheduling of video tasks;

[0022] The task scheduling information is configured through the pre-configuration module. The task scheduling information includes the resources for task scheduling and the rules for task scheduling.

[0023] The assembly of tasks, the device for scheduling task information through the task client module, and the ability to dynamically expand tasks;

[0024] The flow of tasks is handled through the task center module for task scheduling.

[0025] Task operations are performed through the task plugin module, which provides task operation information to the task center module. The task operation information includes unique task scheduling rules, definitions of task functionalities, and dynamically expandable tasks. Task operations include starting, stopping, and deleting tasks.

[0026] To address the aforementioned technical problems, the present invention also provides a storage medium comprising a storage medium implemented with a dynamically expanding task scheduling system.

[0027] To address the aforementioned technical problems, the present invention also provides an electronic device comprising an electronic device implemented with a dynamically expanding task scheduling system.

[0028] This invention, by adopting the above technical solutions, has significant technical effects:

[0029] This invention designs a dynamic task scheduling system that can aggregate different types of tasks and can add or remove certain types of tasks without affecting the scheduling and execution of other tasks. It also supports custom scheduling rules for each type of task. Attached Figure Description

[0030] Figure 1 This is a system schematic diagram of the present invention;

[0031] Figure 2 This is a schematic diagram of the system in Embodiment 1 of the present invention;

[0032] Figure 3 This is a flowchart of the present invention. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0034] Example 1

[0035] A dynamically expandable task scheduling system for video task scheduling includes a task center module, a task plugin module, a task client module, and a pre-configuration module.

[0036] The pre-configuration module is used to set up pre-issued tasks;

[0037] The task plugin module is used to implement the definition of the standard interface of the task center module and load it into the task center module;

[0038] The task client is used to generate task information, which is then handed over to the task center module for scheduling and execution; and there is a one-to-one correspondence between the task client module and the task plugin module.

[0039] The task center module is used to schedule the flow of the system, interfaces with the task client module, and interacts with the task plugin module.

[0040] The dynamically expandable task scheduling system can aggregate different types of tasks and add or remove certain types of tasks without affecting the scheduling and execution of other tasks. It also supports custom scheduling rules for each type of task.

[0041] The task center module is responsible for the entire scheduling system's workflow. It interfaces with the task client module and interacts with the task plugin module, making it a key module of the scheduling platform. The task center module includes an abstraction module and a matching module.

[0042] The abstract module includes scheduling rule definition, functional feature definition, scheduling rule matching, and task execution.

[0043] The scheduling rule definition references task information from pre-configured modules and sets plugin-specific scheduling rules. These rules serve as conditions for task filtering; for example, a video transcoding task plugin might need to provide filtering by resolution and video bitrate. The plugin module should implement this interface and respond with a set of supported scheduling rule information. When the pre-configured module performs routing configuration, the task center module will look up the implementation of this interface in the task plugin to obtain the provided scheduling rules for the pre-configured module to configure. Java pseudocode example:

[0044] public interface: Scheduling rules definition interface {

[0045] List<DispatchRules> buildDispatchRules();

[0046] }

[0047] The feature definitions are referenced by the pre-configured modules and specify the scope supported by the task plugin. For example, if a video transcoding task plugin supports different transcoding vendors, then the plugin module should implement this interface and respond with a set of supported vendor information. When the pre-configured modules perform cluster configuration, the task center module will look up the implementation of this interface in the task plugin and obtain the provided features for the pre-configured modules to configure. Java pseudocode example:

[0048] public interface defines the functional characteristics of the interface {

[0049] List<features> buildSupportedFeatures();

[0050] }

[0051] The scheduling rule matching is referenced by the matching module of the task center module and used for route matching and filtering. If a task plugin provides unique scheduling rules, it must also implement the scheduling rule matching interface. When the matching module references this interface, it passes the task information from the task client module and the route configuration of the pre-set module as parameters. When the task plugin implements this interface, it will receive these parameters and only needs to respond with a boolean value indicating whether a match has occurred. For example, if a video transcoding task plugin provides two unique scheduling rules: by resolution and by video bitrate, then when the task plugin implements the scheduling rule matching interface, it needs to parse the task content to obtain the specific resolution or video bitrate, and then perform matching based on the scheduling rules and scheduling rule values ​​configured in the passed route configuration. Java pseudocode example:

[0052] public interface scheduling rule matching interface {

[0053] booleanmatch(task information, routing configuration);

[0054] }

[0055] Task execution is referenced by the matching module of the task center module and is used to execute specific task content. Task execution includes common operations such as starting, stopping, and deleting. All operations need to be implemented by the task plugin. When the matching module references the task, it passes the task information of the task client module and the cluster configuration and resource configuration of the pre-set module as parameters. The task plugin will obtain these parameters when implementing this interface. The start interface needs to return the actual task identifier, which will be passed as a parameter to the stop and delete interfaces. For example, when the video transcoding task plugin implements task startup, it can obtain the service address from the resource configuration and then convert the task content. The reason for conversion is that the transcoding parameters of different vendors may be different. It is necessary to obtain the functional characteristics required by this task (in this example, the vendor) from the cluster configuration before starting the task. Java pseudocode example:

[0056] public interface Task execution interface {

[0057] String start(task information, cluster configuration information, resource configuration information);

[0058] boolean stop(task information, task id, cluster configuration information, resource configuration information);

[0059] boolean delete(task information, task id, cluster configuration information, resource configuration information);

[0060] }

[0061] The matching module is responsible for task filtering and ultimately locating the resource information that can execute the task, which is then handed over to the task execution in the abstract module. This includes: route matching, cluster matching, and resource matching.

[0062] Route matching is used for direct connection between the task center module and the task client module. When route matching receives client parameters, it filters all routes of that type from the route configuration in the pre-set module according to the passed task type, and then matches each route. The route information and task information are passed to the scheduling rule matching in the abstract module. The scheduling rule matching will refer to the plugin implementation of the corresponding task type. The response true means that the route meets the rule and no further route matching is needed. If there is no matching route, the client exception can be responded directly. When the matched route is obtained, it is then handed over to the next level cluster matching for filtering. If it is a task stop or task deletion operation, this step is skipped because the task is already running and the associated route, cluster and resource information are already in the task center module.

[0063] Cluster matching is used to filter eligible clusters under routes that meet certain criteria. The criteria for meeting the criteria is that there are corresponding available resources under the cluster. When the route information is received from the route matching, the associated cluster set is obtained according to the cluster configuration in the preset module. Then, the resource information under each cluster is queried to filter out idle resource information. The comparison condition for idle is: the number of tasks running on the resource is less than the maximum number of tasks in the cluster configuration. When the matched cluster information and resource information are obtained, they are then handed over to the next level of resource matching for processing. If it is a task stop or task deletion operation, this step is skipped because the task is already running and the associated route, cluster, and resource information are already in the task center module.

[0064] Resource matching is used for processing before task distribution. When resource information is received from the cluster matching, the current resource information is repeatedly checked to see if the number of tasks meets the requirements. Repeated checking is to prevent the resource from being used during the process, which would lead to excessive resource load. If the resource no longer meets the requirements, an idle resource needs to be obtained again based on the cluster information. After obtaining the resource that meets the conditions, the resource information, task information, and cluster information are passed to the task execution in the abstract module. The task execution will refer to the plugin implementation of the corresponding task type. If it is to start a task, the number of running tasks in the resource information needs to be incremented by 1, and the information associated with the current task is recorded in the task center module. If it is to stop or delete a task, the number of running tasks in the resource information needs to be decremented by 1.

[0065] Routing configuration mainly includes task type settings, routing rule settings, and route value settings.

[0066] The task type is obtained from the task plugin module;

[0067] Routing rules consist of two categories. One category is the common rules defined by the task center module, such as by name or by title. The other category is custom scheduling rules obtained from the task plugin module. Custom scheduling rules must be implemented according to the standard defined by the task center module > abstract module > scheduling rules. For example, video transcoding tasks can be scheduled according to different resolutions, so the scheduling rule provided by the video transcoding task plugin should be: by resolution.

[0068] The routing value refers to the value corresponding to the routing rule. For example, if the routing rule is by name, then the routing value should be the specific value of the name. If the name of the task matches the value, then the route matching is complete.

[0069] Cluster configuration mainly includes routing settings, feature settings, and task limit settings.

[0070] The associated route is the route information.

[0071] Functional features are settings provided to enable more flexible task flow. Custom functional features need to be obtained from the task plugin module. Custom functional features must be implemented according to the standard of Task Center Module > Abstract Module > Functional Feature Definition. For example, in video transcoding tasks, if a task at the same resolution is to be executed on devices produced by different manufacturers, then the video transcoding task plugin should provide the functional features of Manufacturer 1 and Manufacturer 2. If there are different functional features, the corresponding task plugin should implement the task execution logic under each functional feature internally.

[0072] The task limit is used to restrict the number of tasks running on each resource in the cluster, preventing a single resource from being overloaded.

[0073] Resource configuration mainly involves information about the machines that can execute tasks. This is not limited to machines; it can also be a service address. It mainly includes: the cluster to which the task belongs, the resource address (IP or service name), and the port. In the Task Center module > Matching module > Resource Matching, the configured resource information will be passed to the task plugin, which will then distribute the task. The resource configuration also needs to include the number of tasks to run, which is maintained by the Task Center module > Matching module > Resource Matching.

[0074] The task client module corresponds one-to-one with the task plugin module. That is, if a new task plugin is added, a new task client will be added. It is mainly used to generate task information and then hand it over to the task center module for scheduling and execution. The task information must include: task type and task content.

[0075] The task type must correspond to the task plugin. For example, if the video transcoding task plugin is named transcode, then the task type in the task client must also be transcode.

[0076] The task content refers to the specific content that needs to be performed. Taking video transcoding as an example, the task content needs to include the input source, transcoding parameters, output address, etc. There are no restrictions on the task content. The task content corresponding to different task plugins is also different. It is all parsed by the corresponding plugin. After the task center module matches the specific resource, it forwards the task information to the task plugin.

[0077] Example 2

[0078] Based on Example 1, this example is a method for scheduling dynamically extended tasks, used in the scheduling of video tasks;

[0079] The task scheduling information is configured through the pre-configuration module. The task scheduling information includes the resources for task scheduling and the rules for task scheduling.

[0080] The assembly of tasks, the device for scheduling task information through the task client module, and the ability to dynamically expand tasks;

[0081] The flow of tasks is handled through the task center module for task scheduling.

[0082] Task operations are performed through the task plugin module, which provides task operation information to the task center module. The task operation information includes unique task scheduling rules, definitions of task functionalities, and dynamically expandable tasks. Task operations include starting, stopping, and deleting tasks.

[0083] Example 3

[0084] Based on Example 1, this example is a storage medium.

[0085] Example 4

[0086] Based on Example 1, this example is an electronic device.

Claims

1. A dynamically expanding task scheduling system for video task scheduling, characterized in that: The task center module, the task plug-in module, the task client module and the pre-configuration module are included. The pre-configuration module is used for setting the pre-delivered task. The task plug-in module is used for realizing the definition of the standard interface of the task center module and loading to the task center module. The task client is used for generating task information and then handing over the task information to the task center module for scheduling and execution. The task center module is used for scheduling the flow of the system, interfacing with the task client module and interacting with the task plug-in module. The pre-configuration module includes a routing configuration module, a cluster configuration module and a resource configuration module. The routing value setting is the routing value corresponding to the routing rule. 2.The dynamic task expansion scheduling system of claim 1, wherein: The task center module includes an abstraction module and a matching module. The matching module is used for screening the scheduled task and locating the resource information of the executable task.

3. The dynamic task expansion scheduling system of claim 1, wherein: The cluster configuration module includes the belonging routing setting, the function characteristic setting and the task upper limit setting.

4. The dynamic task expansion scheduling system of claim 2, wherein: The routing matching is used for the task center module to interface with the task client module. The cluster matching is used for screening the cluster under the routing meeting the condition. The resource matching is used for processing before the task is delivered.

5. A dynamic extension task scheduling method for the scheduling of a video task, the method comprising the dynamic extension task scheduling system of any one of claims 1-4. The task scheduling information is set by the pre-configuration module. The task is assembled by the task client module.

6. A storage medium, characterized by The task is operated by the task plug-in module.

7. An electronic device, comprising: The task operation information includes the unique task scheduling rule, the definition of the task function characteristic and the dynamically extendable task. The task operation includes the starting, stopping and deleting of the task. The storage medium is realized by the dynamic extension task scheduling system of any one of claims 1-4. The electronic device is realized by the dynamic extension task scheduling system of any one of claims 1-4.

Images