A task scheduling system

Abstract

An embodiment of the present invention relates to a task scheduling system, comprising: an infrastructure cluster, a resource manager, a decision engine, a cloud controller, a workflow engine, a real-time task client, a real-time task processing engine, an offline task client, and an offline task processing engine; the resource manager is configured to monitor resources and synchronize monitoring information with the cloud controller; the decision engine is configured to plan tasks and send planning results to the cloud controller; the cloud controller is configured to push workflow tasks to the workflow engine; the workflow engine is configured to create, edit, query, decompose, distribute, and update status of workflow tasks; the real-time task processing engine is configured to perform real-time task calculations on a specified infrastructure; and the offline task processing engine is configured to perform offline task calculations on the specified infrastructure during the offline task startup period. The system of the present invention ensures that computing tasks with strong real-time requirements are executed in a timely manner.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a task scheduling system. Background Art

[0002] In connected vehicle applications, there are a large number of highly concurrent vehicle-road collaborative data computing tasks. Conventional task scheduling systems use a first-come, first-served basis to allocate the software and hardware resources of the cloud computing infrastructure for these highly concurrent tasks, without considering the real-time nature of the tasks themselves. However, some computing tasks in connected vehicle applications do not require strong real-time performance, but they do require significant hardware and software resources. If conventional task scheduling systems are still used, a large number of these highly real-time computing tasks may not be completed in a timely manner. Summary of the Invention

[0003] The purpose of the present invention is to address the shortcomings of the existing technology and provide a task scheduling system. The system includes: an infrastructure cluster, a resource manager, a decision engine, a cloud controller, a workflow engine, a real-time task client, a real-time task processing engine, an offline task client, and an offline task processing engine. The system of the present invention fully considers the real-time characteristics of tasks when scheduling tasks and also adds a priority attribute to each task. The system of the present invention not only ensures that real-time tasks are extracted in advance, but also further optimizes the extraction order of real-time tasks based on priority. Applying the system of the present invention to vehicle networks can ensure that a large number of computing tasks with strong real-time requirements can be executed in a timely manner.

[0004] To achieve the above objectives, an embodiment of the present invention provides a task scheduling system, the system comprising: an infrastructure cluster, a resource manager, a decision engine, a cloud controller, a workflow engine, a real-time task client, a real-time task processing engine, an offline task client, and an offline task processing engine;

[0005] The infrastructure cluster includes multiple infrastructures;

[0006] The resource manager is connected to the infrastructure cluster, the cloud controller, the real-time task processing engine, and the offline task processing engine respectively; the resource manager is used to monitor the resource usage of each infrastructure in the infrastructure cluster to obtain a corresponding real-time monitoring data set; the resource manager is also used to share data with the cloud controller based on the real-time monitoring data set; the resource manager is also used to allocate infrastructure to the real-time task processing engine or the offline task processing engine;

[0007] The decision engine is connected to the cloud controller; the decision engine is used to perform task planning processing based on the input vehicle-road cooperative data to obtain a corresponding planning task data set, and send a decision task push instruction carrying the planning task data set to the cloud controller; the planning task data set includes a planning task name, a planning task type, a task resource configuration list and task input data, and the planning task type includes a real-time task type and an offline task type;

[0008] The cloud controller is connected to the workflow engine; the cloud controller is used to perform workflow task data conversion processing based on the planning task data set and the real-time monitoring data set to generate corresponding workflow task data packets, and push workflow tasks to the workflow engine based on the workflow task data packets;

[0009] The workflow engine is connected to the real-time task client and the offline task client respectively; the workflow engine is used to create a workflow task according to the received workflow task data packet; the workflow engine is also used to edit and query the workflow task; the workflow engine is also used to decompose, distribute and update the status of the workflow task;

[0010] The real-time task client is connected to the real-time task processing engine; the real-time task client is used to forward the current subtask data packet sent by the workflow engine to the real-time task processing engine, and forward the subtask execution result data sent back by the real-time task processing engine to the workflow engine;

[0011] The real-time task processing engine is connected to the resource manager and the infrastructure cluster respectively; the real-time task processing engine is used to perform facility allocation application processing on the resource manager according to the current subtask data packet to obtain the corresponding allocated facility resources in the infrastructure cluster, and use the allocated facility resources to perform real-time task calculation processing on the current subtask data packet to obtain corresponding subtask execution result data, and send the subtask execution result data back to the real-time task client;

[0012] The offline task client is connected to the offline task processing engine; the offline task client is used to forward the current subtask data packet sent by the workflow engine to the offline task processing engine, and forward the subtask execution result data sent back by the offline task processing engine to the workflow engine;

[0013] The offline task processing engine is connected to the resource manager and the infrastructure cluster respectively; the offline task processing engine is used to perform offline task start time query processing based on the current subtask data packet to obtain the corresponding offline task start time period, and to perform facility allocation application processing on the resource manager based on the offline task start time period and the current subtask data packet to obtain the corresponding allocated facility resources in the infrastructure cluster, and to use the allocated facility resources to perform offline task calculation processing based on the current subtask data packet to obtain the corresponding subtask execution result data, and to send the subtask execution result data back to the offline task client.

[0014] Preferably, the infrastructure includes at least a switch, a computer, a server, and a database;

[0015] The infrastructure resources include at least CPU resources, memory resources, hard disk resources, network traffic resources, and computing interface resources.

[0016] Preferably, the resource manager includes a resource monitoring module and a resource allocation module; the resource monitoring module is connected to the cloud controller and the resource allocation module respectively; the resource allocation module is connected to the real-time task processing engine or the offline task processing engine respectively;

[0017] The resource manager is specifically used to, when performing resource monitoring on the resource usage status of each infrastructure in the infrastructure cluster, pre-create an infrastructure resource set by the resource monitoring module to store the usage status of various types of resources of each infrastructure in the infrastructure cluster, and copy the content of the infrastructure resource set once every specified time interval to obtain the corresponding real-time monitoring data set; wherein, the infrastructure resource set includes multiple infrastructure arrays, the infrastructure array includes infrastructure identification and multiple facility resource arrays; the facility resource array includes resource type, the resource type includes CPU type, memory type, hard disk type, network traffic type and computing interface type; when the resource type is CPU type, memory type, hard disk type or network traffic type, the facility resource array also includes the total amount of resources, the amount of resources occupied and the amount of resources remaining to be allocated, and when the resource type is computing interface type, the facility resource array also includes a computing interface list;

[0018] The resource manager is specifically configured to, when performing data sharing processing with the cloud controller based on the real-time monitoring data set, send a data sharing instruction carrying the real-time monitoring data set to the cloud controller each time the resource monitoring module generates the real-time monitoring data set;

[0019] The resource manager is specifically configured to receive, by the resource allocation module, an infrastructure application instruction sent by the real-time task processing engine or the offline task processing engine when performing infrastructure allocation processing on the real-time task processing engine or the offline task processing engine, and extract a current resource configuration list from the infrastructure application instruction, and mark the infrastructure array whose remaining allocatable amount of each resource in the infrastructure resource set meets the requirements of the current resource configuration list as a matching facility array, and select one from all the obtained matching facility arrays as the currently allocated facility array, and adjust the resource occupied amount and the remaining allocatable amount of the currently allocated facility array according to the current resource configuration list, and use the infrastructure in the infrastructure cluster corresponding to the currently allocated facility array as the allocated facility resource corresponding to the currently allocated infrastructure application instruction of the real-time task processing engine or the offline task processing engine.

[0020] Preferably, the decision engine is specifically used to extract data labels and data bodies from the vehicle-road cooperative data when performing task planning processing based on the input vehicle-road cooperative data; and use the data body as the corresponding task input data; and use the corresponding record in which the data label field in the locally preset vehicle-road cooperative data task correspondence table matches the data label as the matching corresponding record, and extract the task name field, task type field and task configuration list field of the matching corresponding record as the corresponding planned task name, the planned task type and the task resource configuration list; and the corresponding planned task data set is composed of the obtained planned task name, the planned task type, the task resource configuration list and the task input data; wherein, the vehicle-road cooperative data task correspondence table includes multiple corresponding records; the corresponding record includes the data label field, the task name field, the task type field and the task configuration list field.

[0021] Preferably, the cloud controller includes a decision engine interface, a resource information synchronization interface, a task scheduling module, a dynamic task module and a workflow engine interface; the decision engine interface is connected to the decision engine and the task scheduling module respectively; the resource information synchronization interface is connected to the resource manager and the dynamic task module respectively; the task scheduling module is connected to the dynamic task module; the dynamic task module is connected to the workflow engine interface; the workflow engine interface is connected to the workflow engine;

[0022] The cloud controller is specifically used to extract the real-time monitoring data set from the data sharing instruction sent by the resource manager and send it to the dynamic task module when performing workflow task data conversion processing according to the planned task data set and the real-time monitoring data set by the resource information synchronization interface; and extract the planned task data set from the decision task push instruction sent by the decision engine by the decision engine interface and send it to the task scheduling module; and the task scheduling module queries the local preset task priority and subtask chain relationship table according to the planned task name of the planned task data set, and records the relationship record in the relationship table that matches the task name field and the planned task name as the corresponding matching relationship record, and extracts the task priority field and subtask chain field of the matching relationship record as the corresponding Plan task priorities and plan subtask chains, and send a corresponding orchestration data set composed of the planned task priorities, the planned subtask chains and the planned task data set to the dynamic task module; and the dynamic task module compares the task resource configuration list of the orchestration data set with the real-time monitoring data set. If the resources in the real-time monitoring data set can meet the configuration requirements of the task resource configuration list, the orchestration data set is data-encapsulated according to the data encapsulation rules agreed with the workflow engine to generate the corresponding workflow task data packet; wherein, the task priority and subtask chain relationship table includes a plurality of relationship records; the relationship record includes the task name field, the task priority field and the subtask chain field; the subtask chain stored in the subtask chain field is composed of a plurality of subtask names in sorted order;

[0023] The cloud controller is specifically configured to, when the workflow engine is pushed to process the workflow task according to the workflow task data packet, send the workflow task push instruction carrying the workflow task data packet to the workflow engine through the workflow engine interface by the dynamic task module.

[0024] Preferably, the workflow engine includes a visualization module, an interface module, a database module, a task decomposer module and a timer module; the interface module includes a JAVA API interface unit, a Rest API interface unit and a CLI interface unit, the Rest API interface unit is connected to the cloud controller and the database module respectively, and the JAVA API interface unit and the CLI interface unit are connected to the visualization module and the database module respectively; the task decomposer module includes a task extraction unit, a task coordination unit and a task execution unit, the task extraction unit is connected to the database module and the task coordination unit respectively, the task coordination unit is connected to the task execution unit, and the task execution unit is connected to the timer module; the timer module is connected to the real-time task client and the offline task client respectively.

[0025] Preferably, the workflow engine is specifically used to, when performing workflow task creation processing according to the received workflow task data packet, parse the workflow task data packet in the workflow task creation instruction sent by the cloud controller by the Rest API interface unit to obtain the corresponding planned task priority, planned subtask chain, the planned task name, the planned task type, the task resource configuration list and the task input data; and perform a new workflow task record addition operation in the workflow task database locally preset by the database module; and set the task name field of the newly added workflow task record to the planned task name, the task priority field to the planned task priority, the task type field to the planned task type, the task resource configuration list field to the task resource configuration list, the initial input data field to the task input data, the subtask chain field to the planned subtask chain, the task execution condition field to the unconditional state, the task status field to the unstarted state, and the task result field to empty;

[0026] The workflow task record includes the task name field, the task priority field, the task type field, the task resource configuration list field, the initial input data field, the subtask chain field, the task execution condition field, the task status field and the task result field.

[0027] Preferably, the workflow engine body is specifically used to, when editing and querying the workflow task, have the visualization module edit and process each field of the workflow task record in the workflow task database through the JAVA API interface unit or the CLI interface unit, and have the visualization module query and process each workflow task record in the workflow task database through the JAVA API interface unit or the CLI interface unit.

[0028] Preferably, the workflow engine is specifically configured to, when the workflow tasks are decomposed, distributed and status updated, query the workflow task records in the workflow task database whose task status field is in the unstarted state by the task extraction unit, group all the workflow task records obtained by the query according to the task priority field, select one workflow task record from the corresponding group whose task priority field is the highest priority as the current workflow task record, modify the task status field of the current workflow task record to the started state, push the current workflow task record to the task coordination unit, and after the push is completed, wait for receiving the returned data from the task coordination unit, and when the current task result data sent back by the task coordination unit is received, set the task result field of the current workflow task record in the workflow task database to the current task result data, and modify the task status field to the completed state;

[0029] The task coordination unit identifies the task execution condition field of the current workflow task record pushed by the task extraction unit, and pushes the current workflow task record to the task execution unit if the task execution condition field is in an unconditional state; if the task execution condition field is not in an unconditional state, pushes the current workflow task record to the task execution unit when the execution condition set in the task execution condition field is satisfied, and waits for receiving the returned data from the task execution unit after pushing, and immediately sends the returned data to the task extraction unit when receiving the current task result data sent back by the task execution unit;

[0030] The task execution unit extracts the task type field as the current task type, the task resource configuration list field as the current resource configuration list, the initial input data field as the current input data, and the subtask chain field as the current subtask name sequence from the current workflow task record pushed by the task coordination unit, and extracts the first subtask name in the current subtask name sequence as the current subtask name, and the current task type, the current resource configuration list, the current input data and the current subtask name form the corresponding current subtask data packet, and sets the task execution time parameter of the timer module and pushes the current subtask data packet to the timer module, and the timer The subtask execution result data returned by the timer module is used as the new current input data, and the next subtask name in the current subtask name sequence is extracted as the new current subtask name, and a new current subtask data packet is composed of the current task type, the current resource configuration list, the new current input data and the new current subtask name, and the task execution time parameter of the timer module is reset and the new current subtask data packet is pushed to the timer module again, and so on, until the timer module returns the subtask execution result data corresponding to the last subtask name, and sends the last subtask execution result data as the corresponding current task result data to the task extraction unit;

[0031] The timer module identifies the current task type in the current subtask data packet pushed by the task execution unit. If the current task type is a real-time task type, the current subtask data packet is pushed to the real-time task client at a time point that meets the task execution time parameter. If the current task type is an offline task type, the current subtask data packet is pushed to the offline task client at a time point that meets the task execution time parameter. After pushing, the module waits for receiving the returned data from the real-time task client or the offline task client, and immediately sends the subtask execution result data returned by the real-time task client or the offline task client back to the task execution unit when receiving it.

[0032] Preferably, the real-time task processing engine is specifically configured to extract a current resource configuration list from the current subtask data packet when processing the facility allocation application to the resource manager according to the current subtask data packet, and send an infrastructure application instruction carrying the current resource configuration list to the resource manager to obtain the corresponding allocated facility resources in the infrastructure cluster;

[0033] The real-time task processing engine is specifically configured to extract current input data and a current subtask name from the current subtask data packet when performing real-time task computing processing according to the current subtask data packet with the aid of the allocated facility resources, record a local computing interface that matches the current subtask name as the current computing interface, use the allocated facility resources as computing resources for the current computing interface, send the current input data to the current computing interface, and use the return data from the current computing interface as the corresponding subtask execution result data;

[0034] The offline task processing engine is specifically configured to extract the current subtask name from the current subtask data packet when performing offline task start time query processing based on the current subtask data packet, and extract the start period field of the time record whose task name field in the locally preset offline task start schedule matches the current subtask name as the corresponding offline task start period; wherein the offline task start schedule includes a plurality of time records; and each time record includes the task name field and the start period field;

[0035] The offline task processing engine is specifically configured to, when performing facility allocation application processing on the resource manager based on the offline task startup period and the current subtask data packet, extract a current resource configuration list from the current subtask data packet at a time point when the offline task startup period is satisfied, and send an infrastructure application instruction carrying the current resource configuration list to the resource manager, thereby obtaining the corresponding allocated facility resources in the infrastructure cluster;

[0036] The offline task processing engine is specifically used to extract the current input data and the current subtask name from the current subtask data packet when performing offline task calculation processing based on the current subtask data packet with the help of the allocated facility resources, and record the local computing interface that matches the current subtask name as the current computing interface, and use the allocated facility resources as the computing resources of the current computing interface, and send the current input data to the current computing interface, and use the return data of the current computing interface as the corresponding subtask execution result data.

[0037] An embodiment of the present invention provides a task scheduling system, which includes: an infrastructure cluster, a resource manager, a decision engine, a cloud controller, a workflow engine, a real-time task client, a real-time task processing engine, an offline task client, and an offline task processing engine. Among them, the resource manager is used to monitor the resource usage status of each infrastructure in the infrastructure cluster, and synchronize the real-time monitoring data set obtained by monitoring to the cloud controller; the decision engine is used to plan the planning task data set of real-time or offline tasks that need to be executed based on the vehicle-road collaborative data and send it to the cloud controller; the cloud controller uses the task orchestration module to set task priorities, configure task resource configuration lists, and refine subtasks according to the planning task data set to obtain an orchestration data set, and then the dynamic task module encapsulates the orchestration data set into a workflow task data packet that can be recognized by the workflow engine under the premise that the resources in the real-time monitoring data set can meet the task resource configuration list, and then the workflow task data packet is sent to the workflow engine by the workflow engine interface; the workflow engine is used to create workflow tasks based on the received workflow task data packet, and the task decomposer The module extracts and decomposes the created workflow tasks according to their priorities, and pushes the decomposed subtask data packets to the real-time or offline task client through the timer module based on the task type of the extracted task; the real-time or offline task client further distributes the subtask data packets to the corresponding real-time or offline task processing engine; the real-time task processing engine immediately applies for available infrastructure through the resource manager after receiving the subtask data packets, performs real-time task calculation based on the applied infrastructure, and promptly sends the calculation results back to the workflow engine; the offline task processing engine does not immediately apply for resources to perform task calculation after receiving the subtask data packets, but sets different task execution times for different subtasks, and applies for available infrastructure through the resource manager to perform offline task calculation at the time specified by the task execution time, and then sends the calculation results back to the workflow engine. The system of the present invention fully considers the real-time characteristics of tasks when performing task scheduling and adds a priority attribute to each task; the system of the present invention not only ensures that real-time tasks are extracted in advance, but also further optimizes the extraction order of real-time tasks based on priority; the application of the system of the present invention to the vehicle network can ensure that a large number of computing tasks with strong real-time requirements can be executed in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A structural diagram of a task scheduling system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0040] Figure 1 A structural diagram of a task scheduling system provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the task scheduling system of an embodiment of the present invention includes: an infrastructure cluster 1, a resource manager 2, a decision engine 3, a cloud controller 4, a workflow engine 5, a real-time task client 6, a real-time task processing engine 7, an offline task client 8 and an offline task processing engine 9.

[0041] (1) Infrastructure Cluster 1

[0042] Infrastructure cluster 1 includes multiple infrastructures;

[0043] Among them, the infrastructure includes at least switches, computers, servers, and databases; the infrastructure resources include at least CPU resources, memory resources, hard disk resources, network traffic resources, and computing interface resources.

[0044] Here, each infrastructure can be regarded as an independent device, which can be a switch, computer, server or database in the network, or other devices in the network that can be used to perform task computing. They are not listed one by one here; based on the task computing requirements in the system, attention will be paid to the following infrastructure resources: central processing unit (CPU) resources, memory resources, hard disk resources, network traffic resources, and computing interface resources, among which: CPU resources, memory resources, hard disk resources, and network traffic resources are hardware resources, and computing interface resources are software interface resources.

[0045] (2) Explorer 2

[0046] The resource manager 2 is connected to the infrastructure cluster 1, the cloud controller 4, the real-time task processing engine 7 and the offline task processing engine 9 respectively; wherein, the resource manager 2 includes a resource monitoring module 21 and a resource allocation module 22; the resource monitoring module 21 is connected to the cloud controller 4 and the resource allocation module 22 respectively; the resource allocation module 22 is connected to the real-time task processing engine 7 or the offline task processing engine 9 respectively.

[0047] The resource manager 2 is used to monitor the resource usage of each infrastructure in the infrastructure cluster 1 to obtain the corresponding real-time monitoring data set; the resource manager 2 is also used to share data with the cloud controller 4 based on the real-time monitoring data set; the resource manager 2 is also used to perform infrastructure allocation processing on the real-time task processing engine 7 or the offline task processing engine 9.

[0048] In another specific implementation of the embodiment of the present invention, the resource manager 2 is specifically configured to, when performing resource monitoring processing on the resource usage status of each infrastructure in the infrastructure cluster 1, pre-create an infrastructure resource set by the resource monitoring module 21 to store the usage status of various types of resources of each infrastructure in the infrastructure cluster 1, and replicate the content of the infrastructure resource set at a specified time interval to obtain a corresponding real-time monitoring data set;

[0049] Among them, the infrastructure resource collection includes multiple infrastructure arrays, the infrastructure array includes infrastructure identification and multiple facility resource arrays; the facility resource array includes resource types, and the resource types include CPU type, memory type, hard disk type, network traffic type and computing interface type; when the resource type is CPU type, memory type, hard disk type or network traffic type, the facility resource array also includes the total resource amount, the occupied resource amount and the remaining allocatable resource amount; when the resource type is a computing interface type, the facility resource array also includes a computing interface list; the computing interface list is composed of multiple computing interface records, and each computing interface record includes a computing interface name field and a computing interface call field.

[0050] Here, each infrastructure array in the infrastructure resource set corresponds to one infrastructure, and the infrastructure identifier of the infrastructure array is the unique identifier of the device of the corresponding infrastructure; each facility resource array includes a resource type (CPU type, memory type, hard disk type, network traffic type and computing interface type); when the resource type is CPU type, memory type, hard disk type or network traffic type, the complete structure of the facility resource array consists of the resource type, the total amount of resources, the occupied amount of resources and the remaining allocatable amount of resources; when the resource type is computing interface type, the complete structure of the facility resource array consists of the resource type and the computing interface list; each computing interface record in the computing interface list corresponds to a software computing interface, and the computing interface name field in the computing interface record is used to store the name of the software computing interface, and the computing interface call field is used to store the remote call socket of the software computing interface;

[0051] It should be noted that not every infrastructure has the above six types of resources at the same time. In the facility resource array corresponding to the CPU type, memory type, hard disk type or network traffic type that is not available, the total resource amount, resource occupied amount and resource remaining allocatable amount are all 0. In the facility resource array corresponding to the computing interface type that is not available, the computing interface list is empty. In the facility resource array corresponding to the CPU type, memory type, hard disk type or network traffic type that is available, the total resource amount, resource occupied amount and resource remaining allocatable amount should satisfy the following relationship: total resource amount > resource remaining allocatable amount + resource occupied amount;

[0052] In addition, the statistical calculation methods for the total amount of resources, occupied resources, and remaining allocatable resources in the facility resource array corresponding to the CPU type, memory type, hard disk type, or network traffic type are different: if the resource type is CPU type, the total amount of resources is actually the total number of cores in the corresponding processor (CPU), the occupied resources are actually the total number of occupied cores in the corresponding processor (CPU), and the remaining allocatable resources = (total number of cores - total number of occupied cores - number of reserved cores), where the number of reserved cores ≥ 1; if the resource type is memory / hard disk type, the total amount of resources is actually the total storage capacity of the corresponding memory (memory / hard disk), and the occupied resources are actually the total number of cores in the corresponding memory (memory / hard disk). For example, if the resource type is a network traffic type, the total amount of resources is actually the rated data forwarding capacity of the data exchange device on the current infrastructure (such as the network card of a computer, server, or database, or the switching backplane of a switch), and the occupied amount of resources is actually the current data forwarding capacity of the data exchange device on the current infrastructure (such as the network card of a computer, server, or database, or the switching backplane of a switch). The remaining allocatable amount of resources is = (rated data forwarding capacity - current data forwarding capacity) * β, where β is the preset available percentage, and β < 100%.

[0053] Here, the resource monitoring module 21 of the resource manager 2 is a specific execution module for monitoring the resource usage status of each infrastructure in the infrastructure cluster 1. The resource monitoring module 21 creates an infrastructure resource collection locally at the initial stage of system construction, and updates the infrastructure resource collection accordingly according to the current operation content each time infrastructure is added or deleted or the available resources of the infrastructure in the infrastructure cluster 1 are configured or adjusted; during the operation of the system, the resource monitoring module 21 will copy the content of the infrastructure resource collection once at a pre-set time interval, that is, at a specified time interval, to obtain a real-time monitoring data collection. The real-time monitoring data collection is actually the mirror data of the infrastructure resource collection at a certain moment.

[0054] In another specific implementation method of an embodiment of the present invention, the resource manager 2 is specifically used to perform data sharing processing with the cloud controller 4 based on the real-time monitoring data set, and the resource monitoring module 21 sends a data sharing instruction carrying the real-time monitoring data set to the cloud controller 4 each time the real-time monitoring data set is generated.

[0055] In another specific implementation of the embodiment of the present invention, the resource manager 2 is specifically used to receive the infrastructure application instruction sent by the real-time task processing engine 7 or the offline task processing engine 9 by the resource allocation module 22 when performing infrastructure allocation processing on the real-time task processing engine 7 or the offline task processing engine 9, and extract the current resource configuration list from the infrastructure application instruction, and mark the infrastructure array whose remaining allocatable amount of each type of resource in the infrastructure resource set meets the requirements of the current resource configuration list as a matching facility array, and select one from all the obtained matching facility arrays as the current allocation facility array, and adjust the resource occupied amount and the resource remaining allocatable amount of the current allocation facility array according to the current resource configuration list, and use the infrastructure in the infrastructure cluster corresponding to the current allocation facility array as the allocation facility resource corresponding to the current infrastructure application instruction of the real-time task processing engine 7 or the offline task processing engine 9;

[0056] Among them, the current resource configuration list is composed of multiple configuration resource arrays; the configuration resource array includes the configuration resource type; the configuration resource type includes CPU type, memory type, hard disk type, network traffic type and computing interface type; when the configuration resource type is CPU type, memory type, hard disk type or network traffic type, the configuration resource array also includes the configuration resource quantity, and when the configuration resource type is computing interface type, the configuration resource array also includes a configuration computing interface sequence; the configuration computing interface sequence includes multiple configuration computing interface names.

[0057] In another specific implementation of the embodiment of the present invention, the resource allocation module 22 is specifically configured to poll each infrastructure array in the infrastructure resource set when marking an infrastructure array whose remaining allocatable amount of each type of resource in the infrastructure resource set meets the requirements of the current resource configuration list as a matching facility array; during polling, the infrastructure array currently being polled is used as the current infrastructure array; and when the remaining allocatable amount of resources of the facility resource arrays whose resource types are specifically CPU type, memory type, hard disk type, and network traffic type in the current infrastructure array is not less than the number of configuration resources of the configuration resource arrays whose configuration resource types correspond to CPU type, memory type, hard disk type, and network traffic type in the current resource configuration list, confirm whether the configuration computing interface sequence in the current resource configuration list is empty; if it is confirmed that the configuration computing interface sequence is not empty, confirm whether each configuration computing interface name in the configuration computing interface sequence has a matching computing interface name field in the computing interface list of the current infrastructure array; if it is confirmed that each configuration computing interface name has a matching computing interface name field, the current infrastructure array is recorded as the corresponding matching facility array and the polling is continued to the next infrastructure array until the polling of the last infrastructure array is completed.

[0058] In another specific implementation of the embodiment of the present invention, the resource allocation module 22 is specifically used to adjust the resource occupied amount and the remaining allocatable amount of the currently allocated facility array according to the current resource configuration list, and add the resource occupied amount of the facility resource array of the currently allocated facility array with the corresponding resource types of CPU type, memory type, hard disk type, and network traffic type according to the number of configured resources of the configured resource array with the configured resource types of CPU type, memory type, hard disk type, and network traffic type in the current resource configuration list, and calculate and set the new remaining allocatable amount of each resource type according to the new resource occupied amount of each type according to the calculation method of the remaining allocatable amount of each resource type.

[0059] As will be explained later, each time a real-time task processing engine 7 or offline task processing engine 9 in the system processes a corresponding real-time or offline task computation, it first requests infrastructure resources from the resource manager 2. Upon initiating this request, the real-time task processing engine 7 or offline task processing engine 9 sends an infrastructure request instruction to the resource manager 2, including a list of required resources for the task, or the current resource configuration list. The resource allocation module 22 of the resource manager 2 is the specific execution module for allocating infrastructure resources to the real-time task processing engine 7 or offline task processing engine 9. After receiving the infrastructure application instruction, the resource allocation module 22 first extracts the current resource configuration list from the instruction and compares it with the infrastructure array corresponding to each infrastructure in the infrastructure resource set. During the comparison, if the remaining allocatable amount of resources such as CPU, memory, hard disk, network traffic, etc. of the current infrastructure array is not less than the application quantity in the current resource configuration list, that is, the number of configuration resources corresponding to each type, and the calculation interface list of the current infrastructure array also includes all the configuration calculation interface names required by the current resource configuration list, then the current infrastructure array is regarded as a matching facility array that meets the application requirements; there may be more than one matching facility array in the infrastructure resource set that meets the requirements of the current resource configuration list, so the resource allocation module 22 will select one from them as the current allocation facility array; after selecting the current allocation facility array, the resource allocation module 22 will select one from them as the current allocation facility array. The source allocation module 22 needs to pre-occupy relevant resources in the infrastructure resource set, that is, according to the number of configured resources in the configuration resource array with the resource types of CPU type, memory type, hard disk type and network traffic type in the current resource configuration list, the resource occupied amount of the facility resource array with the corresponding resource types of CPU type, memory type, hard disk type and network traffic type in the current allocation facility array is incrementally set and the remaining allocable amount of resources corresponding to the resource occupied amount is reduced and reset at the same time; after the resource allocation module 22 pre-occupies relevant resources in the infrastructure resource set, the infrastructure corresponding to the current allocation facility array is used as the allocation facility resource corresponding to the current infrastructure application instruction of the real-time task processing engine 7 or the offline task processing engine 9 for calling by the real-time task processing engine 7 or the offline task processing engine 9.

[0060] (3) Decision Engine 3

[0061] The decision engine 3 is connected to the cloud controller 4; the decision engine 3 is used to perform task planning processing based on the input vehicle-road cooperative data to obtain a corresponding planning task data set, and send a decision task push instruction carrying the planning task data set to the cloud controller 4;

[0062] Among them, vehicle-road collaborative data includes data labels and data bodies; the planning task data set includes the planning task name, planning task type, task resource configuration list and task input data, and the planning task type includes real-time task type and offline task type.

[0063] In another specific implementation of the embodiment of the present invention, the decision engine 3 is specifically used to extract data labels and data bodies from the vehicle-road cooperative data when performing task planning processing based on the input vehicle-road cooperative data; and use the data bodies as corresponding task input data; and use the corresponding records in which the data label fields and data labels in the locally preset vehicle-road cooperative data task correspondence table match as matching corresponding records, and extract the task name field, task type field and task configuration list field of the matching corresponding records as the corresponding planning task name, planning task type and task resource configuration list respectively; and the corresponding planning task data set is composed of the obtained planning task name, planning task type, task resource configuration list and task input data;

[0064] Among them, the vehicle-road collaborative data task correspondence table includes multiple corresponding records; the corresponding records include data label fields, task name fields, task type fields and task configuration list fields.

[0065] Here, the data body of the V2X data is the original input data used for the calculation, namely the task input data, and the data tag corresponds to a specific computational task tag. The decision engine 3 can obtain the name information of the computational task corresponding to the computational task tag, namely the planned task name, the task type information of the computational task, namely the planned task type, and the resource configuration information required to execute the computational task, namely the task resource configuration list, by looking up the table. The data format of the task resource configuration list is the same as the data format of the current resource configuration list mentioned above. Finally, the decision engine 3 sends the planned task data set consisting of the planned task name, planned task type, task resource configuration list, and task input data to the cloud controller 4. It should be noted that the decision engine 3 is a pre-data preparation module of the task scheduling system of the present invention. The decision engine 3 can be directly set up locally on a vehicle, road infrastructure, or other front-end device of the Internet of Vehicles, or it can be set up in the cloud. If it is set up locally on a vehicle, road infrastructure, or other front-end device, the vehicle, road infrastructure, or other front-end device inputs the V2X data to the decision engine 3 via a local call interface. If it is set up in the cloud, the vehicle, road infrastructure, or other front-end device inputs the V2X data to the decision engine 3 via a remote call interface.

[0066] (4) Cloud Controller 4

[0067] The cloud controller 4 is connected to the workflow engine 5; wherein, the cloud controller 4 includes a decision engine interface 41, a resource information synchronization interface 42, a task orchestration module 43, a dynamic task module 44 and a workflow engine interface 45; the decision engine interface 41 is connected to the decision engine 3 and the task orchestration module 43 respectively; the resource information synchronization interface 42 is connected to the resource manager 2 and the dynamic task module 44 respectively; the task orchestration module 43 is connected to the dynamic task module 44; the dynamic task module 44 is connected to the workflow engine interface 45; the workflow engine interface 45 is connected to the workflow engine 5.

[0068] The cloud controller 4 is used to perform workflow task data conversion processing based on the planning task data set and the real-time monitoring data set to generate corresponding workflow task data packets, and to push workflow tasks to the workflow engine 5 based on the workflow task data packets.

[0069] In another specific implementation of the embodiment of the present invention, the cloud controller 4 is specifically configured to, when performing workflow task data conversion processing based on the planned task data set and the real-time monitoring data set, extract the real-time monitoring data set from the data sharing instruction sent by the resource manager 2 by the resource information synchronization interface 42 and send it to the dynamic task module 44; and extract the planned task data set from the decision task push instruction sent by the decision engine 3 by the decision engine interface 41 and send it to the task orchestration module 43;

[0070] The task scheduling module 43 queries the locally preset task priority and subtask chain relationship table according to the planned task name of the planned task data set, records the relationship record in the relationship table that matches the task name field and the planned task name as the corresponding matching relationship record, and extracts the task priority field and subtask chain field of the matching relationship record as the corresponding planned task priority and planned subtask chain, and sends the corresponding scheduling data set composed of the planned task priority, planned subtask chain and planned task data set to the dynamic task module 44; wherein, the task priority and subtask chain relationship table includes multiple relationship records; the relationship record includes a task name field, a task priority field and a subtask chain field; the storage content of the task priority field is the priority level; the subtask chain stored in the subtask chain field is composed of multiple subtask names in order; it should be noted that the so-called subtask chain is actually a subtask name sequence, through which all subtask names of the current task can be obtained, and the subtask name sorting relationship of the subtask name sequence corresponds to the execution order of each subtask;

[0071] The dynamic task module 44 compares the task resource configuration list of the orchestration data set with the real-time monitoring data set. If the resources in the real-time monitoring data set can meet the configuration requirements of the task resource configuration list, the orchestration data set is encapsulated according to the data encapsulation rules agreed with the workflow engine 5 to generate a corresponding workflow task data packet.

[0072] In another specific implementation of an embodiment of the present invention, the cloud controller 4 is specifically used to push workflow tasks to the workflow engine 5 according to the workflow task data packet, and the dynamic task module 44 sends the workflow task push instruction carrying the workflow task data packet to the workflow engine 5 through the workflow engine interface 45.

[0073] Here, the cloud controller 4 receives the synchronized real-time monitoring data set received by the resource manager 2 through the resource information synchronization interface 42, receives the planning task data set sent by the decision engine 3 through the decision engine interface 41, and obtains the orchestration data set by performing task priority setting, task resource configuration list configuration, and sub-task refinement according to the planning task data set through the task orchestration module 43. The dynamic task module 44 then encapsulates the orchestration data set into a workflow task data packet that can be recognized by the workflow engine on the premise that the resources in the real-time monitoring data set can meet the task resource configuration list, and then the workflow engine interface 45 sends the workflow task data packet to the workflow engine 5.

[0074] (V) Workflow Engine 5

[0075] The workflow engine 5 is connected to the real-time task client 6 and the offline task client 8 respectively; wherein, the workflow engine 5 includes a visualization module 51, an interface module 52, a database module 53, a task decomposer module 54 and a timer module 55; the interface module 52 includes a JAVA API interface unit 521, a Rest API interface unit 522 and a command-line interface (CLI) interface unit 523, the Rest API interface unit 522 is connected to the cloud controller 4 and the database module 53 respectively, the JAVA API interface unit 521 and the CLI interface unit 523 are connected to the visualization module 51 and the database module 53 respectively; the task decomposer module 54 includes a task extraction unit 541, a task coordination unit 542 and a task execution unit 543, the task extraction unit 541 is connected to the database module 53 and the task coordination unit 542 respectively, the task coordination unit 542 is connected to the task execution unit 543, and the task execution unit 543 is connected to the timer module 55; the timer module 55 is connected to the real-time task client 6 and the offline task client 8 respectively.

[0076] The workflow engine 5 is used to create workflow tasks based on received workflow task data packets; the workflow engine 5 is also used to edit and query workflow tasks; the workflow engine 5 is also used to decompose, distribute and update the status of workflow tasks.

[0077] In another specific implementation of the embodiment of the present invention, the workflow engine 5 is specifically used to, when performing workflow task creation processing based on the received workflow task data packet, parse the workflow task data packet in the workflow task creation instruction sent by the cloud controller 4 by the Rest API interface unit 522 to obtain the corresponding planning task priority, planning subtask chain, planning task name, planning task type, task resource configuration list and task input data; and perform a new workflow task record operation in the workflow task database locally preset by the database module 53; and set the task name field of the newly added workflow task record to the planning task name, the task priority field to the planning task priority, the task type field to the planning task type, the task resource configuration list field to the task resource configuration list, the initial input data field to the task input data, the subtask chain field to the planning subtask chain, the task execution condition field to the unconditional state, the task status field to the unstarted state, and the task result field to empty;

[0078] Among them, the workflow task record includes the task name field, task priority field, task type field, task resource configuration list field, initial input data field, subtask chain field, task execution condition field, task status field and task result field; the storage content of the task priority field is the priority level; the task type field includes real-time task type and offline task type; the task status field includes at least the unstarted state, started state and completed state.

[0079] Here, the database module 53 stores the relevant information of all workflow tasks in the workflow task database preset locally; each workflow task record corresponds to a workflow task; the task execution condition field in the record is used to store the executable restriction condition of the corresponding workflow task. When creating, the field is initialized to an unconditional state, that is, no restriction condition. It can be modified through the visualization interface provided by the visualization module 51. There are two setting modes for this field: one is to set it to an unconditional state, and the other is to set it in a number pair such as condition state <state value>. For example, if the field is set to the time state <2022-01-01 9:30>, it means that the executable restriction condition of the current workflow task is a time restriction condition, that is, at the time 2022-01-01 The current workflow task can only be executed after 9:30. It is not difficult to see that the embodiment of the present invention can delay the execution of any workflow task whose task type field is an offline task type by setting the corresponding conditional status <status value> number pair setting rule; the task status field includes at least the unstarted state, the started state, and the completed state. When the field is created, it is initialized to the unstarted state, indicating that the current workflow task has not yet started execution; the task result field is used to store the final return data of the current workflow task. When created, this field is initialized to empty.

[0080] In another specific implementation of an embodiment of the present invention, the workflow engine 5 is specifically used to edit and process the various fields of the workflow task records in the workflow task database through the JAVA API interface unit 521 or the CLI interface unit 523 when editing and querying the workflow tasks, and the visualization module 51 is used to query and process the various workflow task records in the workflow task database through the JAVA API interface unit 521 or the CLI interface unit 523.

[0081] Here, the visualization module 51 of the workflow engine 5 provides two visualization services, one of which is a browser page-based visualization service. When the visualization module 51 processes this type of service, it provides users with editing, browsing and query functions of workflow task records through a page, and connects with the workflow task database through the JAVA API interface unit 521 to complete the corresponding editing, browsing and query operations; the other is a command line visualization service based on a command console or client. When the visualization module 51 processes this type of service, it provides users with editing, browsing and query functions of workflow task records through a command console or client, and connects with the workflow task database through the CLI interface unit 523 to complete the corresponding editing, browsing and query operations.

[0082] In another specific implementation of the embodiment of the present invention, the workflow engine 5 is specifically configured to, when decomposing, distributing and updating the status of a workflow task, query the workflow task record in the workflow task database whose task status field is in the unstarted state by the task extraction unit 541, group all the workflow task records obtained by the query according to the task priority field, select a workflow task record from the corresponding group whose task priority field is the highest priority as the current workflow task record, modify the task status field of the current workflow task record to the started state, push the current workflow task record to the task coordination unit 542, and after the push is completed, receive and wait for the return data of the task coordination unit 542, and when the current task result data sent back by the task coordination unit 542 is received, set the task result field of the current workflow task record in the workflow task database to the current task result data, and modify the task status field to the completed state;

[0083] It is not difficult to see here that in the embodiment of the present invention, the workflow tasks with higher priorities will be executed in advance;

[0084] The task coordination unit 542 identifies the task execution condition field of the current workflow task record pushed by the task extraction unit 541. If the task execution condition field is in an unconditional state, the current workflow task record is pushed to the task execution unit 543. If the task execution condition field is not in an unconditional state, the current workflow task record is pushed to the task execution unit 543 when the execution condition set in the task execution condition field is satisfied. After pushing, the task coordination unit 542 waits for receiving the returned data from the task execution unit 543. When receiving the current task result data returned by the task execution unit 543, it immediately sends it back to the task extraction unit 541.

[0085] Here, as described above, there are two ways to set the task execution condition field: one is to set it to an unconditional state, and the other is to set it in a number pair format such as condition state < state value >. In the embodiment of the present invention, a setting and identification rule for the condition state < state value > number pair can be set in advance according to application requirements, and based on the rule, it is identified whether the execution condition set in the task execution condition field has been met;

[0086] The task execution unit 543 extracts the task type field as the current task type, the task resource configuration list field as the current resource configuration list, the initial input data field as the current input data, and the subtask chain field as the current subtask name sequence from the current workflow task record pushed by the task coordination unit 542, and extracts the first subtask name in the current subtask name sequence as the current subtask name, and the current task type, current resource configuration list, current input data and current subtask name form the corresponding current subtask data packet, and sets the task execution time parameters of the timer module 55 and pushes the current subtask data packet to the timer module 55, and the timer The subtask execution result data returned by the timer module 55 is used as the new current input data, and the next subtask name in the current subtask name sequence is extracted as the new current subtask name, and a new current subtask data packet is composed of the current task type, the current resource configuration list, the new current input data and the new current subtask name, and the task execution time parameter of the timer module 55 is set again and the new current subtask data packet is pushed to the timer module 55 again, and so on, until the timer module 55 returns the subtask execution result data corresponding to the last subtask name, and sends the last subtask execution result data as the corresponding current task result data to the task extraction unit 541;

[0087] It is not difficult to see here that the task execution unit 543 actually extracts subtasks one by one from the current workflow task record pushed by the task coordination unit 542, and executes each subtask one by one through the timer module 55. During the execution of each subtask, the subtask execution result data of each subtask returned by the timer module 55 is used as the input data of the next subtask;

[0088] The timer module 55 identifies the current task type in the current subtask data packet pushed by the task execution unit 543. If the current task type is a real-time task type, the current subtask data packet is pushed to the real-time task client 6 at the time point when the task execution time parameters are met. If the current task type is an offline task type, the current subtask data packet is pushed to the offline task client 8 at the time point when the task execution time parameters are met. After pushing, the module waits for receiving the returned data from the real-time task client 6 or the offline task client 8. When receiving the subtask execution result data returned by the real-time task client 6 or the offline task client 8, it is immediately sent back to the task execution unit 543.

[0089] Here, there are multiple ways to set the execution time parameter of the timer module 55. If it is set to 0, it means that the immediate execution timer module 55 will immediately push the current subtask data packet to the real-time task client 6 or the offline task client 8 for calculation. If it is set to an absolute time or a relative delay time, the timer module 55 will push the current subtask data packet to the real-time task client 6 or the offline task client 8 for calculation after the corresponding absolute time or relative delay time; the execution time parameter of the timer module 55 can also be set to multiple absolute times or multiple relative delay times, and the corresponding timer module 55 will push the current subtask data packet to the real-time task client 6 or the offline task client 8 for multiple calculations after multiple absolute times or multiple relative delay times.

[0090] (6) Real-time Task Client 6

[0091] The real-time task client 6 is connected to the real-time task processing engine 7; the real-time task client 6 is used to forward the current subtask data packet sent by the workflow engine 5 to the real-time task processing engine 7, and forward the subtask execution result data sent back by the real-time task processing engine 7 to the workflow engine 5.

[0092] (VII) Real-time task processing engine 7

[0093] The real-time task processing engine 7 is connected to the resource manager 2 and the infrastructure cluster 1 respectively; the real-time task processing engine 7 is used to perform facility allocation application processing on the resource manager 2 according to the current subtask data packet to obtain the corresponding allocated facility resources in the infrastructure cluster, and use the allocated facility resources to perform real-time task calculation processing on the current subtask data packet to obtain the corresponding subtask execution result data, and send the subtask execution result data back to the real-time task client 6.

[0094] In another specific implementation method of an embodiment of the present invention, the real-time task processing engine 7 is specifically used to extract the current resource configuration list from the current subtask data packet when processing the facility allocation application to the resource manager 2 based on the current subtask data packet, and send the infrastructure application instruction carrying the current resource configuration list to the resource manager 2 to obtain the corresponding allocated facility resources in the infrastructure cluster.

[0095] Here, the processing flow corresponding to the resource manager 2 is the processing flow of the resource manager 2 performing infrastructure allocation processing on the real-time task processing engine 7 or the offline task processing engine 9 as mentioned above.

[0096] In another specific implementation method of an embodiment of the present invention, the real-time task processing engine 7 is specifically used to extract the current input data and the current subtask name from the current subtask data packet when performing real-time task calculation processing according to the current subtask data packet with the help of allocated facility resources, and record the local computing interface that matches the current subtask name as the current computing interface, and use the allocated facility resources as the computing resources of the current computing interface, and send the current input data to the current computing interface, and use the return data of the current computing interface as the corresponding subtask execution result data.

[0097] (8) Offline Task Client 8

[0098] The offline task client 8 is connected to the offline task processing engine 9; the offline task client 8 is used to forward the current subtask data packet sent by the workflow engine 5 to the offline task processing engine 9, and forward the subtask execution result data sent back by the offline task processing engine 9 to the workflow engine 5.

[0099] (9) Offline Task Processing Engine 9

[0100] The offline task processing engine 9 is connected to the resource manager 2 and the infrastructure cluster 1 respectively; the offline task processing engine 9 is used to perform offline task start time query processing based on the current subtask data packet to obtain the corresponding offline task start period, and to perform facility allocation application processing to the resource manager 2 based on the offline task start period and the current subtask data packet to obtain the corresponding allocated facility resources in the infrastructure cluster, and to use the allocated facility resources to perform offline task calculation processing based on the current subtask data packet to obtain the corresponding subtask execution result data, and then send the subtask execution result data back to the offline task client 8.

[0101] In another specific implementation of the embodiment of the present invention, the offline task processing engine 9 is specifically configured to extract the current subtask name from the current subtask data packet when performing offline task start time query processing based on the current subtask data packet, and extract the start period field of the time record whose task name field in the locally preset offline task start timetable matches the current subtask name as the corresponding offline task start period;

[0102] The offline task start schedule includes multiple time records; the time record includes a task name field and a start period field.

[0103] In another specific implementation of an embodiment of the present invention, the offline task processing engine 9 is specifically used to extract the current resource configuration list from the current subtask data packet at the time point when the offline task start period is met when processing the facility allocation application to the resource manager 2 based on the offline task start period and the current subtask data packet, and send the infrastructure application instruction carrying the current resource configuration list to the resource manager 2 to obtain the corresponding allocated facility resources.

[0104] In addition to the previously mentioned method of delaying offline task execution by setting the task execution condition field of the workflow task record in the workflow task database, embodiments of the present invention also enable delayed offline task execution by setting a local offline task startup schedule in the offline task processing engine 9. Specifically, this is accomplished by adding time records corresponding to each offline task to the offline task startup schedule and setting the startup period field of each time record. Similar to the real-time task processing engine 7, after task startup, the offline task processing engine 9 sends an infrastructure request instruction to the resource manager 2 to obtain allocated facility resources from the infrastructure cluster.

[0105] In another specific implementation of an embodiment of the present invention, the offline task processing engine 9 is specifically used to extract the current input data and the current subtask name from the current subtask data packet when performing offline task calculation processing according to the current subtask data packet with the help of allocated facility resources, and record the local computing interface that matches the current subtask name as the current computing interface, and use the allocated facility resources as the computing resources of the current computing interface, and send the current input data to the current computing interface, and use the return data of the current computing interface as the corresponding subtask execution result data.

[0106] An embodiment of the present invention provides a task scheduling system, which includes: an infrastructure cluster, a resource manager, a decision engine, a cloud controller, a workflow engine, a real-time task client, a real-time task processing engine, an offline task client, and an offline task processing engine. Among them, the resource manager is used to monitor the resource usage status of each infrastructure in the infrastructure cluster, and synchronize the real-time monitoring data set obtained by monitoring to the cloud controller; the decision engine is used to plan the planning task data set of real-time or offline tasks that need to be executed based on the vehicle-road collaborative data and send it to the cloud controller; the cloud controller uses the task orchestration module to set task priorities, configure task resource configuration lists, and refine subtasks according to the planning task data set to obtain an orchestration data set, and then the dynamic task module encapsulates the orchestration data set into a workflow task data packet that can be recognized by the workflow engine under the premise that the resources in the real-time monitoring data set can meet the task resource configuration list, and then the workflow task data packet is sent to the workflow engine by the workflow engine interface; the workflow engine is used to create workflow tasks based on the received workflow task data packet, and the task decomposer The module extracts and decomposes the created workflow tasks according to their priorities, and pushes the decomposed subtask data packets to the real-time or offline task client through the timer module based on the task type of the extracted task; the real-time or offline task client further distributes the subtask data packets to the corresponding real-time or offline task processing engine; the real-time task processing engine immediately applies for available infrastructure through the resource manager after receiving the subtask data packets, performs real-time task calculations based on the applied infrastructure, and promptly sends the calculation results back to the workflow engine; the offline task processing engine does not immediately apply for resources to perform task calculations after receiving the subtask data packets, but sets different task execution times for different subtasks, and applies for available infrastructure through the resource manager to perform offline task calculations at the time specified by the task execution time, and then sends the calculation results back to the workflow engine. The system of the present invention fully considers the real-time characteristics of tasks when performing task scheduling and adds a priority attribute to each task; the system of the present invention can not only ensure that real-time tasks are extracted in advance, but also further optimize the extraction order of real-time tasks based on priority; the application of the system of the present invention to the vehicle network can ensure that a large number of computing tasks with strong real-time requirements can be executed in a timely manner.

[0107] Professionals should also be further aware that the systems, modules, units, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0108] The systems, modules, units, or steps of the algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, software modules executed by a processor, or a combination of the two. The software modules may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0109] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A task scheduling system, characterized in that: The system includes: an infrastructure cluster, a resource manager, a decision engine, a cloud controller, a workflow engine, a real-time task client, a real-time task processing engine, an offline task client, and an offline task processing engine; The infrastructure cluster includes multiple infrastructures; The resource manager is connected to the infrastructure cluster, the cloud controller, the real-time task processing engine, and the offline task processing engine respectively; the resource manager is used to monitor the resource usage of each infrastructure in the infrastructure cluster to obtain a corresponding real-time monitoring data set; the resource manager is also used to share data with the cloud controller based on the real-time monitoring data set; the resource manager is also used to allocate infrastructure to the real-time task processing engine or the offline task processing engine; The decision engine is connected to the cloud controller; the decision engine is used to perform task planning processing based on the input vehicle-road cooperative data to obtain a corresponding planning task data set, and send a decision task push instruction carrying the planning task data set to the cloud controller; the planning task data set includes a planning task name, a planning task type, a task resource configuration list and task input data, and the planning task type includes a real-time task type and an offline task type; The cloud controller is connected to the workflow engine; the cloud controller is used to perform workflow task data conversion processing based on the planning task data set and the real-time monitoring data set to generate corresponding workflow task data packets, and push workflow tasks to the workflow engine based on the workflow task data packets; The workflow engine is connected to the real-time task client and the offline task client respectively; the workflow engine is used to create a workflow task according to the received workflow task data packet; the workflow engine is also used to edit and query the workflow task; the workflow engine is also used to decompose, distribute and update the status of the workflow task; The real-time task client is connected to the real-time task processing engine; the real-time task client is used to forward the current subtask data packet sent by the workflow engine to the real-time task processing engine, and forward the subtask execution result data sent back by the real-time task processing engine to the workflow engine; The real-time task processing engine is connected to the resource manager and the infrastructure cluster respectively; the real-time task processing engine is used to perform facility allocation application processing on the resource manager according to the current subtask data packet to obtain the corresponding allocated facility resources in the infrastructure cluster, and use the allocated facility resources to perform real-time task calculation processing on the current subtask data packet to obtain corresponding subtask execution result data, and send the subtask execution result data back to the real-time task client; The offline task client is connected to the offline task processing engine; the offline task client is used to forward the current subtask data packet sent by the workflow engine to the offline task processing engine, and forward the subtask execution result data sent back by the offline task processing engine to the workflow engine; The offline task processing engine is connected to the resource manager and the infrastructure cluster respectively; the offline task processing engine is used to perform offline task start time query processing based on the current subtask data packet to obtain the corresponding offline task start time period, and to perform facility allocation application processing on the resource manager based on the offline task start time period and the current subtask data packet to obtain the corresponding allocated facility resources in the infrastructure cluster, and to perform offline task calculation processing based on the current subtask data packet with the help of the allocated facility resources to obtain the corresponding subtask execution result data, and to send the subtask execution result data back to the offline task client; The cloud controller includes a decision engine interface, a resource information synchronization interface, a task scheduling module, a dynamic task module and a workflow engine interface; the decision engine interface is connected to the decision engine and the task scheduling module respectively; the resource information synchronization interface is connected to the resource manager and the dynamic task module respectively; the task scheduling module is connected to the dynamic task module; the dynamic task module is connected to the workflow engine interface; the workflow engine interface is connected to the workflow engine; The cloud controller is specifically used to extract the real-time monitoring data set from the data sharing instruction sent by the resource manager and send it to the dynamic task module when performing workflow task data conversion processing according to the planned task data set and the real-time monitoring data set by the resource information synchronization interface; and extract the planned task data set from the decision task push instruction sent by the decision engine by the decision engine interface and send it to the task scheduling module; and the task scheduling module queries the local preset task priority and subtask chain relationship table according to the planned task name of the planned task data set, and records the relationship record in the relationship table that matches the task name field and the planned task name as the corresponding matching relationship record, and extracts the task priority field and subtask chain field of the matching relationship record as the corresponding Plan task priorities and plan subtask chains, and send a corresponding orchestration data set composed of the planned task priorities, the planned subtask chains and the planned task data set to the dynamic task module; and the dynamic task module compares the task resource configuration list of the orchestration data set with the real-time monitoring data set. If the resources in the real-time monitoring data set can meet the configuration requirements of the task resource configuration list, the orchestration data set is data-encapsulated according to the data encapsulation rules agreed with the workflow engine to generate the corresponding workflow task data packet; wherein, the task priority and subtask chain relationship table includes a plurality of relationship records; the relationship record includes the task name field, the task priority field and the subtask chain field; the subtask chain stored in the subtask chain field is composed of a plurality of subtask names in sorted order; The cloud controller is specifically configured to, when the workflow engine is pushed to process the workflow task according to the workflow task data packet, send the workflow task push instruction carrying the workflow task data packet to the workflow engine through the workflow engine interface by the dynamic task module; The workflow engine is used to extract and decompose the created workflow tasks according to their priorities through the task decomposer module, and push the decomposed subtask data packets to the real-time or offline task client through the timer module based on the task type of the extracted tasks.

2. The task scheduling system according to claim 1, characterized in that: The infrastructure at least includes switches, computers, servers, and databases; The infrastructure resources include at least CPU resources, memory resources, hard disk resources, network traffic resources, and computing interface resources.

3. The task scheduling system according to claim 1, characterized in that: The resource manager includes a resource monitoring module and a resource allocation module; the resource monitoring module is connected to the cloud controller and the resource allocation module respectively; the resource allocation module is connected to the real-time task processing engine or the offline task processing engine respectively; The resource manager is specifically used to, when performing resource monitoring on the resource usage status of each infrastructure in the infrastructure cluster, pre-create an infrastructure resource set by the resource monitoring module to store the usage status of various types of resources of each infrastructure in the infrastructure cluster, and copy the content of the infrastructure resource set once every specified time interval to obtain the corresponding real-time monitoring data set; wherein, the infrastructure resource set includes multiple infrastructure arrays, the infrastructure array includes infrastructure identification and multiple facility resource arrays; the facility resource array includes resource type, the resource type includes CPU type, memory type, hard disk type, network traffic type and computing interface type; when the resource type is CPU type, memory type, hard disk type or network traffic type, the facility resource array also includes the total amount of resources, the amount of resources occupied and the amount of resources remaining to be allocated, and when the resource type is computing interface type, the facility resource array also includes a computing interface list; The resource manager is specifically configured to, when performing data sharing processing with the cloud controller based on the real-time monitoring data set, send a data sharing instruction carrying the real-time monitoring data set to the cloud controller each time the resource monitoring module generates the real-time monitoring data set; The resource manager is specifically used to, when performing infrastructure allocation processing on the real-time task processing engine or the offline task processing engine, receive the infrastructure application instruction sent by the real-time task processing engine or the offline task processing engine by the resource allocation module, extract the current resource configuration list from the infrastructure application instruction, mark the infrastructure array whose remaining allocatable amount of each type of resource in the infrastructure resource set meets the requirements of the current resource configuration list as a matching facility array, and select one from all the obtained matching facility arrays as the currently allocated facility array, adjust the resource occupied amount and the remaining allocatable amount of the currently allocated facility array according to the current resource configuration list, and use the infrastructure corresponding to the currently allocated facility array as the allocated facility resource corresponding to the currently allocated infrastructure application instruction of the real-time task processing engine or the offline task processing engine.

4. The task scheduling system according to claim 1, characterized in that: The decision engine is specifically configured to extract data labels and data bodies from the vehicle-road cooperative data when performing task planning processing based on the input vehicle-road cooperative data; and use the data bodies as the corresponding task input data; The corresponding record whose data tag field in the locally preset vehicle-road collaborative data task correspondence table matches the data tag is taken as the matching corresponding record, and the task name field, task type field and task configuration list field of the matching corresponding record are respectively extracted as the corresponding planned task name, the planned task type and the task resource configuration list; and the corresponding planned task data set is composed of the obtained planned task name, the planned task type, the task resource configuration list and the task input data; wherein, the vehicle-road collaborative data task correspondence table includes multiple corresponding records; the corresponding record includes the data tag field, the task name field, the task type field and the task configuration list field.

5. The task scheduling system according to claim 1, characterized in that: The workflow engine includes a visualization module, an interface module, a database module, a task decomposer module and a timer module; the interface module includes a JAVA API interface unit, a Rest API interface unit and a CLI interface unit, the RestAPI interface unit is connected to the cloud controller and the database module respectively, and the JAVA API interface unit and the CLI interface unit are connected to the visualization module and the database module respectively; the task decomposer module includes a task extraction unit, a task coordination unit and a task execution unit, the task extraction unit is connected to the database module and the task coordination unit respectively, the task coordination unit is connected to the task execution unit, and the task execution unit is connected to the timer module; the timer module is connected to the real-time task client and the offline task client respectively.

6. The task scheduling system according to claim 5, characterized in that: The workflow engine is specifically configured to, when performing workflow task creation processing according to the received workflow task data packet, parse the workflow task data packet in the workflow task creation instruction sent by the cloud controller by the Rest API interface unit to obtain the corresponding planning task priority, planning subtask chain, the planning task name, the planning task type, the task resource configuration list and the task input data; and performing a new workflow task record addition operation in the workflow task database locally preset in the database module; and setting the task name field of the newly added workflow task record to the planned task name, the task priority field to the planned task priority, the task type field to the planned task type, the task resource configuration list field to the task resource configuration list, the initial input data field to the task input data, the subtask chain field to the planned subtask chain, the task execution condition field to the unconditional state, the task status field to the unstarted state, and the task result field to empty; The workflow task record includes the task name field, the task priority field, the task type field, the task resource configuration list field, the initial input data field, the subtask chain field, the task execution condition field, the task status field and the task result field.

7. The task scheduling system according to claim 6, characterized in that: The workflow engine body is specifically used to, when editing and querying the workflow task, have the visualization module edit and process each field of the workflow task record in the workflow task database through the JAVA API interface unit or the CLI interface unit, and have the visualization module query and process each workflow task record in the workflow task database through the JAVA API interface unit or the CLI interface unit.

8. The task scheduling system according to claim 6, characterized in that: The workflow engine is specifically configured to, when decomposing, distributing and updating the status of the workflow tasks, query the workflow task records in the workflow task database whose task status field is in the unstarted state by the task extraction unit, group all the workflow task records obtained by the query according to the task priority field, select one workflow task record from the corresponding group whose task priority field is the highest priority as the current workflow task record, modify the task status field of the current workflow task record to the started state, push the current workflow task record to the task coordination unit, and after the push is completed, wait for the return data of the task coordination unit, and when the current task result data sent back by the task coordination unit is received, set the task result field of the current workflow task record in the workflow task database to the current task result data, and modify the task status field to the completed state; The task coordination unit identifies the task execution condition field of the current workflow task record pushed by the task extraction unit, and pushes the current workflow task record to the task execution unit if the task execution condition field is in an unconditional state; if the task execution condition field is not in an unconditional state, pushes the current workflow task record to the task execution unit when the execution condition set in the task execution condition field is satisfied, and waits for receiving the returned data from the task execution unit after pushing, and immediately sends the returned data to the task extraction unit when receiving the current task result data sent back by the task execution unit; The task execution unit extracts the task type field as the current task type, the task resource configuration list field as the current resource configuration list, the initial input data field as the current input data, and the subtask chain field as the current subtask name sequence from the current workflow task record pushed by the task coordination unit, and extracts the first subtask name in the current subtask name sequence as the current subtask name, and the current task type, the current resource configuration list, the current input data and the current subtask name form the corresponding current subtask data packet, and sets the task execution time parameter of the timer module and pushes the current subtask data packet to the timer module, and the timer The subtask execution result data returned by the timer module is used as the new current input data, and the next subtask name in the current subtask name sequence is extracted as the new current subtask name, and a new current subtask data packet is composed of the current task type, the current resource configuration list, the new current input data and the new current subtask name, and the task execution time parameter of the timer module is reset and the new current subtask data packet is pushed to the timer module again, and so on, until the timer module returns the subtask execution result data corresponding to the last subtask name, and sends the last subtask execution result data as the corresponding current task result data to the task extraction unit; The timer module identifies the current task type in the current subtask data packet pushed by the task execution unit. If the current task type is a real-time task type, the current subtask data packet is pushed to the real-time task client at a time point that meets the task execution time parameter. If the current task type is an offline task type, the current subtask data packet is pushed to the offline task client at a time point that meets the task execution time parameter. After pushing, the module waits for receiving the returned data from the real-time task client or the offline task client, and immediately sends the subtask execution result data returned by the real-time task client or the offline task client back to the task execution unit when receiving it.

9. The task scheduling system according to claim 1, characterized in that: The real-time task processing engine is specifically configured to extract a current resource configuration list from the current subtask data packet when processing a facility allocation application to the resource manager according to the current subtask data packet, and send an infrastructure application instruction carrying the current resource configuration list to the resource manager to obtain the corresponding allocated facility resources in the infrastructure cluster; The real-time task processing engine is specifically configured to extract current input data and a current subtask name from the current subtask data packet when performing real-time task computing processing according to the current subtask data packet with the aid of the allocated facility resources, record a local computing interface that matches the current subtask name as the current computing interface, use the allocated facility resources as computing resources for the current computing interface, send the current input data to the current computing interface, and use the return data from the current computing interface as the corresponding subtask execution result data; The offline task processing engine is specifically configured to extract the current subtask name from the current subtask data packet when performing offline task start time query processing based on the current subtask data packet, and extract the start period field of the time record whose task name field in the locally preset offline task start schedule matches the current subtask name as the corresponding offline task start period; wherein the offline task start schedule includes a plurality of time records; and each time record includes the task name field and the start period field; The offline task processing engine is specifically configured to, when performing facility allocation application processing on the resource manager based on the offline task startup period and the current subtask data packet, extract a current resource configuration list from the current subtask data packet at a time point when the offline task startup period is satisfied, and send an infrastructure application instruction carrying the current resource configuration list to the resource manager, thereby obtaining the corresponding allocated facility resources in the infrastructure cluster; The offline task processing engine is specifically used to extract the current input data and the current subtask name from the current subtask data packet when performing offline task calculation processing based on the current subtask data packet with the help of the allocated facility resources, and record the local computing interface that matches the current subtask name as the current computing interface, and use the allocated facility resources as the computing resources of the current computing interface, and send the current input data to the current computing interface, and use the return data of the current computing interface as the corresponding subtask execution result data.

Images