Intelligent task scheduling method and device, equipment and medium
Through intelligent task scheduling methods, task information is collected regularly, dependencies are queried, and target tasks are selected based on intelligent algorithms, the problem of confusing task scheduling time in traditional scheduling systems is solved, and efficient resource utilization and accurate task matching are achieved.
Patent Information
- Application Number
- CN202411737484.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-03-21
AI Technical Summary
In the traditional data warehouse task scheduling system, the task scheduling time is chaotic, and the upstream tasks and downstream tasks cannot be well connected, resulting in insufficient resource utilization and increasing the task running time cost.
By collecting task information regularly, querying the dependencies in the scheduling system database, selecting the optimal target task for operation based on intelligent algorithms, and updating the task status in the scheduling system database.
It realizes accurate matching of task scheduling, improves resource utilization, reduces task running time costs, and adapts to dynamically changing dependencies.
Smart Images
Figure CN119668804A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent scheduling technology, and in particular to an intelligent task scheduling method, device, equipment and medium. Background Art
[0002] In the field of big data, traditional data warehouse task scheduling systems are based on time scheduling. Developers need to manually configure the task running time when launching tasks. This approach brings two disadvantages. First, tasks are directly dependent on each other. In order to avoid upstream tasks not being completed and downstream tasks already started, the scheduling time will be staggered and reserved, which invisibly increases the cost of task running time. Second, when the scheduling time is manually set, the cluster is overloaded at some time points and particularly idle at other time points, resulting in ineffective utilization of cluster resources. Summary of the invention
[0003] The present invention provides an intelligent task scheduling method, device, computer equipment and medium to solve the technical problems in the traditional data warehouse task scheduling system that the task scheduling time is chaotic and the upstream tasks and downstream tasks cannot be well matched and connected.
[0004] In a first aspect, an intelligent task scheduling method is provided, comprising:
[0005] Collect task information regularly and query the corresponding dependencies in the scheduling system database;
[0006] Obtain the dependencies corresponding to the task information, and search for the corresponding executable task list in the intelligent scheduling module based on the dependencies;
[0007] Based on intelligent algorithms, the optimal target task is selected for operation according to the CPU and memory margins;
[0008] Execute the target task and update the task status in the scheduling system database.
[0009] In a second aspect, an intelligent task scheduling device is provided, comprising:
[0010] The collection module is used to collect task information at regular intervals and query the corresponding dependency relationship of the task information in the scheduling system database;
[0011] The search module is used to obtain the dependency relationship corresponding to the task information, and search for the corresponding executable task list in the intelligent scheduling module according to the dependency relationship;
[0012] The computing module is used to select the optimal target task to run based on the CPU and memory margin based on intelligent algorithms;
[0013] The execution module is used to execute the target task and update the task status in the scheduling system database.
[0014] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned intelligent task scheduling method when executing the computer program.
[0015] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned intelligent task scheduling method are implemented.
[0016] In the scheme implemented by the above-mentioned intelligent task scheduling method, device, computer equipment and storage medium, task information can be collected at a regular interval and the corresponding dependency relationship of the task information can be queried in the scheduling system database; the dependency relationship corresponding to the task information can be obtained, and the corresponding executable task list can be searched in the intelligent scheduling module according to the dependency relationship; based on the intelligent algorithm, the optimal target task is selected for operation according to the CPU and memory margin; the target task is executed and the task status in the scheduling system database is updated.
[0017] In the task-based scheduling system database, each task is independent, and the execution order and priority of tasks can be adjusted dynamically as needed. The task-based scheduling system database is more suitable for handling dynamic changes and complex dependencies, and can flexibly handle new tasks and dependencies. The reasonable design and optimization of the scheduling system database can significantly improve the processing efficiency and quality of data. When selecting scheduling algorithms and formulating scheduling strategies, it is necessary to consider specific scenarios and needs to ensure the efficient operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0019] Figure 1 is a schematic diagram of an application environment of an intelligent task scheduling method in one embodiment of the present invention;
[0020] Figure 2 is a flow chart of an intelligent task scheduling method in one embodiment of the present invention;
[0021] Figure 3 is a structural diagram of an intelligent task scheduling device in one embodiment of the present invention;
[0022] Figure 4 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;
[0023] Figure 5 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0025] The intelligent task scheduling method provided by the embodiment of the present invention can be applied in Figure 1 In the application environment, the client collects task information at regular intervals, queries the corresponding dependencies of the task information in the scheduling system database; obtains the dependencies corresponding to the task information, and searches the corresponding executable task list in the intelligent scheduling module according to the dependencies; based on the intelligent algorithm, selects the optimal target task to run according to the CPU and memory margin; executes the target task and updates the task status in the scheduling system database. It is realized that in the task-based scheduling system database, each task is independent, and the execution order and priority of the task can be dynamically adjusted as needed. The task-based scheduling system database is more suitable for handling dynamic changes and complex dependencies, and can flexibly handle new tasks and dependencies. The data processing efficiency and quality can be significantly improved by reasonably designing and optimizing the scheduling system database. When selecting a scheduling algorithm and formulating a scheduling strategy, it is necessary to consider the specific scenarios and requirements to ensure the efficient operation of the system. The present invention is described in detail below through specific embodiments.
[0026] See also Figure 2 As shown, Figure 2 A flowchart of an intelligent task scheduling method provided by an embodiment of the present invention includes the following steps:
[0027] S10: Collect task information regularly and query the corresponding dependency relationship of the task information in the scheduling system database;
[0028] The dispatching system database is a database used to store, manage and analyze large amounts of data. With the growing amount of data, the size and complexity of the dispatching system database are also increasing. The dispatching system database is responsible for coordinating and optimizing the data processing flow to ensure the efficient operation of the dispatching system database. Scheduling refers to the process of allocating tasks, arranging processing order and optimizing resource utilization in the data warehouse. Scheduling mainly improves the efficiency and quality of data processing while ensuring the stability and reliability of the dispatching system database. The dispatching system database needs to comprehensively consider factors such as the dependencies between tasks, resource utilization, and data processing requirements in order to formulate a reasonable dispatching strategy.
[0029] Scheduling algorithm: Scheduling is performed in the order in which tasks enter, and the tasks that enter first are processed first. This is used in scenarios where there are dependencies between tasks; scheduling is performed in reverse order in the order in which tasks enter, and the tasks that enter later are processed first. This is used in scenarios where there are dependencies between tasks, but it may cause some tasks to be blocked for a long time; scheduling is performed according to the processing time of the tasks, and the tasks with the shortest processing time are processed first, which helps to improve processing efficiency, but may cause uneven resource allocation. In this embodiment, no matter which scheduling algorithm is used, the subsequent downstream tasks will accurately match the task time to run. Collect task information regularly, and query the corresponding dependencies of the task information in the scheduling system database.
[0030] Scheduling methods: task allocation, resource allocation, and scheduling times. Task allocation: Assign tasks to appropriate processing resources to maximize system efficiency. Usually, tasks are matched based on the nature of the task and the capacity of the resource. For example, tasks that process large amounts of data can be assigned to resources with more processing power. Resource allocation: Rationally allocate system resources based on the nature and needs of the task. For example, for tasks that require a lot of memory, more memory resources can be allocated. Scheduling times: Determine a reasonable scheduling frequency to ensure that tasks are processed in a timely manner while avoiding the impact of over-scheduling on system performance.
[0031] In one embodiment: Taking a data warehouse of an online shopping platform as an example, the scheduling system needs to process a large amount of user purchase behavior data. In order to analyze the user's purchase behavior in real time, the data warehouse needs to process this data efficiently. In the selection of the scheduling algorithm, the SLAB algorithm can be considered, because the user purchase behavior data with a shorter processing time can be processed first, thereby improving the processing efficiency.
[0032] In terms of scheduling strategy, the following measures can be taken: Task allocation: Divide the purchase behavior data by user, and each user's data can be allocated to different processing resources to fully utilize the system's processing power. Resource allocation: For data that requires real-time analysis of user purchase behavior, more computing and storage resources can be allocated to ensure that the analysis task can be completed in a timely manner. Scheduling frequency: Scheduling can be set to be performed every certain period of time (such as 1 hour) to ensure that the data can be updated and analyzed in a timely manner.
[0033] The scheduling system database can significantly improve the data processing efficiency and quality through reasonable design and optimization. When selecting scheduling algorithms and formulating scheduling strategies, it is necessary to consider specific scenarios and requirements to ensure the efficient operation of the system.
[0034] S20: Obtain the dependency relationship corresponding to the task information, and search for the corresponding executable task list in the intelligent scheduling module according to the dependency relationship;
[0035] The intelligent scheduling module is located downstream of the scheduling system database, and triggers the corresponding executable task list in the downstream intelligent scheduling module according to the dependency relationship corresponding to the task information; wherein, the dependency relationship between the intelligent scheduling module and the scheduling system database is initially defined, and the execution order of the tasks is calculated by the breadth-first or depth-first algorithm by defining the dependency relationship between the tasks, and then the tasks are automatically scheduled and executed. Among them, scheduling is based on a single task, each task is executed independently, and the execution order and priority of the tasks can be dynamically adjusted as needed. The advantage of this scheduling mode is that it can be flexibly extended to downstream tasks, and the scheduling strategy can be dynamically adjusted according to the execution status of the tasks to improve the scalability of the tasks. Moreover, the interdependence between the tasks forms a divergent execution from point to surface, which can be arbitrarily expanded and added. Of course, if the tasks in the scheduling system database change, the tasks in the intelligent scheduling module corresponding to the dependency relationship are triggered and updated. In the task-based scheduling system database, each task is independent, and the execution order and priority of the tasks can be dynamically adjusted as needed. The task-based scheduling system database is more suitable for handling dynamic changes and complex dependencies, and can flexibly handle new tasks and dependencies.
[0036] S30: Based on intelligent algorithms, the optimal target task is selected for execution according to the CPU and memory availability.
[0037] Specifically, based on the intelligent algorithm, the time range of the tasks in the executable task list is calculated according to the CPU and memory margins; the optimal target task is obtained for execution. The optimal target task is the target task whose end time of the real-time task is most consistent with the start time of the target task.
[0038] It should be noted that the intelligent algorithm includes at least one of the following: linear regression, logistic regression, decision tree, K nearest neighbor algorithm, random forest support vector machine, principal component analysis, reinforcement learning, recursive neural network, convolutional neural network, neural network.
[0039] Linear regression is used to establish a linear relationship between the independent variable (real-time task) and the dependent variable (target task). Its goal is to find the best fit line to minimize the error between the predicted value and the actual value. The error between the predicted value and the actual value is minimized by the least squares method to obtain the parameters of the linear regression equation. It is simple to understand, easy to implement and explain, and works well for data with linear relationships.
[0040] Logistic regression is used for binary classification problems. The Sigmoid function is used to map the input of the linear combination to between 0 and 1, and the output is the probability of the event. The maximum likelihood estimation is used to optimize the model parameters so that the predicted probability matches the actual label. It has high computational efficiency, is suitable for large-scale data, and outputs probabilities that are easy to understand.
[0041] Decision trees make decisions through a tree structure, and the path from the root node to the leaf node represents the classification rule. Use information gain or Gini index to select the best features for node splitting until the stopping condition is met. It is easy to understand and explain, and can handle classification and regression tasks.
[0042] Support vector machines work by finding the best hyperplane to separate data points of different categories, maximizing the interval between classes. Support vectors and hyperplanes are found using an optimization algorithm, usually implemented by the Lagrange multiplier method. They perform well on high-dimensional data and can handle nonlinear data using kernel functions.
[0043] The k-nearest neighbor algorithm is an instance-based learning method that performs classification or regression by finding the k nearest neighbors to the target point. There is no explicit training process, and prediction is mainly made by calculating distances. It is simple to understand, easy to implement, and insensitive to outliers.
[0044] Random forest is an ensemble learning method that improves the accuracy of classification or regression through the voting results of multiple decision trees. Multiple decision trees are constructed through random sampling and feature selection, and the results are finally obtained through voting or averaging. It has strong anti-overfitting ability and can handle high-dimensional data.
[0045] Neural networks simulate the structure of the human brain and consist of multiple neurons. They use activation functions to nonlinearly combine input features. They use backpropagation algorithms and gradient descent methods to optimize network parameters to minimize the loss function. They can handle complex nonlinear relationships and are suitable for large-scale data.
[0046] Convolutional neural networks are suitable for image processing. They extract features through convolutional layers and reduce dimensions through pooling layers. They use backpropagation algorithms to optimize the weights of convolutional kernels and fully connected layers. They perform well on image data, automatically extract features, and reduce manual intervention.
[0047] Recurrent neural networks process sequential data, have memory capabilities, and can capture dependencies in time series. They use the back propagation through time (BPTT) algorithm to update weights. They are suitable for processing sequential data and can capture time dependencies.
[0048] Reinforcement learning obtains feedback through interaction with the environment and learns how to take actions to maximize cumulative rewards. Use algorithms such as policy gradient or Q-learning to update the strategy so that the expected reward obtained by the action selected in the current state is maximized. It can handle complex decision-making problems and does not require a large amount of labeled data.
[0049] After the step of calculating the time range for the execution of tasks in the executable task list based on the CPU and memory margins based on the intelligent algorithm, the method further includes: arranging the priorities of the tasks according to preset conditions based on the result of calculating the time range for the execution of the tasks; and selecting the optimal target task based on the priority of the tasks. The preset conditions include the importance level of the task or the time margin for the task to be executed.
[0050] S40: Execute the target task and update the task status in the scheduling system database.
[0051] Execute the target task. The start time of the target task should be consistent with the end time of the real-time task. Determine a reasonable scheduling frequency to ensure timely processing of tasks, avoid the impact of excessive scheduling on system performance, and improve data processing efficiency and quality. While executing the target task, update the task status in the scheduling system database to facilitate the execution of the next real-time task.
[0052] Before the step of regularly collecting task information and querying the corresponding dependency relationship of the task information in the scheduling system database, the method also includes: collecting task group information, the task group information includes the CPU and memory usage of each queue in the cluster; storing the task group information in a time series database.
[0053] After obtaining the dependency corresponding to the task information and searching the corresponding executable task list in the intelligent scheduling module according to the dependency, the method also includes: obtaining the CPU and memory margin in each queue in the timing database; and transferring the CPU and memory margin in each queue into the intelligent scheduling module.
[0054] In this embodiment, the scheduled task collects task information, dependencies, task priority, the maximum running time of the task within 7 days, and the CPU and memory consumed by the task running. According to the latest time of the report on the task configuration leaf node and the coefficient of the maximum running time of the task within 7 days * 1 to 1.2 (the higher the task priority, the larger the coefficient, mainly to prevent the actual running time of the task from exceeding the historical maximum running time, causing task delays), the latest start time of each task is inferred. The time margin of all tasks will be refreshed every minute (current time - latest start time).
[0055] Business personnel only need to set the latest display time of the report, and the system will automatically calculate the latest start time of each dependent task. The intelligent scheduling device combines the CPU, memory, task time margin, task importance level, task CPU consumption, task memory consumption, and task running time of each queue in the cluster to optimally and timely schedule tasks, so as to make full use of cluster resources.
[0056] The present invention provides an intelligent task scheduling method, which first collects task information at a regular interval and queries the corresponding dependency relationship of the task information in a scheduling system database; obtains the dependency relationship corresponding to the task information, and searches for the corresponding executable task list in an intelligent scheduling module according to the dependency relationship; based on an intelligent algorithm, selects the optimal target task to run according to the CPU and memory margin; executes the target task and updates the task status in the scheduling system database.
[0057] In the task-based scheduling system database, each task is independent, and the execution order and priority of tasks can be adjusted dynamically as needed. The task-based scheduling system database is more suitable for handling dynamic changes and complex dependencies, and can flexibly handle new tasks and dependencies. The reasonable design and optimization of the scheduling system database can significantly improve the processing efficiency and quality of data. When selecting scheduling algorithms and formulating scheduling strategies, it is necessary to consider specific scenarios and needs to ensure the efficient operation of the system.
[0058] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0059] In one embodiment, an intelligent task scheduling device is provided, which corresponds one-to-one to the intelligent task scheduling method in the above embodiment. Figure 3 As shown, the intelligent task scheduling device includes a receiving module 101, a distribution module 102, an allocation module 103, and a calling module 104. The detailed description of each functional module is as follows:
[0060] The collection module 101 is used to collect task information at regular intervals and query the corresponding dependency relationship of the task information in the scheduling system database;
[0061] The search module 102 is used to obtain the dependency relationship corresponding to the task information, and search the corresponding executable task list in the intelligent scheduling module according to the dependency relationship;
[0062] The calculation module 103 is used to select the optimal target task to run based on the CPU and memory margin based on the intelligent algorithm;
[0063] The execution module 104 is used to execute the target task and update the task status in the scheduling system database.
[0064] In one embodiment, the device is further configured to:
[0065] A task group establishment module is used to collect task group information, including the CPU and memory usage of each queue in the cluster;
[0066] The storage module is used to store the task group information into the time series database.
[0067] In one embodiment, the device is further configured to:
[0068] The acquisition module is used to obtain the CPU and memory remaining in each queue in the time series database;
[0069] The import module is used to import the CPU and memory margins in each queue into the intelligent scheduling module.
[0070] In one embodiment, the acquisition module 101 is specifically used for:
[0071] The intelligent calculation submodule is used to calculate the time range of task execution in the executable task list based on the CPU and memory margin based on the intelligent algorithm;
[0072] The acquisition submodule is used to obtain the optimal target task for running.
[0073] In one embodiment, the device is further configured to:
[0074] A calculation module is used to arrange the priorities of tasks according to preset conditions based on the time range results of the calculation task operation;
[0075] The optimal selection module is used to select the optimal target task based on the priority of the task.
[0076] The present invention provides an intelligent task scheduling device, which first collects task information at regular intervals, queries the corresponding dependency relationship of the task information in the scheduling system database; obtains the dependency relationship corresponding to the task information, and searches for the corresponding executable task list in the intelligent scheduling module according to the dependency relationship; based on the intelligent algorithm, selects the optimal target task to run according to the CPU and memory margin; executes the target task and updates the task status in the scheduling system database.
[0077] In the task-based scheduling system database, each task is independent, and the execution order and priority of tasks can be adjusted dynamically as needed. The task-based scheduling system database is more suitable for handling dynamic changes and complex dependencies, and can flexibly handle new tasks and dependencies. The reasonable design and optimization of the scheduling system database can significantly improve the processing efficiency and quality of data. When selecting scheduling algorithms and formulating scheduling strategies, it is necessary to consider specific scenarios and needs to ensure the efficient operation of the system.
[0078] For the specific definition of the intelligent task scheduling device, please refer to the definition of the intelligent task scheduling method above, which will not be repeated here. Each module in the above-mentioned intelligent task scheduling device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0079] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of an intelligent task scheduling method.
[0080] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps of the client side of an intelligent task scheduling method
[0081] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
[0082] Collect task information regularly and query the corresponding dependencies in the scheduling system database;
[0083] Obtain the dependencies corresponding to the task information, and search for the corresponding executable task list in the intelligent scheduling module based on the dependencies;
[0084] Based on intelligent algorithms, the optimal target task is selected for operation according to the CPU and memory margins;
[0085] Execute the target task and update the task status in the scheduling system database.
[0086] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.
[0087] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0088] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0089] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. An intelligent task scheduling method, characterized in that: include: Collect task information regularly and query the corresponding dependencies in the scheduling system database; Obtain the dependencies corresponding to the task information, and search for the corresponding executable task list in the intelligent scheduling module based on the dependencies; Based on intelligent algorithms, the optimal target task is selected for operation according to the CPU and memory margins; Execute the target task and update the task status in the scheduling system database.
2. The intelligent task scheduling method according to claim 1, characterized in that: Before the step of collecting task information at regular intervals and searching the scheduling system database for corresponding dependencies of the task information, the method further includes: Collect task group information, including CPU usage and memory usage of each queue in the cluster; The task group information is stored in the time series database.
3. The intelligent task scheduling method according to claim 2, characterized in that: After the step of obtaining the dependency relationship corresponding to the task information and searching the corresponding executable task list in the intelligent scheduling module according to the dependency relationship, the method further includes: Get the CPU and memory usage of each queue in the time series database; Transfer the CPU and memory margins in each queue into the intelligent scheduling module.
4. The intelligent task scheduling method according to claim 1, characterized in that: The step of selecting the optimal target task to run based on the CPU and memory margin based on the intelligent algorithm includes: Based on intelligent algorithms, the time range for tasks in the executable task list is calculated according to the CPU and memory remaining; Get the optimal target task to run.
5. The intelligent task scheduling method according to claim 4, characterized in that: The optimal target task is a target task whose end time of the real-time task is most consistent with the start time of the target task.
6. The intelligent task scheduling method according to claim 5, characterized in that: After the step of calculating the time range for executing tasks in the executable task list based on the CPU and memory margins based on the intelligent algorithm, the method further includes: Arrange the priority of tasks according to preset conditions based on the time range results of calculating task execution; Select the best target task based on task priority.
7. The intelligent task scheduling method according to claim 6, characterized in that: The preset conditions include the importance level of the task or the time margin for the task to be executed.
8. An intelligent task scheduling device, characterized in that: include: The collection module is used to collect task information at regular intervals and query the corresponding dependency relationship of the task information in the scheduling system database; The search module is used to obtain the dependency relationship corresponding to the task information, and search for the corresponding executable task list in the intelligent scheduling module according to the dependency relationship; The computing module is used to select the optimal target task to run based on the CPU and memory margin based on intelligent algorithms; The execution module is used to execute the target task and update the task status in the scheduling system database.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the intelligent task scheduling method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the intelligent task scheduling method according to any one of claims 1 to 7 are implemented.