Intelligent large-scale task scheduling method and system based on computing power host business characteristics

Abstract

The invention discloses an intelligent large-scale task scheduling method and system based on computing power host business characteristics, and belongs to the technical field of cloud computing and edge computing, and the method comprises the following steps: data acquisition and preprocessing of tasks and computing power host nodes; performing multi-dimensional quantitative analysis on the task business characteristics; dynamically monitoring the performance state of the computing power host node; constructing and optimizing a dynamic computing power matching model; task scheduling is optimized in an auxiliary mode based on an intelligent prediction mechanism; and finally distributing and executing the task. The problems that when an existing computing power host processes large-scale concurrent tasks, due to the fact that a scheduling strategy lacks accurate recognition and adaptation on different service characteristics, resources are wasted, tasks are stacked, and response is delayed are solved. The utilization rate of computing power resources can be improved, task response delay is reduced, and user experience and system overall performance are improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing and edge computing technology, and specifically to an intelligent large-scale task scheduling method and system based on the business characteristics of a computing power host. Background Art

[0002] With the rapid development of cloud computing, big data, and artificial intelligence technologies, the efficiency of computing hosts in processing large-scale concurrent tasks has become increasingly prominent. However, existing task scheduling strategies usually adopt static or single optimization objectives when facing multiple business characteristics, which makes it difficult to meet the diverse resource requirements of different tasks. This lack of adaptability often leads to problems such as waste of computing resources, task accumulation, and response delays, which seriously restricts system performance and user experience. Summary of the invention

[0003] The technical task of the present invention is to address the above shortcomings and provide an intelligent large-scale task scheduling method and system based on the business characteristics of the computing power host, which can improve the utilization rate of computing power resources, reduce task response delay, and improve user experience and overall system performance.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] An intelligent large-scale task scheduling method based on the business characteristics of the computing power host is implemented by the following steps:

[0006] 1) Data collection and preprocessing of tasks and computing host nodes;

[0007] 2) Conduct multi-dimensional quantitative analysis of mission business characteristics;

[0008] 3) Dynamic monitoring of the performance status of computing host nodes;

[0009] 4) Construction and optimization of dynamic computing power matching model;

[0010] 5) Assist in optimizing task scheduling based on intelligent prediction mechanism;

[0011] 6) Final task allocation and execution.

[0012] By analyzing business characteristics in multiple dimensions and combining the performance status of computing host nodes, a dynamic computing power matching model is constructed to achieve accuracy and adaptability in task scheduling. By introducing multi-objective optimization algorithms, dynamic weight allocation strategies, and intelligent prediction mechanisms, the utilization rate of computing power resources is improved, task response delays are reduced, and user experience and overall system performance are improved.

[0013] Furthermore, the data collection and preprocessing includes:

[0014] 1.1) Task data collection: The scheduling system receives and stores key task information in real time, including:

[0015] Task deadline: Determine the time urgency of the task and provide a basis for quantifying real-time requirements;

[0016] Historical concurrent task volume: extract concurrent execution records of tasks from the database for concurrent demand prediction;

[0017] Computational complexity: Analyze the resource call frequency and algorithm complexity of the task to determine the computational intensity of different tasks;

[0018] Service Level Agreement (SLA): Define the task priority according to the agreement, for example, VIP service tasks have a higher weight;

[0019] 1.2) Data collection of computing host nodes: Through the distributed monitoring system, the performance data of each computing host node is collected in real time, including:

[0020] CPU usage: indicates the usage of computing resources and reflects the node load capacity;

[0021] Memory utilization: displays the allocation status of node memory and evaluates the adaptability of task allocation;

[0022] Network latency: describes the communication time between nodes and affects the scheduling of cross-node tasks;

[0023] Energy consumption: monitor the power consumption of nodes during operation to optimize energy efficiency;

[0024] 1.3) Data preprocessing:

[0025] In order to make the task characteristics and node performance directly used for model calculation, normalization is required. All indicators are uniformly mapped to the [0,1] interval using the following formula:

[0026]

[0027] The normalized data avoids the impact of dimensional differences on subsequent models and provides consistent input for scheduling optimization.

[0028] Furthermore, the business characteristics include real-time requirements, concurrency requirements, computational complexity and service level agreements;

[0029] Conduct multi-dimensional quantitative analysis of task business characteristics to clarify their scheduling requirements:

[0030] 2.1) Quantification of real-time requirements:

[0031] In order to measure the time urgency of the task and guide the priority allocation of resources, scheduling is achieved by calculating the ratio of the current time to the task deadline. The formula is as follows:

[0032]

[0033] If RTD approaches 1, it means that the task priority is higher and the task needs to be scheduled immediately;

[0034] If the RTD is close to 0, it means that the task is not urgent and is suitable for later scheduling;

[0035] 2.2) Concurrent demand forecasting:

[0036] To perceive the possible concurrency pressure of tasks in advance and optimize resource allocation strategies, we can predict future concurrency requirements by combining the historical maximum concurrency and average value:

[0037] CD=α·H max +(1-α)·E mean

[0038] Where, α is a user-adjustable factor used to adjust the importance of historical maximum concurrency and average concurrency;

[0039] High-concurrency tasks allocate more nodes to meet demand;

[0040] 2.3) Computational complexity evaluation:

[0041] In order to obtain the level of computing power occupied by quantitative tasks, the overall complexity is calculated by analyzing the complexity of the subtasks after task decomposition:

[0042]

[0043] Among them, C i : Complexity index of the task in part i (e.g., computational complexity, I / O operations, etc.);

[0044] W i : Weight, indicating the importance of the subtask;

[0045] 2.4) Service Level Agreement Priority Calculation:

[0046] Weights are directly assigned based on user needs and service terms (such as high-priority tasks requiring quick response).

[0047] Furthermore, the dynamic monitoring of the performance status of the computing host node includes:

[0048] 3.1) Real-time monitoring indicators:

[0049] By deploying monitoring probes on each computing host node, we continuously collect indicators including CPU occupancy and memory usage, forming a node performance database that is dynamically updated at a high frequency.

[0050] 3.2) Standardization and weight distribution:

[0051] Normalize the node performance indicators according to the formula to make them comparable;

[0052] Dynamic weights are assigned to different metrics (for example, CPU usage may be more important than memory usage), and the weight values ​​can be adjusted based on the actual load.

[0053] Furthermore, the dynamic computing power matching model optimizes the task allocation rules based on a multi-objective optimization algorithm and a dynamic weight allocation strategy;

[0054] The construction and optimization of the dynamic computing power matching model specifically include:

[0055] 4.1) Objective function definition:

[0056] The dynamic computing power matching model optimizes resource utilization efficiency and response time by combining the following two goals:

[0057] Minimize resource waste: Ensure that the allocated resources are close to the actual demand and avoid idleness;

[0058]

[0059] Minimize response time: Make task completion time close to user expectations;

[0060]

[0061] 4.2) Dynamic Weight Strategy:

[0062] According to the real-time system load, the weights of the two major goals are dynamically adjusted. The formula is as follows:

[0063] W final =λ 1 ·W RW +λ 2 ·W RD

[0064] Among them, λ 1 , 2 : Weight factor, which can be automatically adjusted by threshold conditions;

[0065] 4.3) Genetic Algorithm Optimization:

[0066] Initial population generation: Each population represents the allocation scheme of tasks and nodes;

[0067] Fitness calculation: Calculate the fitness of each solution based on the objective function;

[0068] Crossover and mutation: Introduce mutation factors into the population to improve the diversity of solutions and avoid falling into local optimality;

[0069] Iterative optimization: The population is updated repeatedly until the objective function converges.

[0070] Furthermore, the intelligent prediction mechanism uses a time series model to analyze historical task data and perceive load change trends in advance; the task scheduling optimization includes resource preheating and task migration strategies;

[0071] The specific implementation of assisted optimization of task scheduling based on intelligent prediction mechanism includes:

[0072] 5.1) Build a load prediction model to predict task load changes through time series. The formula is:

[0073] R future =β·R current +(1-β)·R historical

[0074] Where, β: weight factor, used to calculate the current value R current and historical value R historical The weighted average is performed between the two, and the impact of current trends and historical data on future forecasts is balanced by adjusting the value of β;

[0075] When β is close to 1, the predicted value R future More dependent on the current value R current , that is, believing that current trends have a greater impact on the future;

[0076] When β is close to 0, the predicted value R future More dependent on historical values ​​R historical , that is, it is believed that historical data has a greater impact on the future;

[0077] When β is equal to 0.5, the predicted value R future It is a simple average of the current value and the historical value, that is, the impact of both is equal.

[0078] Optimize resource allocation strategies in advance by predicting future high-load periods;

[0079] 5.2) Resource preheating and task migration:

[0080] Allocate resources in advance to reduce startup delays for predicted high-priority task loads;

[0081] Dynamically migrate non-critical tasks to low-load nodes to reduce resource occupation conflicts.

[0082] Furthermore, the task allocation and execution includes:

[0083] Prioritization: prioritize tasks based on real-time requirements (RTD), concurrency requirements (CD) and SLA;

[0084] Allocation and adjustment: Execute resource allocation according to the output of the dynamic computing power matching model and continuously monitor the results;

[0085] Adaptive adjustment: Recalculate task priorities and resource allocation strategies based on changes in system load.

[0086] The present invention also claims protection for an intelligent large-scale task scheduling system based on the business characteristics of a computing host, including:

[0087] Data collection and preprocessing module, used to realize data collection and preprocessing of tasks and computing host nodes;

[0088] Mission business characteristics quantitative analysis module, used to conduct multi-dimensional quantitative analysis of mission business characteristics;

[0089] The computing power host node performance status monitoring module is used for dynamic monitoring of the performance status of the computing power host node;

[0090] Dynamic computing power matching model construction and optimization module, used to realize the construction and optimization of dynamic computing power matching model;

[0091] Intelligent prediction mechanism assisted optimization module, used to assist in optimizing task scheduling based on intelligent prediction mechanism;

[0092] Task allocation and execution module, used to realize final task allocation and execution;

[0093] The system can implement the above method.

[0094] The present invention also claims protection for an intelligent large-scale task scheduling implementation device based on the business characteristics of a computing power host, comprising: at least one memory and at least one processor;

[0095] The at least one memory is used to store a machine-readable program;

[0096] The at least one processor is used to call the machine-readable program to implement the above method.

[0097] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which when executed by a processor can implement the above method.

[0098] Compared with the prior art, the intelligent large-scale task scheduling method and system based on the business characteristics of the computing host of the present invention has the following beneficial effects:

[0099] 1. Improve scheduling accuracy:

[0100] Through multi-dimensional quantitative analysis of the business characteristics of tasks (such as real-time performance, concurrency requirements, computational complexity, and service level agreements) and the performance status of nodes (such as CPU occupancy, memory utilization, network latency, and energy consumption), we achieve precise matching of tasks with computing host resources, effectively reducing resource waste and task backlogs.

[0101] 2. Enhance system adaptability:

[0102] The introduction of a dynamic computing power matching model and a multi-objective optimization algorithm, combined with real-time load and business priority to adjust task allocation rules, enables the system to flexibly respond to complex task requirements and load changes, and improves the robustness and reliability of the system.

[0103] 3. Reduce response delay:

[0104] The intelligent prediction mechanism analyzes historical task data, perceives load change trends in advance, and performs resource preheating and task migration optimization, which significantly shortens task response time and improves user experience.

[0105] 4. Optimize resource utilization:

[0106] The dynamic weight allocation strategy balances resource utilization and task execution efficiency in multi-objective optimization, maximizes the utilization efficiency of computing host nodes, and is particularly suitable for large-scale concurrent task scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] Figure 1 It is a flowchart of an intelligent large-scale task scheduling method based on the business characteristics of a computing power host provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0108] The present invention will be further described below in conjunction with specific embodiments.

[0109] The embodiment of the present invention provides an intelligent large-scale task scheduling method based on the business characteristics of the computing power host, including: multi-dimensional quantitative analysis of the business characteristics; dynamic monitoring of the performance status of the computing power host nodes; construction of a dynamic computing power matching model; and optimization of task scheduling based on an intelligent prediction mechanism. The business characteristics include real-time requirements, concurrency requirements, computational complexity, and service level agreements. The dynamic computing power matching model optimizes the task allocation rules based on a multi-objective optimization algorithm and a dynamic weight allocation strategy. The intelligent prediction mechanism uses a time series model to analyze historical task data and perceive load change trends in advance. Task scheduling optimization includes resource preheating and task migration strategies. The implementation of this method includes the following steps:

[0110] S1, task and computing host node data collection;

[0111] S2, quantitative analysis of mission business characteristics;

[0112] S3, performance status monitoring of computing host nodes;

[0113] S4. Construction and optimization of dynamic computing power matching model;

[0114] S5, intelligent prediction mechanism assisted optimization;

[0115] S6. Final task allocation and execution.

[0116] The above implementation steps are described in detail below.

[0117] Step one: data collection and preprocessing.

[0118] 1. Task data collection,

[0119] The scheduling system receives and stores key information of tasks in real time:

[0120] Task deadline: Determine the time urgency of the task and provide a basis for quantifying real-time requirements;

[0121] Historical concurrent task volume: extract concurrent execution records of tasks from the database for concurrent demand prediction;

[0122] Computational complexity: Analyze the resource call frequency and algorithm complexity of the task to determine the computational intensity of different tasks;

[0123] Service Level Agreement (SLA): Define task priorities according to the agreement, for example, VIP service tasks have a higher weight.

[0124] 2. Data collection of computing host nodes,

[0125] Through the distributed monitoring system, the performance data of each computing host node is collected in real time, including:

[0126] CPU usage: indicates the usage of computing resources and reflects the node load capacity;

[0127] Memory utilization: displays the allocation status of node memory and evaluates the adaptability of task allocation;

[0128] Network latency: describes the communication time between nodes and affects the scheduling of cross-node tasks;

[0129] Energy consumption: Monitor the power consumption of nodes during operation to optimize energy efficiency.

[0130] 3. Data preprocessing,

[0131] In order to make the task characteristics and node performance directly used for model calculation, normalization is required. All indicators are uniformly mapped to the [0,1] interval using the following formula:

[0132]

[0133] The normalized data avoids the impact of dimensional differences on subsequent models and provides consistent input for scheduling optimization.

[0134] Step 2: Quantitative analysis of mission business characteristics.

[0135] Conduct multi-dimensional quantitative analysis of task business characteristics to clarify their scheduling requirements:

[0136] 1. Quantify real-time requirements,

[0137] In order to measure the time urgency of the task and guide the priority allocation of resources, scheduling is achieved by calculating the ratio of the current time to the task deadline. The formula is as follows:

[0138]

[0139] If RTD approaches 1, it means that the task priority is higher and the task needs to be scheduled immediately;

[0140] If the RTD is close to 0, it means that the task is not urgent and is suitable for scheduling later.

[0141] 2. Concurrent demand forecasting,

[0142] In order to perceive the possible concurrency pressure of tasks in advance and optimize resource allocation strategies, the future concurrency requirements are predicted by combining the historical maximum concurrency and the average value:

[0143] CD=α·H max +(1-α)·E mean

[0144] Among them, α is a user-adjustable factor used to adjust the importance of historical maximum concurrency and average concurrency.

[0145] High-concurrency tasks allocate more nodes to meet demand.

[0146] 3. Computational complexity evaluation,

[0147] In order to obtain the level of computing power occupied by quantitative tasks, the overall complexity is calculated by analyzing the complexity of the subtasks after task decomposition:

[0148]

[0149] Among them, C i : Complexity index of the task in part i (e.g., computational complexity, I / O operations, etc.);

[0150] W i : Weight, indicating the importance of the subtask.

[0151] 4. Service level agreement priority calculation,

[0152] Weights are directly assigned based on user needs and service terms (such as high-priority tasks requiring quick response).

[0153] Step 3: Monitoring the performance status of computing host nodes.

[0154] 1. Real-time monitoring indicators,

[0155] By deploying monitoring probes on each computing host node, we continuously collect indicators including CPU occupancy and memory usage to form a node performance database that is dynamically updated at a high frequency.

[0156] 2. Standardization and weight distribution,

[0157] The node performance indicators are normalized according to the formula to make them comparable.

[0158] Dynamic weights are assigned to different metrics (for example, CPU usage may be more important than memory usage), and the weight values ​​can be adjusted based on the actual load.

[0159] Step 4: Construction and optimization of dynamic computing power matching model.

[0160] 1. Objective function definition:

[0161] The dynamic computing power matching model aims to optimize resource utilization efficiency and response time, combining the following two goals:

[0162] Minimize resource waste: Ensure that the allocated resources are close to the actual demand and avoid idleness;

[0163]

[0164] Minimize response time: Make task completion time close to user expectations;

[0165]

[0166] 2. Dynamic weight strategy:

[0167] According to the real-time system load, the weights of the two major goals are dynamically adjusted. The formula is as follows:

[0168] W final =λ 1 ·W RW +λ 2 ·W RD

[0169] Among them, λ 1 , 2 : Weight factor, which can be automatically adjusted by threshold conditions.

[0170] 3. Genetic algorithm optimization:

[0171] Initial population generation: Each population represents the allocation scheme of tasks and nodes;

[0172] Fitness calculation: Calculate the fitness of each solution based on the objective function;

[0173] Crossover and mutation: Introduce mutation factors into the population to improve the diversity of solutions and avoid falling into local optimality;

[0174] Iterative optimization: The population is updated repeatedly until the objective function converges.

[0175] Step 5: Intelligent prediction mechanism assists optimization.

[0176] 1. Build a load prediction model.

[0177] The task load change is predicted by time series, and the formula is:

[0178] R future =β·R current +(1-β)·R historical

[0179] Where β is a weight factor used to current and historical value R historical By taking a weighted average between the two, the impact of current trends and historical data on future forecasts can be balanced by adjusting the value of β; the details are as follows:

[0180] When β is close to 1, the predicted value R future More dependent on the current value R current , that is, believing that current trends have a greater impact on the future;

[0181] When β is close to 0, the predicted value R future More dependent on historical values ​​R historical , that is, it is believed that historical data has a greater impact on the future;

[0182] When β is equal to 0.5, the predicted value R future It is a simple average of the current value and the historical value, that is, the impact of both is equal.

[0183] By predicting future high-load periods, you can optimize resource allocation strategies in advance.

[0184] 2. Resource preheating and task migration,

[0185] Allocate resources in advance to reduce startup delays for predicted high-priority task loads;

[0186] Dynamically migrate non-critical tasks to low-load nodes to reduce resource occupation conflicts.

[0187] Step 6: Task allocation and execution.

[0188] Prioritization: prioritize tasks based on real-time requirements (RTD), concurrency requirements (CD) and SLA;

[0189] Allocation and adjustment: Execute resource allocation according to the output of the dynamic computing power matching model and continuously monitor the results;

[0190] Adaptive adjustment: Recalculate task priorities and resource allocation strategies based on changes in system load.

[0191] This method solves the problems of resource waste, task accumulation and response delay caused by the lack of accurate identification and adaptation of different business characteristics in the scheduling strategy of existing computing power hosts when processing large-scale concurrent tasks. Through multi-dimensional quantitative analysis of business characteristics and combining the performance status of computing power host nodes, a dynamic computing power matching model is constructed, and a multi-objective optimization algorithm and dynamic weight allocation strategy are used to intelligently adjust the task allocation rules. At the same time, an intelligent prediction mechanism is introduced to perceive the load change trend in advance, realize resource preheating and task migration optimization. This method improves the accuracy and adaptability of task scheduling, and has the technical advantages of high efficiency, flexibility and reliability.

[0192] The embodiment of the present invention also provides an intelligent large-scale task scheduling system based on the business characteristics of the computing power host, including:

[0193] Data collection and preprocessing module, used to realize data collection and preprocessing of tasks and computing host nodes;

[0194] Mission business characteristics quantitative analysis module, used to conduct multi-dimensional quantitative analysis of mission business characteristics;

[0195] The computing power host node performance status monitoring module is used for dynamic monitoring of the performance status of the computing power host node;

[0196] Dynamic computing power matching model construction and optimization module, used to realize the construction and optimization of dynamic computing power matching model;

[0197] Intelligent prediction mechanism assisted optimization module, used to assist in optimizing task scheduling based on intelligent prediction mechanism;

[0198] Task allocation and execution module, used to realize final task allocation and execution;

[0199] The system can implement the intelligent large-scale task scheduling method based on the business characteristics of the computing host described in the above embodiment. The specific implementation is as follows:

[0200] 1. Data acquisition and preprocessing module, including:

[0201] 1. Task data collection,

[0202] The scheduling system receives and stores key information of tasks in real time:

[0203] Task deadline: Determine the time urgency of the task and provide a basis for quantifying real-time requirements;

[0204] Historical concurrent task volume: extract concurrent execution records of tasks from the database for concurrent demand prediction;

[0205] Computational complexity: Analyze the resource call frequency and algorithm complexity of the task to determine the computational intensity of different tasks;

[0206] Service Level Agreement (SLA): Define task priorities according to the agreement, for example, VIP service tasks have a higher weight.

[0207] 2. Data collection of computing host nodes,

[0208] Through the distributed monitoring system, the performance data of each computing host node is collected in real time, including:

[0209] CPU usage: indicates the usage of computing resources and reflects the node load capacity;

[0210] Memory utilization: displays the allocation status of node memory and evaluates the adaptability of task allocation;

[0211] Network latency: describes the communication time between nodes and affects the scheduling of cross-node tasks;

[0212] Energy consumption: Monitor the power consumption of nodes during operation to optimize energy efficiency.

[0213] 3. Data preprocessing,

[0214] In order to make the task characteristics and node performance directly used for model calculation, normalization is required. All indicators are uniformly mapped to the [0,1] interval using the following formula:

[0215]

[0216] The normalized data avoids the impact of dimensional differences on subsequent models and provides consistent input for scheduling optimization.

[0217] 2. Quantitative analysis module of mission business characteristics,

[0218] Conduct multi-dimensional quantitative analysis of task business characteristics to clarify their scheduling requirements:

[0219] 1. Quantify real-time requirements,

[0220] In order to measure the time urgency of the task and guide the priority allocation of resources, scheduling is achieved by calculating the ratio of the current time to the task deadline. The formula is as follows:

[0221]

[0222] If RTD approaches 1, it means that the task priority is higher and the task needs to be scheduled immediately;

[0223] If the RTD is close to 0, it means that the task is not urgent and is suitable for scheduling later.

[0224] 2. Concurrent demand forecasting,

[0225] In order to perceive the possible concurrency pressure of tasks in advance and optimize resource allocation strategies, the future concurrency requirements are predicted by combining the historical maximum concurrency and the average value:

[0226] CD=α·H max +(1-α)·E mean

[0227] Among them, α is a user-adjustable factor used to adjust the importance of historical maximum concurrency and average concurrency.

[0228] High-concurrency tasks allocate more nodes to meet demand.

[0229] 3. Computational complexity evaluation,

[0230] In order to obtain the level of computing power occupied by quantitative tasks, the overall complexity is calculated by analyzing the complexity of the subtasks after task decomposition:

[0231]

[0232] Among them, C i : Complexity index of the task in part i (e.g., computational complexity, I / O operations, etc.);

[0233] W i : Weight, indicating the importance of the subtask.

[0234] 4. Service level agreement priority calculation,

[0235] Weights are directly assigned based on user needs and service terms (such as high-priority tasks requiring quick response).

[0236] 3. Computing host node performance status monitoring module, including:

[0237] 1. Real-time monitoring indicators,

[0238] By deploying monitoring probes on each computing host node, we continuously collect indicators including CPU occupancy and memory usage to form a node performance database that is dynamically updated at a high frequency.

[0239] 2. Standardization and weight distribution,

[0240] The node performance indicators are normalized according to the formula to make them comparable.

[0241] Dynamic weights are assigned to different metrics (for example, CPU usage may be more important than memory usage), and the weight values ​​can be adjusted based on the actual load.

[0242] 4. Dynamic computing power matching model construction and optimization module, including:

[0243] 1. Objective function definition:

[0244] The dynamic computing power matching model aims to optimize resource utilization efficiency and response time, combining the following two goals:

[0245] Minimize resource waste: Ensure that the allocated resources are close to the actual demand and avoid idleness;

[0246]

[0247] Minimize response time: Make task completion time close to user expectations;

[0248]

[0249] 2. Dynamic weight strategy:

[0250] According to the real-time system load, the weights of the two major goals are dynamically adjusted. The formula is as follows:

[0251] W final =λ 1 ·W RW +λ 2 ·W RD

[0252] Among them, λ 1 , 2 : Weight factor, which can be automatically adjusted by threshold conditions.

[0253] 3. Genetic algorithm optimization:

[0254] Initial population generation: Each population represents the allocation scheme of tasks and nodes;

[0255] Fitness calculation: Calculate the fitness of each solution based on the objective function;

[0256] Crossover and mutation: Introduce mutation factors into the population to improve the diversity of solutions and avoid falling into local optimality;

[0257] Iterative optimization: The population is updated repeatedly until the objective function converges.

[0258] 5. Intelligent prediction mechanism auxiliary optimization module, including:

[0259] 1. Build a load prediction model.

[0260] The task load change is predicted by time series, and the formula is:

[0261] R future =β·R current +(1-β)·R historical

[0262] Where β is a weight factor used to current and historical value R historical By taking a weighted average between the two, the impact of current trends and historical data on future forecasts can be balanced by adjusting the value of β; the details are as follows:

[0263] When β is close to 1, the predicted value R future More dependent on the current value R current , that is, believing that current trends have a greater impact on the future;

[0264] When β is close to 0, the predicted value R future More dependent on historical values ​​R historical , that is, it is believed that historical data has a greater impact on the future;

[0265] When β is equal to 0.5, the predicted value R future It is a simple average of the current value and the historical value, that is, the impact of both is equal.

[0266] By predicting future high-load periods, you can optimize resource allocation strategies in advance.

[0267] By predicting future high-load periods, you can optimize resource allocation strategies in advance.

[0268] 2. Resource preheating and task migration,

[0269] Allocate resources in advance to reduce startup delays for predicted high-priority task loads;

[0270] Dynamically migrate non-critical tasks to low-load nodes to reduce resource occupation conflicts.

[0271] 6. Task allocation and execution module, including:

[0272] Prioritization: prioritize tasks based on real-time requirements (RTD), concurrency requirements (CD) and SLA;

[0273] Allocation and adjustment: Execute resource allocation according to the output of the dynamic computing power matching model and continuously monitor the results;

[0274] Adaptive adjustment: Recalculate task priorities and resource allocation strategies based on changes in system load.

[0275] The embodiment of the present invention also provides an intelligent large-scale task scheduling implementation device based on the business characteristics of the computing power host, including: at least one memory and at least one processor;

[0276] The at least one memory is used to store a machine-readable program;

[0277] The at least one processor is used to call the machine-readable program to implement the intelligent large-scale task scheduling method based on the business characteristics of the computing power host described in the above embodiment.

[0278] The embodiment of the present invention also provides a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by the processor, the intelligent large-scale task scheduling method based on the business characteristics of the computing power host described in the above embodiment is implemented. Specifically, a system or device equipped with a storage medium can be provided, on which the software program code that implements the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0279] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0280] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0281] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0282] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0283] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. An intelligent large-scale task scheduling method based on the business characteristics of computing power hosts, characterized in that: The implementation of this method includes the following steps: 1) Data collection and preprocessing of tasks and computing host nodes; 2) Conduct multi-dimensional quantitative analysis of mission business characteristics; 3) Dynamic monitoring of the performance status of computing host nodes; 4) Construction and optimization of dynamic computing power matching model; 5) Assist in optimizing task scheduling based on intelligent prediction mechanism; 6) Final task allocation and execution.

2. The intelligent large-scale task scheduling method based on computing power host service characteristics according to claim 1 is characterized in that: The data collection and preprocessing include: 1.1) Task data collection: The scheduling system receives and stores key task information in real time, including: Task deadline: Determine the time urgency of the task and provide a basis for quantifying real-time requirements; Historical concurrent task volume: extract concurrent execution records of tasks from the database for concurrent demand prediction; Computational complexity: Analyze the resource call frequency and algorithm complexity of the task to determine the computational intensity of different tasks; Service Level Agreement: Define task priorities according to the agreement; 1.2) Data collection of computing host nodes: Through the distributed monitoring system, the performance data of each computing host node is collected in real time, including: CPU usage; Memory utilization; Network latency; Energy consumption; 1.3) Data preprocessing: Normalize and map all indicators to the [0,1] interval using the following formula:

3. The intelligent large-scale task scheduling method based on computing host service characteristics according to claim 1 is characterized in that: The business characteristics include real-time requirements, concurrency requirements, computational complexity and service level agreements; Conduct multi-dimensional quantitative analysis of task business characteristics to clarify their scheduling requirements: 2.1) Quantification of real-time requirements: Calculate the ratio of the current time to the task deadline. The formula is as follows: If RTD approaches 1, it means that the task priority is high and the task needs to be scheduled immediately; If the RTD is close to 0, it means that the task is not urgent and is suitable for later scheduling; 2.2) Concurrent demand forecasting: Combining the historical maximum concurrency and average value to predict future concurrency requirements: CD=α·H max +(1-a)·E mean Where, α is a user-adjustable factor used to adjust the importance of historical maximum concurrency and average concurrency; High-concurrency tasks allocate more nodes to meet demand; 2.3) Computational complexity evaluation: Analyze the complexity of subtasks after task decomposition and calculate the overall complexity: Among them, C i : Complexity index of the task in part i; W i : Weight, indicating the importance of the subtask; 2.4) Service Level Agreement Priority Calculation: Weights are directly assigned based on user needs and terms of service.

4. The intelligent large-scale task scheduling method based on computing host service characteristics according to claim 1 is characterized in that: The dynamic monitoring of the performance status of the computing host node includes: 3.1) Real-time monitoring indicators: By deploying monitoring probes on each computing host node, we continuously collect indicators including CPU occupancy and memory usage, forming a node performance database that is dynamically updated at a high frequency. 3.2) Standardization and weight distribution: Normalize the node performance indicators according to the formula to make them comparable; Dynamic weights are assigned to different indicators, and the weight values ​​can be adjusted according to the actual load.

5. The intelligent large-scale task scheduling method based on computing host service characteristics according to claim 1 is characterized in that: The dynamic computing power matching model optimizes the task allocation rules based on a multi-objective optimization algorithm and a dynamic weight allocation strategy; The construction and optimization of the dynamic computing power matching model specifically include: 4.1) Objective function definition: The dynamic computing power matching model optimizes resource utilization efficiency and response time by combining the following two goals: Minimize resource waste: Minimize response time: 4.2) Dynamic Weight Strategy: According to the real-time system load, the weights of the two major goals are dynamically adjusted. The formula is as follows: W final =λ1·W RW +λ2·W RD Among them, λ1, λ2: weight factors, which can be automatically adjusted by threshold conditions; 4.3) Genetic Algorithm Optimization: Initial population generation: Each population represents the allocation scheme of tasks and nodes; Fitness calculation: Calculate the fitness of each solution based on the objective function; Crossover and mutation: introducing mutation factors into the population; Iterative optimization: The population is updated repeatedly until the objective function converges.

6. The intelligent large-scale task scheduling method based on computing host service characteristics according to claim 1 is characterized in that: The intelligent prediction mechanism uses a time series model to analyze historical task data and perceive load change trends in advance; the task scheduling optimization includes resource preheating and task migration strategies; The specific implementation of assisted optimization of task scheduling based on intelligent prediction mechanism includes: 5.1) Build a load prediction model to predict task load changes through time series. The formula is: R future =β·R current +(1-β)·R historical Among them, β is a weight factor, which is used to perform a weighted average between the current value and the historical value. By adjusting the value of β, the impact of the current trend and historical data on future predictions can be balanced; Optimize resource allocation strategies in advance by predicting future high-load periods; 5.2) Resource preheating and task migration: Allocate resources in advance to reduce startup delays for predicted high-priority task loads; Dynamically migrate non-critical tasks to low-load nodes to reduce resource occupation conflicts.

7. The intelligent large-scale task scheduling method based on computing host service characteristics according to claim 1 is characterized in that: The task allocation and execution include: Prioritization: prioritize tasks based on real-time requirements, concurrency requirements, and SLA priorities; Allocation and adjustment: Execute resource allocation according to the output of the dynamic computing power matching model and continuously monitor the results; Adaptive adjustment: Recalculate task priorities and resource allocation strategies based on changes in system load.

8. An intelligent large-scale task scheduling system based on the business characteristics of computing power hosts, characterized by: include: Data collection and preprocessing module, used to realize data collection and preprocessing of tasks and computing host nodes; Mission business characteristics quantitative analysis module, used to conduct multi-dimensional quantitative analysis of mission business characteristics; The computing power host node performance status monitoring module is used for dynamic monitoring of the performance status of the computing power host node; Dynamic computing power matching model construction and optimization module, used to realize the construction and optimization of dynamic computing power matching model; Intelligent prediction mechanism assisted optimization module, used to assist in optimizing task scheduling based on intelligent prediction mechanism; Task allocation and execution module, used to realize final task allocation and execution; The system can implement the method described in any one of claims 1 to 7.

9. An intelligent large-scale task scheduling implementation device based on the business characteristics of the computing power host, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.

10. A computer readable medium, characterized in that The computer readable medium stores computer instructions, which, when executed by a processor, can implement the method according to any one of claims 1 to 7.

Images