Task scheduling method, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a task scheduling method, electronic equipment, a storage medium and a program product. The method comprises the following steps: in response to an execution instruction for obtaining a target task, obtaining a demand parameter of the target task, a partition subset of a server cluster of a to-be-executed target task and a performance parameter of each server in the server cluster; determining an initial parameter of an ant colony algorithm according to the demand parameter and the performance parameter of each server; executing an ant colony algorithm to determine a target server in the partition subset, and sending a task execution message to the target server; wherein the target server is a server selected by ants in the ant colony algorithm; the task execution message is used for indicating the target server to execute the target task; according to the target server, the pheromone concentration in the ant colony algorithm is updated, and the ant colony algorithm is iteratively executed until all the servers in the server cluster execute the target task. The method is used for achieving the effect of improving the resource utilization rate after task scheduling.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task scheduling method, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of information technology and the widespread application of technologies such as cloud computing, big data, artificial intelligence, and virtualization, the resource utilization efficiency of server systems supporting these technologies has become a critical issue. These server systems contain a large number of servers, making manual management and operation extremely difficult. Therefore, automated operation and maintenance are becoming increasingly important in enterprises and organizations.

[0003] One application scenario for automated operations and maintenance is as follows: For large-scale architectures such as large distributed systems containing multiple servers, when all servers need to execute the same task, but it is not possible to configure all servers to execute it simultaneously, it is necessary to determine the task execution order of each server based on factors such as the current working status of the servers and network conditions. Mathematically, this can be approximated as a path selection problem.

[0004] For path selection problems, the ant colony algorithm is a heuristic algorithm that simulates the foraging behavior of ants to perform a distributed and parallel search of the problem space, quickly finding a near-optimal solution. However, the standard ant colony algorithm is prone to getting trapped in local optima and struggles to handle dynamic environments. If directly applied to the aforementioned automated operation and maintenance tasks, it will result in an unreasonable task execution order and low cluster resource utilization efficiency. Summary of the Invention

[0005] This application provides a task scheduling method, electronic device, storage medium, and program product to achieve the effect of reasonably determining the order in which servers execute tasks and improving resource utilization after task scheduling.

[0006] In a first aspect, embodiments of this application provide a task scheduling method, including:

[0007] In response to receiving the execution instruction of the target task, the system obtains the requirement parameters of the target task, as well as the partition subset of the server cluster to be executed and the performance parameters of each server in the server cluster; wherein, the server cluster includes at least one partition subset, and each partition subset contains at least one server.

[0008] Based on the requirements and the performance parameters of each server, determine the initial parameters of the ant colony algorithm;

[0009] The ant colony algorithm is executed to determine the target server in the partition subset, and a task execution message is sent to the target server; wherein, the target server is the server selected by the ants in the ant colony algorithm; the task execution message is used to instruct the target server to execute the target task;

[0010] Based on the target server, update the pheromone concentration in the ant colony algorithm and iteratively execute the ant colony algorithm until all servers in the server cluster have completed the target task.

[0011] In one possible implementation, the ant colony algorithm is executed to determine the target server within a subset of partitions, including:

[0012] Each partition subset is treated as an execution node, and the node performance index of the execution node is calculated based on the performance parameters of each server in the partition subset.

[0013] Based on the demand parameters and node performance parameters, the probability of each ant choosing each execution node is calculated to obtain the first selection probability;

[0014] For each ant, based on preset selection conditions and according to the first selection probability, the target execution node selected by each ant is determined;

[0015] In the target partition subset corresponding to the target execution node, the server ranked first in the target partition subset is selected according to the preset partition server sorting conditions, and the target server selected by the ant is obtained.

[0016] In one possible implementation, the partition subset includes several region subsets, and each region subset includes several servers;

[0017] Within the target partition subset corresponding to the target execution node, the server ranked first in the target partition subset is selected according to the preset partition server sorting criteria. This yields the target servers selected by the ant colony, including:

[0018] Based on the requirement parameters, the heuristic information index corresponding to the target task is obtained;

[0019] For the target partition subset, based on the pheromone concentration of ants in the region subset of the target partition subset and the heuristic information index, the probability of the ant choosing each region subset is calculated to obtain the second selection probability;

[0020] Determine the subset of regions with the highest probability of the second choice as the target region subset;

[0021] Within the target region subset, the server ranked first in the target region subset is selected according to the preset region subset server sorting conditions, thus obtaining the target server selected by the ant.

[0022] In one possible implementation, the server ranked first in the target region subset is selected according to a preset region subset server sorting condition to obtain the target server selected by the ant, including:

[0023] Based on the server's current performance parameters, the fitness value of the server for the target task is obtained using the fitness function.

[0024] Based on the pheromone concentration and fitness value of ants on each server, the third probability of ants choosing each server is calculated.

[0025] Select the server with the highest probability (third server) to obtain the target server.

[0026] In one possible implementation, the initial parameters of the ant colony algorithm are determined based on the demand parameters and the performance parameters of each server, including:

[0027] Initialize the execution nodes and the ant colony; where the number of execution nodes N is equal to the number of partition subsets; and the number of ants M in the ant colony is a positive integer multiple of N.

[0028] Initialize the pheromone matrix; where the size of the pheromone matrix is ​​N×M, and the initial pheromone concentration in the pheromone matrix is ​​a random value within a preset range.

[0029] In one possible implementation, the partition subset is obtained based on the following steps:

[0030] Based on the server's performance parameters, a server vector is constructed; the server vector is used to characterize the server's performance metrics.

[0031] Based on the server's location information, a location vector is constructed; the location vector is used to represent the server's location information.

[0032] The server vector and the position vector are merged according to their correspondence to obtain the server's state matrix;

[0033] According to the preset partitioning conditions, the state matrix is ​​partitioned to obtain a partitioning result that includes multiple partition subsets.

[0034] Secondly, embodiments of this application provide a task scheduling device, comprising:

[0035] The acquisition module is used to respond to the acquisition of the execution instruction of the target task, acquire the requirement parameters of the target task, and the partition subset of the server cluster to be executed and the performance parameters of each server in the server cluster; wherein, the server cluster includes at least one partition subset, and each partition subset contains at least one server.

[0036] The parameter determination module is used to determine the initial parameters of the ant colony algorithm based on the requirement parameters and the performance parameters of each server;

[0037] The execution module is used to execute the ant colony algorithm to determine the target server in the partition subset and send a task execution message to the target server; wherein, the target server is the server selected by the ants in the ant colony algorithm; the task execution message is used to instruct the target server to execute the target task;

[0038] The update and iteration module is used to update the pheromone concentration in the ant colony algorithm based on the target server, and iteratively execute the ant colony algorithm until all servers in the server cluster have completed the target task.

[0039] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0040] The memory stores computer-executed instructions;

[0041] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0042] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0043] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0044] The task scheduling method, electronic device, storage medium, and program product provided in this application determine the initial parameters of the ant colony algorithm based on the server's performance parameters and the target task's requirements parameters. Based on the ant colony algorithm, it finds a target server that meets the conditions to execute the target task, and continuously updates and iterates the ant colony algorithm to find new target servers until all servers have completed the target task. This achieves the effect of reasonably determining the order in which servers execute tasks and improving resource utilization after task scheduling. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0046] Figure 1 Flowchart of the task scheduling method provided in this application Figure 1 ;

[0047] Figure 2 Flowchart of the task scheduling method provided in this application Figure 2 ;

[0048] Figure 3 Flowchart of the task scheduling method provided in this application Figure 3 ;

[0049] Figure 4 A schematic diagram of the task scheduling device provided in this application;

[0050] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0051] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] First, let me explain the terms used in this application:

[0054] Ant Colony Optimization (ACO) is a heuristic algorithm inspired by the process of ants searching for food. This algorithm solves combinatorial optimization problems by simulating the behavior of ants releasing pheromones and choosing paths while searching for food.

[0055] The basic principles of ant colony optimization include the following key points:

[0056] Pheromones: Ants release a substance called "pheromones" as they move, marking the paths they take. The concentration of pheromones along a path reflects the ants' preference for that path.

[0057] Execution nodes: These typically refer to the nodes along the path traversed by the simulated ant during problem-solving. They may also be referred to as nodes in the following text.

[0058] Positive feedback: When an ant chooses a path, it will select it based on the concentration of pheromones along that path. Generally, paths with higher pheromone concentrations are more likely to be chosen, forming a positive feedback mechanism.

[0059] Pheromone Update: After an ant completes a path selection, it updates the pheromone concentration along the path based on the path's quality (such as length or cost). Typically, pheromone evaporation and the release of new pheromones combine to update the pheromone supply.

[0060] Parallel search: Multiple ants search for paths simultaneously. Each ant chooses its next action based on pheromones and heuristic rules, ultimately forming the overall path search process.

[0061] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0062] In a specific application environment, the task scheduling method provided in this application can be implemented based on the following hardware and software architecture conditions:

[0063] The system implementing this method is deployed on a Kubernetes cluster, with each functional module implemented as a microservice. The monitoring module collects resource and business metrics on each node within the cluster using DaemonSets. The scheduling module and knowledge base constitute a StatefulSet for state storage. The visualization module is deployed via Deployments. Different modules communicate with each other through a Service message bus.

[0064] Configuration requirements for the actual execution environment:

[0065] 1. Ansible Configuration: Ensure that the Ansible tool is correctly installed and configured on your execution node in order to execute commands.

[0066] 2. Host connectivity: Ensure uninterrupted network connectivity between the execution node and the host, which can be achieved through network configuration, VPN, etc.

[0067] 3. Permissions and Security: As needed, set appropriate permissions and security controls for execution nodes and hosts to protect sensitive data and system security.

[0068] Leveraging Kubernetes' scalability, the number of system module replicas can be dynamically adjusted based on monitored business load, achieving horizontal scaling. Simultaneously, service discovery and load balancing improve system availability.

[0069] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0070] Figure 1 Flowchart of the task scheduling method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0071] S101. In response to obtaining the execution instruction of the target task, obtain the requirement parameters of the target task, as well as the partition subset of the server cluster to be executed and the performance parameters of each server in the server cluster; wherein, the server cluster includes at least one partition subset, and each partition subset contains at least one server.

[0072] Specifically, a server cluster refers to a solution that connects multiple servers together to work together to provide high performance, high availability, or load balancing. In the application scenario of this application, servers can also be referred to as hosts. Typically, server clusters are interconnected via a network and use technologies such as load balancers to distribute and manage workloads. For ease of management and maintenance, servers in a server cluster can be divided into different subsets based on one or more conditions such as geographical location, network partition, and functional type. Optionally, based on actual operational needs, these subsets can be further subdivided into smaller subsets. In a specific application scenario, partition subsets can be divided based on address location; for example, servers in the same data center or in the same server room can be grouped into the same subset. Optionally, servers in the same network partition can also be grouped into the same subset.

[0073] The target task is a task to be executed by the server cluster, and all servers in the cluster need to execute the target task. However, due to differences in the current working status and performance parameters of each server, it is impossible to send instructions to make all servers execute the target task simultaneously. Therefore, the execution order of each server can be determined by the task scheduling method provided in the embodiments of this application.

[0074] Performance parameters may include, but are not limited to: CPU utilization, memory utilization, disk I / O (Input / Output) throughput, and network traffic. I / O throughput refers to the number of I / O operations successfully completed within a certain time, usually measured in MB / s. Higher throughput indicates a stronger ability of the system to handle I / O requests. Furthermore, a unique server performance parameter can be obtained by normalizing, standardizing, or weighting multiple performance parameters.

[0075] S102. Determine the initial parameters of the ant colony algorithm based on the required parameters and the performance parameters of each server.

[0076] Specifically, this application embodiment uses an improved ant colony algorithm to determine the execution order of each server. For the ant colony algorithm, it is necessary to initialize the execution nodes, the ant colony, and the pheromone matrix.

[0077] In one possible implementation, determining the initial parameters of the ant colony algorithm based on demand parameters and the performance parameters of each server may include the following steps:

[0078] S1021. Initialize the execution nodes and the ant colony; wherein the number of execution nodes N is equal to the number of partition subsets; the number of ants M in the ant colony is a positive integer multiple of N. Optionally, M = N.

[0079] S1022. Initialize the pheromone matrix; where the size of the pheromone matrix is ​​N×M, and the initial pheromone concentration in the pheromone matrix is ​​a random value within a preset range. Specifically, the pheromone concentration in the pheromone matrix represents the degree of attraction of each execution node to each ant. For example, the pheromone concentration τ(i,j) represents the degree of attraction of execution node j to ant i.

[0080] In addition, the fitness value in the ant colony algorithm needs to be calculated based on the target task's requirements and the performance parameters of each server. Fitness refers to the degree of dominance of an individual within the group, used to distinguish between "good" and "bad" individuals. The fitness function, also called the evaluation function, mainly determines the fitness of an individual based on its characteristics. In the application scenario of this application, fitness can be used to evaluate the degree of dominance among different partition subsets or among different servers. Individual characteristics mainly include...

[0081] S103. Execute the ant colony algorithm to determine the target server in the partition subset and send a task execution message to the target server; wherein, the target server is the server selected by the ants in the ant colony algorithm; the task execution message is used to instruct the target server to execute the target task.

[0082] Specifically, an ant colony algorithm can be executed. After the ants in the algorithm select a subset of partitions, the target server is then identified from within that subset. The selected target server is the server / host determined by the ant colony algorithm in this round, and it will be responsible for executing the target task in the current round. For example, based on the selected host, an operational script adapted to the task requirements can be automatically triggered. This script performs operational operations according to the task requirements, such as configuration updates and log analysis.

[0083] S104. Based on the target server, update the pheromone concentration in the ant colony algorithm and iteratively execute the ant colony algorithm until all servers in the server cluster have completed the target task.

[0084] Specifically, in the ant colony algorithm, after an ant completes a path selection, it updates the pheromone concentration along the path based on the path's quality (such as length or cost). In the application scenario of this application, the path chosen by the ant is obtained based on the selected target server, and then the pheromone concentration is updated. Afterward, the ant colony algorithm and pheromone concentration updates can be repeatedly executed iteratively, allowing the ants to select the target server for the next round, until all servers have completed their target tasks.

[0085] The task scheduling method provided in this application determines the initial parameters of the ant colony algorithm based on the server's performance parameters and the target task's requirements. Based on the ant colony algorithm, it finds a target server that meets the conditions to execute the target task, and continuously updates and iterates the ant colony algorithm to find new target servers until all servers have completed the target task. This achieves the effect of reasonably determining the order in which servers execute tasks and improving resource utilization after task scheduling.

[0086] In one possible implementation, such as Figure 2 As shown, the partition subset is obtained based on the following steps:

[0087] S201. Construct server vectors based on server performance parameters; server vectors are used to characterize server performance metrics.

[0088] Specifically, all host performance metrics (i.e., performance parameters) can be collected from the Prometheus monitoring system, including CPU utilization, memory usage, disk I / O throughput, network traffic, etc.

[0089] Based on the performance parameters of the servers, construct a server vector:

[0090] H = (h1, h2, ..., hn)

[0091] Where hi represents the performance metric vector of the i-th server, containing m metrics, hi = (hi1, hi2, ..., him). For example, if m = 4, hi includes four metrics: CPU, memory, IO, and network.

[0092] S202. Based on the server's location information, construct a location vector; the location vector is used to represent the server's location information.

[0093] Specifically, the configuration management system can collect the host's regional information (i.e., location information), including the data center, server room, network partition, etc.

[0094] Based on the location information of the server, construct a location vector:

[0095] Z = (z1, z2, ..., zn)

[0096] Here, zi represents the location information of the i-th server, including data centers, server rooms, etc.

[0097] S203. Merge the server vector and the position vector according to their correspondence to obtain the server's state matrix.

[0098] For example, the server vector and the position vector can be merged according to their correspondence to obtain the server state matrix S:

[0099]

[0100] Where hi represents the performance metric vector of the i-th server, and zi represents the location information of the i-th server.

[0101] S204. According to the preset partitioning conditions, the state matrix is ​​partitioned to obtain a partitioning result that includes multiple partition subsets.

[0102] Optionally, servers can be sharded from multiple dimensions. For example, first, regions can be divided by data center, and then performance levels can be defined based on performance metrics. The sharding result is a partition that divides the host set into subsets.

[0103] For example, the matrix S is partitioned according to preset partitioning conditions to obtain the partitioning result:

[0104] {H1,H2,...,Hk}

[0105] Where Hi is the i-th partition subset.

[0106] After the above steps, the host status information can be integrated, and tasks can be scheduled to the corresponding host subsets according to the sharding results. Based on this host sharding scheduling model, state-aware intelligent scheduling can be achieved.

[0107] The task scheduling method provided in this application constructs a host sharding scheduling model by using the server's performance parameters, thereby providing a basis for path finding for the ant colony algorithm and enabling state-aware intelligent scheduling.

[0108] In one possible implementation, such as Figure 3 As shown, executing the ant colony algorithm to determine the target server in a subset of partitions may include the following steps:

[0109] Step 301: Using each partition subset as an execution node, calculate the node performance index of the execution node based on the performance parameters of each server in the partition subset.

[0110] The node performance index represents the fitness of a partition subset for the target task. For each ant, a fitness value is calculated based on task characteristics and the performance of the executing node.

[0111] Step 302: Based on the demand parameters and node performance parameters, calculate the probability of each ant selecting each execution node to obtain the first selection probability.

[0112] Specifically, the probability P(i,j) that ant i chooses node j can be calculated using the following formula:

[0113]

[0114] Where α and β represent the importance of pheromone concentration and node performance, respectively, pheromone concentration τ(i,j) represents the attraction of execution node j to ant i, and η(i,j) represents the fitness value of node j for the target task.

[0115] Step 303: For each ant, based on the preset selection conditions and according to the first selection probability, determine the target execution node selected by each ant.

[0116] For example, a selection strategy from evolutionary algorithms can be used to select an execution node based on a calculated node selection probability. This process simulates the decision-making process of ants based on the attractiveness of pheromones and node performance.

[0117] Specifically, after node selection, the pheromone concentration is updated. The following pheromone update formula is used:

[0118] τ(i,j)=clip((1-ρ)*τ(i,j)+Δτ,τ_min,τ_max),

[0119] 0<ρ<1

[0120] Where ρ is the pheromone evaporation rate, and Δτ is the pheromone increment. Δτ is no longer a fixed value, but is adjusted based on the automated scheduling status and node performance. Let the probability of automated scheduling be p, and the node performance evaluation be η, then Δτ = p * η. τ_min and τ_max are the upper and lower limits of pheromone levels to prevent pheromone concentration from being too high or too low.

[0121] Step 304: In the target partition subset corresponding to the target execution node, select the server ranked first in the target partition subset according to the preset partition server sorting conditions, and obtain the target server selected by the ant.

[0122] After selecting an execution node, each ant has chosen one. Next, each ant needs to select the target server from the partition subset based on the task's predefined partition server sorting criteria.

[0123] In one possible implementation, the partition subset comprises several region subsets, and each region subset comprises several servers. The region subset can be a subset obtained by secondary partitioning according to the servers corresponding to regions within the data center, and can also be called a host region. After the execution node selection, each ant has selected an execution node.

[0124] After selecting the execution node, each ant needs to choose the optimal host region based on the task characteristics. At this point, heuristic information is introduced, incorporating task characteristics and node performance into the fitness function to guide the ant's search. Each ant selects a region from multiple host regions based on the task characteristics to minimize task transmission latency.

[0125] Within the target partition subset corresponding to the target execution node, the server ranked first in the target partition subset is selected according to the preset partition server sorting criteria. This process, known as ant selection, can include the following steps:

[0126] Step 3041: Based on the requirement parameters, obtain the heuristic information index corresponding to the target task.

[0127] In this context, heuristic information refers to the additional information used to guide an ant's decision-making when choosing a path. For each ant, a heuristic information value (i.e., a heuristic information index) can be calculated based on task characteristics (i.e., the task's requirement parameters). Specifically, the heuristic information function is H(χ), where χ represents task characteristics, such as task size and transmission requirements.

[0128] Step 3042: For the target partition subset, based on the pheromone concentration of ants in the region subset of the target partition subset and the heuristic information index, calculate the probability that the ant selects each region subset to obtain the second selection probability.

[0129] Specifically, the second selection probability is the probability that each ant chooses each host region. The probability that ant i chooses host region a can be calculated using the following formula:

[0130]

[0131] Where α and β represent pheromone concentration and the importance of heuristic information, respectively, and H(a) represents the heuristic information value of host region a for the task.

[0132] Step 3043: Determine the subset of regions with the highest probability of the second selection as the target region subset.

[0133] Specifically, it can simulate the decisions made by ants based on the attractiveness of pheromones and heuristic information, obtain the probability of ants choosing each host region, and determine the host region with the highest probability as the target host region.

[0134] In practical applications, after selecting a host region, the pheromone concentration is updated. The pheromone update formula is used:

[0135] Δτ=p*H

[0136] Here, Δτ is no longer a fixed value, but is adjusted based on the automated scheduling status and heuristic information. Let the probability of the automated scheduling status be p, and the heuristic evaluation be H. The probability of the automated scheduling status is the probability of the task's execution result after being sent to the host compared to the probability of successful execution across the entire host region.

[0137] Step 3044: In the target region subset, according to the preset region subset server sorting conditions, select the server ranked first in the target region subset to obtain the target server selected by the ant.

[0138] A region subset may contain at least one server. Therefore, it is also necessary to select the server ranked first in the target region subset according to the preset server sorting conditions within the region, so as to obtain the target server selected by the ants.

[0139] Specifically, after the host region allocation phase is completed, each ant has selected a host region. Now, each ant only needs to select a specific host within that region to perform maintenance tasks, prioritizing hosts with lower loads. The host selection decision is influenced by a combination of pheromone concentration and task characteristics.

[0140] In one possible implementation, the server ranked first in the target region subset is selected according to a preset region subset server sorting condition to obtain the target server selected by the ant, including:

[0141] Step 30441: Based on the server's current performance parameters, obtain the server's fitness value for the target task using the fitness function.

[0142] For each ant, the host fitness value is calculated within the selected host region based on the host's current load. The fitness function is G(τ,ω). Fitness refers to the degree of dominance of an individual within the group, used to distinguish between "good" and "bad" individuals. The fitness function, also called the evaluation function, primarily determines an individual's fitness based on its characteristics. The host fitness value is an assessment of the host's current performance, used to determine the order in which tasks are executed.

[0143] Step 30442: Based on the pheromone concentration and fitness value of each server for ants, calculate the third probability of ants choosing each server.

[0144] Specifically, the probability of each ant choosing each host can be calculated using the following formula, where the probability P(i,j) of ant i choosing host k is expressed as:

[0145]

[0146] Where α and β represent the importance of pheromone concentration and host fitness value, respectively, and ω(k) represents the fitness value of host k for the task.

[0147] Step 30443: Select the server with the highest probability to obtain the target server.

[0148] Specifically, we can simulate the decisions made by ants based on pheromones and host fitness values ​​to obtain the probability of each host being selected by the ants, and determine the host with the highest probability as the target server.

[0149] In practical applications, after selecting the host region, the pheromone concentration also needs to be updated. The pheromone update formula is used, where Δτ is no longer a fixed value but is adjusted based on task completion status and the host's current load. Let the task completion probability be p, and the host's current load assessment be ω, then Δτ = p * ω.

[0150] To further illustrate the solutions of the embodiments of this application, the following specific example is used to explain the steps of the ant colony algorithm in the automated job execution sharding process:

[0151] The first step is to establish a host fragmentation scheduling model.

[0152] 1) Collect all host performance metrics from the Prometheus monitoring system, including CPU utilization, memory usage, disk I / O, network traffic, etc., and construct the following vector:

[0153] H = (h1, h2, ..., hn)

[0154] Where, hi represents the performance metric vector of the i-th host, containing m metrics:

[0155] hi = (hi1, hi2, ..., him)

[0156] 2) Collect the host's regional information from the configuration management system, including the data center, server room, network partition, etc., and construct the following vector:

[0157] Z = (z1, z2, ..., zn)

[0158] Where zi represents the region information to which the i-th host belongs, including data centers, server rooms, etc.

[0159] 3) Constructing the host state matrix: Merge the performance index vector H and the region vector Z according to the host correspondence to construct the host state matrix:

[0160]

[0161] 4) Host Fragmentation

[0162] Based on matrix S, host resources are sharded from multiple dimensions. First, regions are divided according to data centers, and then performance levels are determined by performance metrics. The sharding result is a partition that divides the host set into subsets.

[0163] {H1,H2,...,Hk}

[0164] By integrating host status information, tasks are scheduled to corresponding host subsets according to the sharding results, achieving state-aware intelligent scheduling.

[0165] Step 2: Initialization

[0166] Initialize the execution nodes and ant colony: There are N execution nodes, each represented by an ant, and a total of M ants forming an ant colony.

[0167] Initialize the pheromone matrix: Create an N×M pheromone matrix to represent the attraction of each execution node to each ant.

[0168] Step 3: Task Issuance and Execution Node Selection Stage:

[0169] 1) Calculate fitness value:

[0170] For each ant, a fitness value is calculated based on task characteristics and node performance. Here, τ represents the pheromone concentration, and η represents the node performance evaluation. The fitness value is F(τ,

[0171] η). Node performance evaluation assesses a node's performance based on the tasks currently being executed by that node (number of tasks, task type, etc.).

[0172] 2) Calculation of node selection probability:

[0173] Calculate the probability that each ant will choose each execution node. Using the probability formula of the ant colony algorithm, the probability that ant i will choose node j is:

[0174]

[0175] Where α and β represent the importance of pheromone concentration and node performance, respectively, and η(i,j) represents the fitness value of node j for the task.

[0176] 3) Decision on selecting the execution node:

[0177] Using a preset selection method, an execution node is selected based on the calculated node selection probability. This process simulates the decision-making process of ants based on the attractiveness of pheromones and node performance.

[0178] 4) Pheromone renewal:

[0179] After node selection, the pheromone concentration is updated. The pheromone update formula is used, where ρ is the pheromone evaporation rate and Δτ is the pheromone increment. Δτ is no longer a fixed value but is adjusted based on the automated scheduling status and node performance. Let the probability of automated scheduling be p, and the node performance evaluation be η, then Δτ = p * η

[0180] The pheromone update formula is:

[0181] τ(i,j)=clip((1-ρ)*τ(i,j)+Δτ,τ_min,τ_max),0<ρ<1

[0182] Wherein, τ_min and τ_max are the upper and lower limits of pheromones, to prevent pheromone concentrations from being too high or too low.

[0183] Step 4: Host Area Allocation

[0184] After selecting an execution node, each ant has chosen one. Now, each ant needs to select the optimal host region based on task characteristics. Heuristic information is introduced here, incorporating task characteristics and node performance into the fitness function to guide the ant's search. Each ant selects a region from multiple host regions based on task characteristics to minimize task transmission latency.

[0185] 1) Calculate heuristic information: For each ant, calculate the heuristic information value based on the task characteristics. The heuristic information function is H(χ), where χ represents the task characteristics, such as the task size and transmission requirements.

[0186] 2) Calculate the probability that each ant chooses each host region. The probability that ant i chooses region a is:

[0187]

[0188] Where α and β represent pheromone concentration and the importance of heuristic information, respectively, and H(a) represents the heuristic information value of host region a for the task.

[0189] 3) Host region selection decision: This simulates the decision made by ants based on the attractiveness of pheromones and heuristic information.

[0190] 4) Pheromone renewal:

[0191] After selecting the host region, the pheromone concentration is updated. The pheromone update formula is used, where Δτ is no longer a fixed value but is adjusted based on the automated scheduling status and heuristic information. Let the probability of the automated scheduling status be p, and the heuristic information evaluation be H, then Δτ = p * H.

[0192] Step 5: Host Selection and Operation / Maintenance Script Execution Phase

[0193] After the host region allocation phase is completed, each ant has selected a host region. Now, each ant only needs to select a specific host within that region to perform maintenance tasks, prioritizing hosts with lower loads. The host selection decision is influenced by a combination of pheromone concentration and task characteristics.

[0194] 1) Calculate the host fitness value: For each ant, evaluate and calculate the host fitness value within the selected host region based on the current host load. The fitness function is G(τ,ω).

[0195] 2) Calculation of host selection probability

[0196] Calculate the probability that each ant chooses each host. The probability that ant i chooses host k is:

[0197]

[0198] Where α and β represent the importance of pheromone concentration and host fitness value, respectively, and ω(k) represents the fitness value of host k for the task.

[0199] 3) Host selection decision: The decision made by ants based on pheromones and host fitness values.

[0200] 4) Automatically trigger maintenance scripts: Based on the selected host, automatically trigger maintenance scripts that meet the task requirements. The scripts perform maintenance operations according to the task requirements, such as configuration updates and log analysis.

[0201] 5) Pheromone renewal:

[0202] After selecting the host region, update the pheromone concentration. Use the pheromone update formula, where Δτ is no longer a fixed value but is adjusted based on task completion status and the host's current load. Let the task completion probability be p, and the host's current load assessment be ω, then Δτ = p * ω

[0203] Step 6: Iteration

[0204] Repeat the above steps until the convergence condition is met, that is, all servers have completed the task.

[0205] In each iteration, the movement of ants and the updating of pheromones interact, and based on the host sharding model, the appropriate execution nodes are selected, appropriate host regions are allocated, and low-load hosts are searched to complete the operation and maintenance tasks.

[0206] Step 7: Output Results

[0207] Finally, based on the iterative results of the ant colony algorithm, the optimal execution path is obtained. These results can be used to execute commands in batches, ensuring that each execution node and host executes commands efficiently with balanced load.

[0208] The task scheduling method provided in this application introduces heuristic information into the ant colony algorithm, incorporating task characteristics, node performance, host load, and other parameters into the fitness function to guide the ants' search. Pheromone updates are adjusted to adaptive updates, dynamically adjusting the update rate based on the ants' exploration progress and the task's complexity. To prevent excessive concentration or rapid dissipation, a pheromone range constraint is added, limiting pheromone values ​​to a reasonable range during updates. This method combines the core ideas and relevant formulas of the ant colony algorithm at each stage, enabling ants to make intelligent choices and decisions based on pheromones, heuristic information, node performance, and task characteristics, thereby achieving fully automated operation and maintenance.

[0209] Figure 4 A schematic diagram of the task scheduling device provided in this application is shown below. Figure 4 As shown, the task scheduling device 40 provided in this embodiment includes:

[0210] The acquisition module 401 is used to acquire the required parameters of the target task in response to the acquisition of the execution instruction of the target task, as well as the partition subset of the server cluster to be executed and the performance parameters of each server in the server cluster; wherein, the server cluster includes at least one partition subset, and each partition subset contains at least one server.

[0211] The parameter determination module 402 is used to determine the initial parameters of the ant colony algorithm based on the requirement parameters and the performance parameters of each server.

[0212] The execution module 403 is used to execute the ant colony algorithm to determine the target server in the partition subset and send a task execution message to the target server; wherein, the target server is the server selected by the ants in the ant colony algorithm; the task execution message is used to instruct the target server to execute the target task;

[0213] The update iteration module 404 is used to update the pheromone concentration in the ant colony algorithm according to the target server, and iteratively execute the ant colony algorithm until all servers in the server cluster have completed the target task.

[0214] In one possible implementation, the execution module is further configured to:

[0215] Each partition subset is treated as an execution node, and the node performance index of the execution node is calculated based on the performance parameters of each server in the partition subset.

[0216] Based on the demand parameters and node performance parameters, the probability of each ant choosing each execution node is calculated to obtain the first selection probability;

[0217] For each ant, based on preset selection conditions and according to the first selection probability, the target execution node selected by each ant is determined;

[0218] In the target partition subset corresponding to the target execution node, the server ranked first in the target partition subset is selected according to the preset partition server sorting conditions, and the target server selected by the ant is obtained.

[0219] In one possible implementation, the partition subset includes several region subsets, and each region subset includes several servers;

[0220] The execution module is also used for:

[0221] Based on the requirement parameters, the heuristic information index corresponding to the target task is obtained;

[0222] For the target partition subset, based on the pheromone concentration of ants in the region subset of the target partition subset and the heuristic information index, the probability of the ant choosing each region subset is calculated to obtain the second selection probability;

[0223] Determine the subset of regions with the highest probability of the second choice as the target region subset;

[0224] Within the target region subset, the server ranked first in the target region subset is selected according to the preset region subset server sorting conditions, thus obtaining the target server selected by the ant.

[0225] In one possible implementation, the execution module is further configured to:

[0226] Based on the server's current performance parameters, the fitness value of the server for the target task is obtained using the fitness function.

[0227] Based on the pheromone concentration and fitness value of ants on each server, the third probability of ants choosing each server is calculated.

[0228] Select the server with the highest probability (third server) to obtain the target server.

[0229] In one possible implementation, the parameter determination module is further configured to:

[0230] Initialize the execution nodes and the ant colony; where the number of execution nodes N is equal to the number of partition subsets; and the number of ants M in the ant colony is a positive integer multiple of N.

[0231] Initialize the pheromone matrix; where the size of the pheromone matrix is ​​N×M, and the initial pheromone concentration in the pheromone matrix is ​​a random value within a preset range.

[0232] In one possible implementation, the partition subset is obtained based on the following steps:

[0233] Based on the server's performance parameters, a server vector is constructed; the server vector is used to characterize the server's performance metrics.

[0234] Based on the server's location information, a location vector is constructed; the location vector is used to represent the server's location information.

[0235] The server vector and the position vector are merged according to their correspondence to obtain the server's state matrix;

[0236] According to the preset partitioning conditions, the state matrix is ​​partitioned to obtain a partitioning result that includes multiple partition subsets.

[0237] The task scheduling device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0238] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus.

[0239] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0240] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0241] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0242] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0243] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0244] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0245] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0246] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0247] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0248] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0249] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0250] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0251] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0252] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0253] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A task scheduling method, characterized by, The method comprises: in response to obtaining an execution instruction of a target task, obtaining a requirement parameter of the target task, and a partition subset of a server cluster to be executed the target task and a performance parameter of each server in the server cluster; wherein the server cluster comprises at least one partition subset, and each partition subset comprises at least one server; determining initial parameters of an ant colony algorithm according to the requirement parameter and the performance parameter of each server; executing the ant colony algorithm to determine a target server in the partition subset and sending a task execution message to the target server; wherein the target server is a server selected by an ant in the ant colony algorithm; and the task execution message is used to instruct the target server to execute the target task; updating pheromone concentration in the ant colony algorithm according to the target server, and iteratively executing the ant colony algorithm until the servers in the server cluster all execute the target task.

2. The method of claim 1, wherein, The method of executing the ant colony algorithm to determine a target server in the partition subset comprises: taking each partition subset as an execution node, calculating a node performance index of the execution node based on the performance parameter of each server in the partition subset; calculating a probability of each ant selecting each execution node based on the requirement parameter and the node performance parameter to obtain a first selection probability; for each ant, determining a target execution node selected by each ant according to the first selection probability based on a preset selection condition; in a target partition subset corresponding to the target execution node, selecting a server ranked first in the target partition subset according to a preset partition server ranking condition to obtain a target server selected by the ant.

3. The method of claim 2, wherein, The partition subset comprises a plurality of region subsets, and each region subset comprises a plurality of servers. The method of selecting a server ranked first in the target region subset according to a preset region subset server ranking condition to obtain a target server selected by the ant comprises: obtaining a heuristic information index corresponding to the target task according to the requirement parameter; for the target partition subset, calculating a probability of the ant selecting each region subset based on the pheromone concentration of the ant in the region subset of the target partition subset and the heuristic information index to obtain a second selection probability; determining a region subset with the highest second selection probability as a target region subset; in the target region subset, selecting a server ranked first in the target region subset according to a preset region subset server ranking condition to obtain a target server selected by the ant.

4. The method of claim 3, wherein, The method of selecting a server ranked first in the target region subset according to a preset region subset server ranking condition to obtain a target server selected by the ant comprises: obtaining an adaptability value of the server for the target task based on an adaptability function according to the current performance parameter of the server. The third probability of the ant selecting each of the servers is calculated based on the pheromone concentration of the ant and the fitness value of each of the servers; The server with the highest third probability is selected as the target server.

5. The method according to any one of claims 1 to 4, characterized in that, The initial parameters of the ant colony algorithm are determined according to the demand parameter and the performance parameter of each server, including: The execution nodes and the ant colony are initialized; the number of the execution nodes N is equal to the number of the partition subsets; the number of the ants in the ant colony M is a positive integer multiple of N; The pheromone matrix is initialized; the size of the pheromone matrix is N×M, and the initial pheromone concentration in the pheromone matrix is a random value in a preset range.

6. The method according to any one of claims 1 to 4, characterized in that, The partition subsets are obtained based on the following steps: The server vector is constructed based on the performance parameter of the server; the server vector is used to represent the performance indicators of the server; The location vector is constructed based on the location information of the server; The location vector is used to represent the location information of the server; The server vector and the location vector are merged according to the corresponding relationship to obtain the state matrix of the server; The state matrix is processed according to the preset sharding condition to obtain the sharding result including multiple partition subsets.

7. A task scheduling apparatus characterized by comprising: including: The acquisition module is configured to acquire the demand parameter of the target task, and the partition subset of the server cluster to be executed, and the performance parameter of each server in the server cluster in response to acquiring the execution instruction of the target task; wherein the server cluster includes at least one partition subset, and each partition subset includes at least one server; The parameter determination module is configured to determine the initial parameters of the ant colony algorithm according to the demand parameter and the performance parameter of each server; The execution module is configured to execute the ant colony algorithm to determine the target server in the partition subset and send a task execution message to the target server; wherein the target server is the server selected by the ant in the ant colony algorithm; the task execution message is used to instruct the target server to execute the target task; The update iteration module is configured to update the pheromone concentration in the ant colony algorithm according to the target server, and iteratively execute the ant colony algorithm until the servers in the server cluster all execute the target task.

8. An electronic device, comprising: including: The memory and the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of claims 1-6.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method of any one of claims 1-6.

Images