A cluster task scheduling method, device, electronic equipment and medium

Abstract

The embodiment of the present invention provides a method, device, electronic device and medium for cluster task scheduling, including: obtaining performance indicator values ​​of multiple resource nodes in a cluster, and dynamically partitioning the multiple resource nodes according to the performance indicator values ​​to obtain multiple resource node intervals; judging whether the resource occupancy rate of the cluster is less than the occupancy rate threshold; when the resource occupancy rate is less than the occupancy rate threshold, determining a target resource node interval from multiple resource node intervals, and generating a target task scheduling strategy according to the number of connections of resource nodes in the target resource node interval; when the resource occupancy rate is greater than or equal to the occupancy rate threshold, using a preset particle swarm algorithm to determine a target task scheduling strategy from multiple task scheduling strategies; and scheduling tasks for the cluster according to the target task scheduling strategy, so as to achieve dynamic partitioning by obtaining the performance indicator values ​​of the resource nodes, and judging the resource occupancy rate to generate a task scheduling strategy for execution.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, electronic equipment and medium for cluster task scheduling. Background Art

[0002] In the existing heterogeneous cluster environment, the design and application of load balancing algorithms face many challenges. Heterogeneous clusters are generally composed of multiple different types of computing nodes, which have differences in hardware configuration, processing power, resource capacity, etc. For example, these nodes can be different types of servers, virtual machines or containers. Therefore, how to effectively achieve load balancing in heterogeneous clusters to improve the performance and efficiency of the overall system has always been the focus of technical personnel. However, the existing load balancing algorithms still have some significant problems in practical applications and are difficult to meet the needs. Summary of the invention

[0003] In view of the above problems, a method, device, electronic device and medium for cluster task scheduling are proposed to overcome the above problems or at least partially solve the above problems, including:

[0004] A cluster task scheduling method, wherein the cluster includes a plurality of resource nodes, and the method includes:

[0005] Acquire performance indicator values ​​of multiple resource nodes in the cluster, and dynamically partition the multiple resource nodes according to the performance indicator values ​​to obtain multiple resource node intervals;

[0006] Determine whether the resource occupancy rate of the cluster is less than an occupancy rate threshold;

[0007] In a case where the resource occupancy rate is less than the occupancy rate threshold, determining a target resource node interval from the multiple resource node intervals, and generating a target task scheduling strategy according to the number of connections of resource nodes in the target resource node interval;

[0008] When the resource occupancy rate is greater than or equal to the occupancy rate threshold, a preset particle swarm algorithm is used to determine a target task scheduling strategy from a plurality of task scheduling strategies;

[0009] According to the target task scheduling strategy, tasks are scheduled for the cluster.

[0010] Optionally, obtaining performance indicator values ​​of multiple resource nodes in the cluster includes:

[0011] For each resource node in the cluster, obtain the available resource amount; wherein the available resource amount includes the available resource amount of one or more types of resources;

[0012] For each type of resource, the resource proportion of the available resource amount to the available resource amount of all resource nodes in the cluster is determined, and the performance indicator value of the resource node is determined according to the resource proportion of all types of resources and the corresponding degree of dependence.

[0013] Optionally, dynamically partitioning the multiple resource nodes according to the performance indicator value to obtain multiple resource node intervals includes:

[0014] Pre-set multiple interval indicator value ranges;

[0015] Dynamically partitioning the multiple resource nodes according to the matching relationship between the performance indicator value and the multiple interval indicator value ranges to obtain multiple resource node intervals;

[0016] The determining of a target resource node interval from the plurality of resource node intervals comprises:

[0017] From the multiple resource node intervals, determine the resource node interval with the largest interval index value range as the target resource node interval.

[0018] Optionally, generating a target task scheduling strategy according to the number of connections of resource nodes in the target resource node interval includes:

[0019] Determine the number of connections of resource nodes in the target resource node interval; wherein the number of connections is positively correlated with the number of tasks being processed by the resource node;

[0020] From the resource nodes in the target resource node interval, determine the resource node with the smallest number of connections, and establish a target task scheduling strategy between the task to be scheduled and the resource node with the smallest number of connections.

[0021] Optionally, the using of a preset particle swarm algorithm to determine a target task scheduling strategy from a plurality of task scheduling strategies includes:

[0022] Use the preset particle swarm algorithm to initialize the particle swarm;

[0023] Iteratively updating the particle swarm and calculating the evaluation value of the particle swarm updated each time; wherein each particle swarm updated each time corresponds to a task scheduling strategy;

[0024] According to the evaluation value, a target particle group is determined from the particle group updated in each iteration, and the task scheduling strategy corresponding to the target particle group is used as the target task scheduling strategy.

[0025] Optionally, the calculating of the evaluation value of the particle swarm updated in each iteration includes:

[0026] For each iteratively updated particle swarm, determine the time and cost required for all scheduled tasks to be executed under the task scheduling strategy corresponding to the particle swarm;

[0027] An evaluation value of the particle swarm is determined according to the time and the cost.

[0028] Optionally, the particle swarm includes a plurality of particles, each particle corresponds to a scheduling relationship between a task to be scheduled and a resource node in the task scheduling strategy, and the iterative updating of the particle swarm includes:

[0029] The positions and velocities of the particles in the particle group are iteratively updated.

[0030] A device for cluster task scheduling, the cluster comprising a plurality of resource nodes, the device comprising:

[0031] A resource interval partitioning module, used to obtain performance indicator values ​​of multiple resource nodes in the cluster, and dynamically partition the multiple resource nodes according to the performance indicator values ​​to obtain multiple resource node intervals;

[0032] A resource occupancy rate determination module, used to determine whether the resource occupancy rate of the cluster is less than an occupancy rate threshold;

[0033] According to the connection number scheduling module, it is used to determine the target resource node interval from the multiple resource node intervals when the resource occupancy rate is less than the occupancy rate threshold, and generate a target task scheduling strategy according to the number of connections of the resource nodes in the target resource node interval;

[0034] According to the particle swarm scheduling module, when the resource occupancy rate is greater than or equal to the occupancy rate threshold, a preset particle swarm algorithm is used to determine a target task scheduling strategy from a plurality of task scheduling strategies;

[0035] The task scheduling module is used to schedule tasks for the cluster according to the target task scheduling strategy.

[0036] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method described above when executed by the processor.

[0037] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0038] The embodiments of the present invention have the following advantages:

[0039] In an embodiment of the present invention, performance indicator values ​​of multiple resource nodes in a cluster are obtained, and the multiple resource nodes are dynamically partitioned according to the performance indicator values ​​to obtain multiple resource node intervals; it is determined whether the resource occupancy rate of the cluster is less than the occupancy rate threshold; when the resource occupancy rate is less than the occupancy rate threshold, a target resource node interval is determined from the multiple resource node intervals, and a target task scheduling strategy is generated according to the number of connections of the resource nodes in the target resource node interval; when the resource occupancy rate is greater than or equal to the occupancy rate threshold, a preset particle swarm algorithm is used to determine the target task scheduling strategy from the multiple task scheduling strategies; according to the target task scheduling strategy, the cluster is scheduled for tasks, and dynamic partitioning is achieved by obtaining the performance indicator values ​​of the resource nodes, and the resource occupancy rate is determined to generate a task scheduling strategy for execution, so that the task allocation strategy can be dynamically adjusted according to the real-time load status of the cluster and the performance differences of the resource nodes, thereby optimizing resource usage and improving system performance and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0041] Figure 1 is a schematic diagram of an adaptive task scheduling provided by some embodiments of the present invention;

[0042] Figure 2 is a flowchart of the steps of a cluster task scheduling method provided by some embodiments of the present invention;

[0043] Figure 3 is a flowchart of the steps of a task scheduling method provided by some embodiments of the present invention;

[0044] Figure 4 is a flowchart of steps for outputting an optimal solution based on a particle swarm algorithm provided by some embodiments of the present invention;

[0045] Figure 5 It is a structural block diagram of a cluster task scheduling device provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] In the related art, load balancing algorithms include Round Robin, Weighted Round Robin, Least Connections, etc.

[0048] The polling method evenly distributes tasks to each resource node in sequence, without making reasonable allocation based on the performance of the resource nodes. It is relatively simple but only applicable to scenarios with similar server performance.

[0049] The minimum number of connections method is suitable for scenarios where server performance varies greatly. The resource node with the least number of connections is selected to process the current request. The weighted minimum number of connections method selects based on the current number of connections and weight of the server. The request is assigned to the server with the least number of connections, which improves the availability and performance of the system. However, the problem is that when adding a new node, it may cause excessive pressure on the new node, causing system instability such as overload. The load performance of the node changes dynamically, and the number of connections and weights are difficult to accurately reflect the actual load of the node.

[0050] In the minimum number of connections method, the algorithm allocates new connection requests based on the current number of connections of each node and the preset weight. However, for newly added nodes, since they have fewer connections in the initial stage, according to the principle of weighted minimum number of connections, the new node may receive too many connection requests in a short period of time, causing it to be overloaded, which in turn affects the processing efficiency and stability of the node. In addition, the number of connections and weights of a node are often difficult to accurately reflect its actual load situation, because the load performance of a node changes dynamically and is affected by many factors; therefore, allocating tasks based solely on the number of connections and weights may result in poor load balancing effects.

[0051] In Flink (an open source stream processing framework), a powerful distributed data stream processing framework and runtime environment are provided. Its default task scheduling strategy is a static scheduling strategy. When a job is submitted, Flink builds a job graph (JobGraph) according to the user-defined logic, and then assigns each operator in the job graph to the appropriate TaskManager node and determines the concurrency of the task. Once the assignment is completed, the task will be executed in a statically assigned manner. Flink's default task scheduling treats all nodes as having the same performance and cannot be adjusted according to the real-time load. In a heterogeneous cluster, resource nodes with different performance complete the same tasks, resulting in uneven resource utilization and distribution.

[0052] In terms of task scheduling strategy optimization in heterogeneous clusters, the following optimizations are mainly implemented:

[0053] 1. Use the random pick-k algorithm to schedule tasks randomly through probability, and implement task scheduling through the smooth weighted polling algorithm. Perform smoothing in the weighted process to avoid nodes with large weights being selected multiple times, and pay attention to the performance and load changes of different nodes.

[0054] 2. Sort tasks according to user levels and task types of different priorities, and use the weighted minimum connection number algorithm to schedule tasks.

[0055] 3. Combine the improved pick-k algorithm to optimize the weighted polling load balancing algorithm, and optimize dynamic load balancing by dynamically adjusting the weight of the server.

[0056] Among them, the pick-k algorithm is an algorithm for randomly extracting K elements from a data set containing a large number of elements. Its characteristic is that K elements are selected in a probabilistic manner without traversing the entire data set, and the probability of each element being selected is equal.

[0057] Regarding the load balancing algorithm in heterogeneous clusters in related technologies, the main problems are as follows:

[0058] 1. The weighted minimum number of connections will cause the newly added nodes to be overloaded in a short period of time, affecting the node efficiency. In addition, the number of connections and weights of the nodes are difficult to accurately reflect their load conditions, and the load performance of the nodes changes dynamically.

[0059] 2. The current load balancing algorithm rarely considers the processing when the resources used by the cluster load balancing algorithm are higher than the set threshold;

[0060] 3. In a distributed heterogeneous cluster, the change of node load is related to multiple factors of the node, and these factors need to be comprehensively considered to dynamically adjust the node load.

[0061] In an embodiment of the present invention, an adaptive task scheduling method in a heterogeneous cluster is proposed, such as Figure 1 , mainly including data acquisition module, data storage module, dynamically adjusted weighted minimum connection number module and task scheduling module based on particle swarm algorithm. Among them, the data acquisition module is responsible for periodically collecting various indicator data of resource nodes, including CPU usage, memory utilization, bandwidth, etc.; the data storage module stores the collected indicator data.

[0062] The dynamically adjusted weighted minimum connection number module dynamically adjusts the weights based on the node's performance indicators, so that the weights better reflect the node's load conditions. It partitions each node according to the performance indicators, groups nodes with similar performance into one area, and compares the number of connections. This avoids the load imbalance that may be caused by the scenario where the newly added node is mounted to multiple load balancing instances, and also better solves the problem of local overload on the newly added node in a short period of time.

[0063] The task scheduling module based on the particle swarm algorithm uses the task execution time and cost as the objective function when the resource usage in the heterogeneous cluster exceeds the set threshold, and uses the particle swarm algorithm to perform a global search to find the optimal solution, that is, the optimal task scheduling solution.

[0064] In a specific implementation, the embodiments of the present invention may include the following:

[0065] 1. Obtain the performance indicators of resource nodes and dynamically adjust the weights according to the actual load conditions;

[0066] 2. The performance proportion of each node is classified by the percentile method. Nodes with similar performance indicators are divided into a range, and the number of server connections is compared in each classification range;

[0067] 3. When the usage of resources in the cluster exceeds the threshold, a task scheduling algorithm based on the particle swarm algorithm is used to perform a global search to find the globally optimal allocation solution.

[0068] The present invention is described below with reference to the accompanying drawings:

[0069] Reference Figure 2 , showing a step flow chart of a cluster task scheduling method provided by some embodiments of the present invention, wherein the cluster includes multiple resource nodes.

[0070] A cluster is a group of computers (or other computing devices) connected via a high-speed network that work together to provide higher performance, reliability, and scalability than a single computer. Clusters can be homogeneous (all nodes have the same hardware configuration and operating system) or heterogeneous (nodes differ in hardware, operating system, processing power, etc.).

[0071] Resource nodes refer to the basic computing units in a cluster, and each node has certain computing resources and storage capabilities. These resources include but are not limited to CPU (Central Processing Unit), memory, disk storage space, network interface, etc. Resource nodes can be physical servers, virtual machines, containers, or other forms of computing instances. In a cluster, resource nodes are usually configured to perform specific tasks or provide specific services, such as database services, Web services (a technology for service-oriented architecture), computing-intensive tasks, etc.

[0072] Specifically, the following steps may be included:

[0073] Step 201 , obtaining performance indicator values ​​of a plurality of resource nodes in the cluster, and dynamically partitioning the plurality of resource nodes according to the performance indicator values ​​to obtain a plurality of resource node intervals.

[0074] As some examples, the system or administrator will collect real-time or recent performance indicator data of each resource node in the cluster through various means (such as monitoring tools, API calls, etc.). These performance indicators usually reflect the status of multiple aspects such as the node's computing power, storage capacity, network bandwidth, response time, etc.

[0075] Among them, performance indicators may include CPU usage, memory occupancy, disk I / O (Input / Output) rate, network throughput, system response time, etc. These indicators can fully reflect the current load and potential performance of resource nodes.

[0076] Dynamic partitioning refers to a method of dividing nodes into different intervals or categories based on the performance index values ​​of resource nodes. This partitioning is dynamic, that is, it can be adjusted as the node performance index changes.

[0077] The system will evaluate and compare resource nodes based on the collected performance indicator values, and divide them into different resource node intervals according to the performance differences and load conditions of the nodes.

[0078] The nodes in each interval have certain similarities in performance, which helps with subsequent task scheduling and resource allocation. For example, high-performance, low-load nodes can be divided into one interval, while low-performance, heavy-load nodes can be divided into another interval. After dynamic partitioning, the system will obtain multiple resource node intervals, each of which contains a group of nodes with similar performance characteristics. These intervals provide the basis for subsequent task scheduling and resource management. The system can select nodes from the appropriate interval to perform tasks based on the requirements of the task and the performance characteristics of the nodes, thereby achieving optimal configuration and efficient utilization of resources.

[0079] In some embodiments of the present invention, obtaining the performance indicator values ​​of multiple resource nodes in the cluster includes: obtaining the available resource amount for each resource node in the cluster; wherein the available resource amount includes the available resource amount of one or more types of resources; for each type of resource, determining the resource ratio of the available resource amount to the available resource amount of all resource nodes in the cluster, and determining the performance indicator value of the resource node based on the resource ratio of all types of resources and the corresponding degree of dependence.

[0080] Among them, for each resource node in the cluster, the available resource amount is obtained; it means that the system will evaluate the resources of each resource node in the cluster to determine its current available resource amount. Resources can be CPU usage, memory size, disk storage space, network bandwidth, etc. For each type of resource, the system can record its current available quantity or capacity.

[0081] For each type of resource, determining the resource proportion of the available resource amount to the available resource amount of the resource nodes in the cluster means that the system will calculate the proportion of the available resource amount of each resource node to the total available amount of resources of this type of all resource nodes in the entire cluster, that is, the resource proportion.

[0082] For example, for CPU resources, the system calculates the ratio of each node's CPU available time to the total CPU available time of the entire cluster; for memory resources, the system calculates the ratio of each node's available memory size to the total available memory size of the entire cluster.

[0083] Determine the performance indicator value of the resource node according to the resource proportion of the type of resource and the corresponding degree of dependence; that is, after obtaining the resource proportion of each type of resource, the system will determine the performance indicator value of each resource node according to these proportions and the degree of dependence of each resource on system performance or task execution.

[0084] As some examples, the degree of dependency can be a subjective evaluation indicator that reflects the importance of a certain resource to system performance or task execution. For example, for compute-intensive tasks, the degree of dependency on CPU resources may be high; while for data-intensive tasks, the degree of dependency on storage resources may be higher. The system can use weighted average or other algorithms to combine the resource proportion of each resource with its degree of dependency to derive a comprehensive performance indicator value. This value can reflect the potential contribution of the resource node to system performance or task execution in its current state.

[0085] For example, first obtain the performance indicators of resource nodes. The CPU value, memory value, and network bandwidth value of a single resource node are C(i), M(i), and B(i), which all represent the amount of resources available for a single node. The performance weight (performance indicator value) of the i-th resource node in the cluster is W(i):

[0086]

[0087] Among them, μ1, μ2, and μ3 represent the degree of dependence of resource nodes on various performance indicators, μ1+μ2+μ3=1; W(w i ) represents the weight of load balancing.

[0088] As some examples, since the performance indicators C(i)C(i), M(i)M(i), and B(i)B(i) of the resource nodes are changing, the required W(i) W(i) also reflects the real performance indicators of the nodes in a timely manner, E(w i )E(w i ) is also adjusted in time with the execution of tasks. Since various performance indicators are collected periodically through the data collection module, they are not real-time, which also reduces the pressure of real-time monitoring data.

[0089] In some embodiments of the present invention, dynamically partitioning the multiple resource nodes according to the performance indicator value to obtain multiple resource node intervals includes: presetting multiple interval indicator value ranges; dynamically partitioning the multiple resource nodes according to the matching relationship between the performance indicator value and the multiple interval indicator value ranges to obtain multiple resource node intervals; determining the target resource node interval from the multiple resource node intervals includes: determining the resource node interval with the largest interval indicator value range as the target resource node interval from the multiple resource node intervals.

[0090] Among them, pre-setting multiple interval indicator value ranges means that the system or administrator can pre-define multiple intervals according to certain standards or requirements, and set an indicator value range for each interval. These indicator value ranges can represent different performance levels, resource utilization, load levels or any other parameters that can be used to measure the status of resource nodes.

[0091] For example, for CPU utilization, you can set three intervals: low (0%-30%), medium (31%-70%), and high (71%-100%).

[0092] Based on the matching relationship between performance indicator values ​​and multiple interval indicator value ranges, multiple resource nodes are dynamically partitioned to obtain multiple resource node intervals; this means that the system will collect the performance indicator values ​​(such as CPU utilization, memory usage, etc.) of each resource node (such as servers, virtual machines, containers, etc.) in real time or regularly. Then, the system will dynamically allocate resource nodes to different intervals based on the matching relationship between these performance indicator values ​​and the previously set interval indicator value ranges. In this way, similar resource nodes will be classified into the same interval, forming multiple resource node intervals.

[0093] For example, if a server's CPU utilization is 65%, it might be assigned to the Medium range above.

[0094] Determine the target resource node interval from the multiple resource node intervals; means that the system needs to select one as the target resource node interval from the multiple resource node intervals that have been formed. Determine the resource node interval with the largest interval index value range as the target resource node interval, that is, the system will find the interval with the highest upper limit (or the lowest lower limit, depending on the specific index) of the interval index value range as the target resource node interval.

[0095] For example, if the index value range of the above "high" interval is 71%-100%, and the index value range of the "medium" interval is 31%-70%, then the "high" interval will be determined as the target resource node interval because it represents a higher performance index value range.

[0096] As some examples, the performance indicator value W(i) W(i) of each node can be classified according to percentiles. The first quartile is represented by Q1, the second quartile is represented by Q2, the third quartile is represented by Q3, and the fourth quartile is represented by Q4. The W(i) W(i) corresponding to each node is divided into four categories according to the percentile size: Q1 (0-25%), Q2 (25-50%), Q3 (50-75%), and Q4 (75-100%), which is equivalent to dividing the nodes into four parts according to the performance indicators.

[0097] In some examples, a counter can be set at each node to count the number of connections of each node in each interval, and the number of connections in each interval is compared; among Q1, Q2, Q3, and Q4, resource nodes with better performance (such as Q4 nodes, Q4 is 75-100% of the interval) and fewer connections are selected.

[0098] Since the value of W(i) W(i) changes periodically, the nodes in each interval also change. When a new node is added, it is divided into the corresponding interval according to its performance index and compared according to the number of connections. This avoids the scenario where the node is mounted to multiple load balancing instances, which leads to unbalanced load distribution. It also better solves the problem of local overload of newly added nodes in a short period of time.

[0099] Step 202: determine whether the resource occupancy rate of the cluster is less than an occupancy rate threshold.

[0100] Resource utilization refers to the current usage of one or more resources in the cluster, usually expressed as a percentage. For example, CPU utilization may indicate the proportion of the CPU time currently in use to the total time.

[0101] The occupancy threshold is a preset limit used to determine whether resources are overused. This threshold can be set based on factors such as cluster specifications, business requirements, and cost-effectiveness.

[0102] In actual applications, the system or algorithm will collect the occupancy data of each resource in the cluster periodically or in real time. Then, this data is compared with the preset occupancy threshold. If the occupancy of a resource is less than the threshold, it is considered that the resource has sufficient remaining capacity to process new tasks or requests.

[0103] If the occupancy is greater than or equal to the threshold, further action may be required, such as adding more resources, optimizing how existing resources are utilized, or rejecting new task requests.

[0104] As some examples, it can be determined whether the resource occupancy of the cluster is less than an occupancy threshold. Based on the determination result, the system can make corresponding decisions, such as scheduling new tasks to nodes with lower resource occupancy, or triggering a resource expansion mechanism to increase the capacity of the cluster.

[0105] Step 203, when the resource occupancy rate is less than the occupancy rate threshold, determine a target resource node interval from the multiple resource node intervals, and generate a target task scheduling strategy according to the number of connections of resource nodes in the target resource node interval.

[0106] In actual applications, the system or algorithm will check the current resource occupancy, that is, the occupancy rate of one or more resources (such as computing resources, storage resources, network resources, etc.). This occupancy rate reflects the current usage or load of the resource. Then, the system will compare this resource occupancy rate with a preset occupancy rate threshold. This threshold is a boundary used to determine whether the resource is overused or has enough remaining capacity to process new tasks or requests.

[0107] If the resource occupancy rate is less than the threshold, it indicates that the current resource has enough remaining capacity to process more tasks. In this case, the system can select one from multiple resource node intervals as the target resource node interval.

[0108] After determining the target resource node interval, the system will generate a target task scheduling strategy based on the number of connections of the resource nodes in this interval. The number of connections can be the number of network connections between nodes, the number of connections between nodes and external systems, or the number of connections of tasks or data flows established on the nodes. The target task scheduling strategy can consider how to most effectively utilize these resource nodes to ensure optimization of task execution efficiency, load balancing, response time, etc.

[0109] In some embodiments of the present invention, generating a target task scheduling strategy according to the number of connections of resource nodes in the target resource node interval includes: determining the number of connections of resource nodes in the target resource node interval; wherein the number of connections is positively correlated with the number of tasks being processed by the resource node;

[0110] From the resource nodes in the target resource node interval, determine the resource node with the smallest number of connections, and establish a target task scheduling strategy between the task to be scheduled and the resource node with the smallest number of connections.

[0111] After determining the target resource node interval from multiple resource node intervals, the system can calculate or obtain the current number of connections of these resource nodes. The number of connections can be understood as the number of active connections established by the resource node with other nodes (which may be other resource nodes, clients, data storage, etc.).

[0112] The number of connections is positively correlated with the number of tasks being processed by the resource node; that is, the more connections a resource node has, the more tasks it may be processing, and therefore its load may be relatively heavy.

[0113] This positive correlation is based on experience or system design assumptions, which can help the system indirectly evaluate the load of resource nodes through the number of connections when scheduling tasks.

[0114] Determine the resource node with the smallest number of connections from the resource nodes in the target resource node interval; refers to the system comparing the number of connections of each resource node in the target resource node interval to find the node with the smallest number of connections. Selecting the node with the smallest number of connections as the target execution node for the task to be scheduled is based on load balancing and optimal resource utilization. Because nodes with smaller numbers of connections may have lighter loads and more capabilities to handle new tasks. Then, based on the above comparison, the system will establish a target task scheduling strategy between the task to be scheduled and the resource node with the smallest number of connections. The target task scheduling strategy may include the task allocation method, execution order, configuration parameters of the resource node, etc., to ensure that the task can be executed efficiently and correctly on the target resource node.

[0115] As some examples, the target task scheduling strategy is generated according to the number of connections of resource nodes in the target resource node interval, such as Figure 3 This can include:

[0116] [1]Number each node.

[0117] [2] Compute the performance weight of the node (performance index value).

[0118] [3] Classify according to performance weight (performance index value) (dynamically partition multiple resource nodes to obtain multiple resource node intervals).

[0119] [4] Count the number of connections of nodes (resource nodes) in each area (resource node interval).

[0120] [5] Calculate the weight (resource occupancy) of each node.

[0121] [6] Compare the number of connections / weights for better performance.

[0122] [7] The new task is assigned to the server with the smallest weighted nodes.

[0123] Step 204 , when the resource occupancy rate is greater than or equal to the occupancy rate threshold, a preset particle swarm algorithm is used to determine a target task scheduling strategy from a plurality of task scheduling strategies.

[0124] If the resource occupancy rate is greater than or equal to the occupancy rate threshold, it means that the current resource usage has approached or reached the limit that the system can bear, which may lead to performance degradation, task delays or resource competition. In this case, the system can use the preset particle swarm algorithm to optimize resource utilization with a smarter and more efficient task scheduling strategy to ensure the smooth completion of the task.

[0125] Among them, the particle swarm optimization (PSO) is an optimization algorithm based on swarm intelligence, which simulates the collective behavior of natural phenomena such as bird flocks foraging. By using the particle swarm algorithm, the optimal target task scheduling strategy is searched from multiple optional task scheduling strategies. Among them, the "particles" in the particle swarm algorithm can represent different task scheduling strategies, and they gradually approach the optimal solution by iteratively updating their positions and velocities.

[0126] Through the search process of the particle swarm algorithm, the system can find an optimal or approximately optimal strategy from multiple candidate task scheduling strategies. This optimal strategy can maximize task completion efficiency, minimize resource usage, or achieve other optimization goals while meeting system performance requirements, resource constraints, and business needs.

[0127] In some embodiments of the present invention, the use of a preset particle swarm algorithm to determine a target task scheduling strategy from multiple task scheduling strategies includes: using a preset particle swarm algorithm to initialize a particle swarm; iteratively updating the particle swarm and calculating an evaluation value of the particle swarm updated each time; wherein each iteratively updated particle swarm corresponds to a task scheduling strategy; based on the evaluation value, determining a target particle swarm from the particle swarm updated each time, and using the task scheduling strategy corresponding to the target particle swarm as the target task scheduling strategy.

[0128] Among them, the preset particle swarm algorithm is used to initialize the particle swarm; it means that the system first initializes a particle swarm according to the preset particle swarm algorithm parameters (such as the number of particles, the position initialization range, the speed initialization range, etc.). Each particle represents a potential solution in the algorithm, that is, a task scheduling strategy. The position of the particle can be regarded as a parameter or configuration of the strategy, and the speed of the particle represents the direction and rate of change of these parameters or configurations in the search space.

[0129] Iteratively update the particle swarm; after initialization, the system will iteratively update the particle swarm multiple times. In each iteration, each particle will update its position and speed based on its own historical best position (pBest) and the historical best position of the entire particle swarm (gBest). This update mechanism enables particles to move to a better position, thereby gradually approaching the global optimal solution.

[0130] Calculate the evaluation value of the particle swarm for each iteration update; this means that after each iteration update, the system will calculate the evaluation value of the particle swarm (or more specifically, each particle in the particle swarm). This evaluation value is obtained by applying the task scheduling strategy represented by the particle to the actual system or simulation environment and observing its performance (such as task completion time, resource occupancy, system throughput, etc.). The high or low evaluation value can reflect the quality of the task scheduling strategy and is the basis for the algorithm to select the optimal solution.

[0131] According to the evaluation value, the target particle group is determined from the particle group updated in each iteration; it means that after multiple iterations, the system will select the optimal particle group (i.e., the particle group with the highest evaluation value) from the particle group updated in each iteration according to the evaluation value; this optimal particle group (i.e., the target particle group) represents the optimal task scheduling strategy currently searched. Among them, the target particle group is determined after iterative updates based on the individual historical optimal value and the group historical optimal value.

[0132] The task scheduling strategy corresponding to the target particle swarm is the target task scheduling strategy; it means that the system will determine the task scheduling strategy corresponding to the optimal particle swarm as the target task scheduling strategy, and apply it to the actual system to achieve efficient scheduling of tasks and optimal utilization of resources.

[0133] In some embodiments of the present invention, the calculation of the evaluation value of the particle swarm updated each time iteratively includes: determining, for the particle swarm updated each time iteratively, the time and cost required for all scheduled tasks to be executed and completed under the task scheduling strategy corresponding to the particle swarm; and determining the evaluation value of the particle swarm based on the time and the cost.

[0134] Among them, for each iteratively updated particle swarm, the time and cost required for all scheduled tasks to be executed and completed under the task scheduling strategy corresponding to the particle swarm are determined; it means that during the iteration process of the particle swarm algorithm, the state of the particle swarm, including the position and speed of each particle, will be updated each time. These updated particles represent different task scheduling strategies. For each updated particle swarm (i.e., each task scheduling strategy), the system needs to simulate or actually execute the strategy to observe the time and cost required for all scheduled tasks to be executed and completed under the strategy. Time can refer to the total time required for a task to be executed from the beginning to the completion, which reflects the efficiency of task scheduling. The cost can be a value including resource occupancy (such as CPU, memory, network bandwidth, etc.), energy consumption, execution cost, etc., reflecting the economy or sustainability of task scheduling.

[0135] Determine the evaluation value of the particle swarm based on time and cost; that is, after obtaining the time and cost corresponding to each particle swarm, the system will calculate the evaluation value of the particle swarm based on these indicators.

[0136] The calculation method of the evaluation value may vary depending on the specific application, but generally the trade-off between time and cost is considered. For example, in some cases, the system may pay more attention to time efficiency, so time is used as the main evaluation indicator; in other cases, the system may pay more attention to resource consumption, so cost is used as the main evaluation indicator. The high or low evaluation value reflects the performance of the particle swarm (i.e., the task scheduling strategy) and is the basis for the particle swarm algorithm to select the optimal solution.

[0137] In some embodiments of the present invention, the particle swarm includes multiple particles, each particle corresponds to the scheduling relationship between the task to be scheduled and the resource node in the task scheduling strategy, and the iterative updating of the particle swarm includes: iteratively updating the position and speed of the particles in the particle swarm.

[0138] In the particle swarm algorithm, each particle has two key attributes: position and velocity. Position represents the current position of the particle in the search space, that is, the parameters or configuration of the task scheduling strategy. Velocity represents the direction and speed of the particle moving in the search space, that is, the adjustment direction and size of the parameters or configuration of the task scheduling strategy.

[0139] Iterative update is one of the core steps of the particle swarm algorithm, which occurs in every iteration of the algorithm. In each iteration, each particle updates its position and velocity based on its own historical optimal position (pBest) and the historical optimal position of the entire particle swarm (gBest). The formula for updating the position can include the current position, the current velocity, and an adjustment item related to pBest and gBest.

[0140] The formula for updating the speed may include the current speed, a random term, and an adjustment term related to the difference between the positions of pBest and gBest.

[0141] In some examples, through multiple iterative updates, the particle swarm algorithm is able to explore the search space, find solutions (task scheduling strategies) with better performance, and eventually converge to the optimal solution or a near-optimal solution.

[0142] As some examples, during the operation, if the cluster occupancy reaches the set resource threshold, the task scheduling algorithm based on the particle swarm algorithm can be enabled. After repeated iterations, the optimal solution is found among all the task scheduling solutions to improve the efficiency of calculation and reduce the waiting time. In the particle swarm, the position of each particle represents an executable task scheduling solution, with execution time and cost as the optimization goals, to obtain the global optimal solution, that is, the optimal allocation solution. The following steps may be included:

[0143] [1] Initialize the particle swarm and set the particle swarm parameters, including particle size, inertia weight, learning factor, number of iterations, particle dimension, and velocity range; the number of initialized particles is 10, the maximum number of iterations is 100, and the learning factors c1 and c2 are both 2.

[0144] The code for initializing the particle swarm is: Xi = {x i，1 , x i，2 , x i，3 ,...x i，n}X i ={x i，1 , x i，2 , x i，3 ,...x i，n}, X i represents the particle swarm, x i represents the i-th particle in the particle swarm; i∈[1,m] represents the allocation scheme of the i-th task to the resource node, m represents the number of tasks; n≤N task , N task is the number of tasks in the task set; x i，j ∈[1,m],x i，j It means that the j-th task is assigned to the i-th resource node.

[0145] The velocity of a particle is defined as: v i =(v i，1 , v i，2 , v i，3 ,...v i，n ), where i∈[1,m], n∈[1,N task ];

[0146] For example, a possible particle code is (2, 4, 3, 1, 3, 1, 3), and the corresponding relationship is as shown in Table 1: tasks 4 and 6 are assigned to resource node 1 for execution, task 1 is assigned to resource node 2 for execution, tasks 3, 5, and 7 are assigned to resource node 3 for execution; task 2 is assigned to resource node 4 for execution.

[0147] Task No. Resource node number 1 2 2 4 3 3 4 1 5 3 6 1 7 3

[0148] Table 1

[0149] The particle decoding is shown in Table 2:

[0150] Resource Node 1 2 3 4 Subtask number 4、6 1 3、5、7 2

[0151] Table 2

[0152] Before each iteration, check whether the particle position exceeds the boundary and set the following constraints:

[0153]

[0154] Among them, boundary refers to the boundary range of movement; x max Refers to the unknown boundary, the maximum value of the position; In the equation, K is the number of iterations, Represents the position of particle i at the Kth iteration.

[0155] [2] Calculate the optimization function: Set the optimization function to evaluate each particle and update the position and velocity by comparing the objective function f value; the total execution time to complete all tasks on resource j is:

[0156]

[0157] In the formula, U time (i, j) represents the execution time of task i on resource j; if its value is 0, it means that task i is not scheduled to be executed on resource j; where 1≤j≤m, m is the number of resource nodes; 1≤j≤n, n is the number of tasks; the total time to complete all tasks is O time for:

[0158] O time =max(Time(j))

[0159] A feasible scheduling scheme corresponds to a particle x in the particle swarm optimization algorithm i ; The total time to complete all tasks is converted into the optimization function:

[0160] f=min(O time (X)),X=(x1,…,xi i ,…,x n )

[0161] Not only the task execution time but also the cost needs to be considered. Here we assume that C cpu C mem C bw They represent the usage costs of CPU, memory and bandwidth respectively. The cost for a user to execute the i-th task is:

[0162] Cost(i)=u1*C cpu +u2*C mem +u3*C bw

[0163] Where u1, u2, and u3 represent the usage of CPU, memory, and bandwidth used to complete task i, respectively.

[0164] The fitness function is defined as:

[0165] f=1 / (O time(X)+Cost(i))

[0166] Among them, f means using a function's f value to evaluate each particle, and the function's f value is equal to 1 divided by the resource's execution time plus the cost.

[0167] [3] Update the current individual optimal value and group optimal value: If the function value f of the i-th particle after the k-th iteration is higher than the current function optimal value Then the individual optimal position Set to the current position and update the optimal value of the function, otherwise and remain unchanged; if the optimal value of the function in all individuals Better than the global optimal value of the function recorded in the group Update is the optimal value of the particle Corresponding position And update the global optimal value of the function

[0168] [4] Update speed and position. The update formula is as follows:

[0169]

[0170] Among them, w represents the inertia weight; c1 and c2 are learning factors; r1 and r2 are random numbers between 0 and 1. represents the individual optimal value of the i-th particle after k iterations, Represents the global optimal value obtained after k iterations.

[0171] When updating the speed, the speed is limited, and the initial maximum speed is set to v max , and its constraint formula is as follows:

[0172]

[0173] [5] Dynamically update the inertia weight: Update the inertia weight according to the linear change strategy:

[0174]

[0175] As shown in the formula, as the number of iterations k increases, w t Continuously decreases, so that the PSO algorithm has a strong global search capability in the early stage and a strong local search capability in the middle and late stages; among them, w max The value of w is 2. min The value of is 0.4.

[0176] [6] Check whether the end condition is met. If the current number of iterations is greater than the set maximum number of iterations, stop the iteration and output the globally optimal particle code. After the iteration, the particle swarm algorithm optimized with task execution time and cost as the objective function obtains the global optimal solution, that is, the global optimal task allocation scheme, and assigns tasks to corresponding resource nodes according to the allocation of the scheme.

[0177] As some examples, taking execution time and cost as optimization objectives, the process of obtaining the global optimal solution, that is, the optimal allocation solution, is as follows Figure 4 , which may include:

[0178] [1] Initialize the particle swarm parameters and input the particle swarm size, particle swarm dimension, and number of iterations.

[0179] [2] Randomly initialize the particle position and velocity, and output the individual historical optimal solution and the group particle optimal solution.

[0180] [3] Input the individual historical optimal solution and the group particle optimal solution, and determine whether the maximum number of iterations has been reached. If not, proceed to [4]; if yes, proceed to [8].

[0181] [4] Calculate the objective function value.

[0182] [5] Update individual historical optimal solutions and positions.

[0183] [6] Update the optimal solution and position of the swarm particles.

[0184] [7] Update other parameters, such as inertia weight, number of iterations, etc., and then proceed to [3].

[0185] [8] Output the optimal solution.

[0186] Step 105: Schedule tasks for the cluster according to the target task scheduling strategy.

[0187] After determining the target task scheduling strategy, the system will guide the task scheduling process in the cluster according to the strategy. The system can assign tasks to be executed to different nodes in the cluster according to the rules, algorithms or models defined in the strategy. During the assignment process, the system can consider various factors, such as the computing requirements of the task, the resource status of the node, network latency, etc., to ensure that the task can be executed efficiently and fully utilize the cluster resources.

[0188] In an embodiment of the present invention, performance indicator values ​​of multiple resource nodes in a cluster are obtained, and the multiple resource nodes are dynamically partitioned according to the performance indicator values ​​to obtain multiple resource node intervals; it is determined whether the resource occupancy rate of the cluster is less than the occupancy rate threshold; when the resource occupancy rate is less than the occupancy rate threshold, a target resource node interval is determined from the multiple resource node intervals, and a target task scheduling strategy is generated according to the number of connections of the resource nodes in the target resource node interval; when the resource occupancy rate is greater than or equal to the occupancy rate threshold, a preset particle swarm algorithm is used to determine the target task scheduling strategy from the multiple task scheduling strategies; according to the target task scheduling strategy, the cluster is scheduled for tasks, and dynamic partitioning is achieved by obtaining the performance indicator values ​​of the resource nodes, and the resource occupancy rate is determined to generate a task scheduling strategy for execution, so that the task allocation strategy can be dynamically adjusted according to the real-time load status of the cluster and the performance differences of the resource nodes, thereby optimizing resource usage and improving system performance and stability.

[0189] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0190] Reference Figure 5 , shows a schematic diagram of the structure of a cluster task scheduling device provided by some embodiments of the present invention, which may specifically include the following modules:

[0191] The resource interval partitioning module 501 is used to obtain performance indicator values ​​of multiple resource nodes in the cluster, and dynamically partition the multiple resource nodes according to the performance indicator values ​​to obtain multiple resource node intervals;

[0192] The resource occupancy rate determination module 502 is used to determine whether the resource occupancy rate of the cluster is less than an occupancy rate threshold;

[0193] According to the connection number scheduling module 503, when the resource occupancy rate is less than the occupancy rate threshold, a target resource node interval is determined from the multiple resource node intervals, and a target task scheduling strategy is generated according to the number of connections of the resource nodes in the target resource node interval;

[0194] According to the particle swarm scheduling module 504, when the resource occupancy rate is greater than or equal to the occupancy rate threshold, a preset particle swarm algorithm is used to determine a target task scheduling strategy from a plurality of task scheduling strategies;

[0195] The task scheduling module 505 is used to schedule tasks for the cluster according to the target task scheduling strategy.

[0196] In some embodiments of the present invention, the resource interval division module 501 is used to:

[0197] For each resource node in the cluster, obtain the available resource amount; wherein the available resource amount includes the available resource amount of one or more types of resources;

[0198] For each type of resource, the resource proportion of the available resource amount to the available resource amount of all resource nodes in the cluster is determined, and the performance indicator value of the resource node is determined according to the resource proportion of all types of resources and the corresponding degree of dependence.

[0199] In some embodiments of the present invention, the resource interval division module 501 is used to:

[0200] Pre-set multiple interval indicator value ranges;

[0201] Dynamically partitioning the multiple resource nodes according to the matching relationship between the performance indicator value and the multiple interval indicator value ranges to obtain multiple resource node intervals;

[0202] The determining of a target resource node interval from the plurality of resource node intervals comprises:

[0203] From the multiple resource node intervals, determine the resource node interval with the largest interval index value range as the target resource node interval.

[0204] In some embodiments of the present invention, the connection number scheduling module 503 is used to:

[0205] Determine the number of connections of resource nodes in the target resource node interval; wherein the number of connections is positively correlated with the number of tasks being processed by the resource node;

[0206] From the resource nodes in the target resource node interval, determine the resource node with the smallest number of connections, and establish a target task scheduling strategy between the task to be scheduled and the resource node with the smallest number of connections.

[0207] In some embodiments of the present invention, the particle swarm scheduling module 504 is used to:

[0208] Use the preset particle swarm algorithm to initialize the particle swarm;

[0209] Iteratively updating the particle swarm and calculating the evaluation value of the particle swarm updated each time; wherein each particle swarm updated each time corresponds to a task scheduling strategy;

[0210] According to the evaluation value, a target particle group is determined from the particle group updated in each iteration, and the task scheduling strategy corresponding to the target particle group is used as the target task scheduling strategy.

[0211] In some embodiments of the present invention, the step of calculating the evaluation value of the particle swarm updated in each iteration includes:

[0212] For each iteratively updated particle swarm, determine the time and cost required for all scheduled tasks to be executed under the task scheduling strategy corresponding to the particle swarm;

[0213] An evaluation value of the particle swarm is determined according to the time and the cost.

[0214] In some embodiments of the present invention, the particle swarm includes a plurality of particles, each particle corresponds to a scheduling relationship between a task to be scheduled and a resource node in the task scheduling strategy, and the iterative updating of the particle swarm includes:

[0215] The positions and velocities of the particles in the particle group are iteratively updated.

[0216] Some embodiments of the present invention further provide an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the above method is implemented when the computer program is executed by the processor.

[0217] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0218] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above method when executed by a processor.

[0219] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0220] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0221] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0222] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0223] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0224] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0225] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0226] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0227] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0228] The above is a detailed introduction to the provided method, device, electronic device and medium for cluster task scheduling. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A cluster task scheduling method, characterized in that: The cluster includes a plurality of resource nodes, and the method includes: Acquire performance indicator values ​​of multiple resource nodes in the cluster, and dynamically partition the multiple resource nodes according to the performance indicator values ​​to obtain multiple resource node intervals; Determine whether the resource occupancy rate of the cluster is less than an occupancy rate threshold; In a case where the resource occupancy rate is less than the occupancy rate threshold, determining a target resource node interval from the multiple resource node intervals, and generating a target task scheduling strategy according to the number of connections of resource nodes in the target resource node interval; When the resource occupancy rate is greater than or equal to the occupancy rate threshold, a preset particle swarm algorithm is used to determine a target task scheduling strategy from a plurality of task scheduling strategies; According to the target task scheduling strategy, tasks are scheduled for the cluster.

2. The method according to claim 1, characterized in that: The obtaining of performance indicator values ​​of multiple resource nodes in the cluster includes: For each resource node in the cluster, obtain the available resource amount; wherein the available resource amount includes the available resource amount of one or more types of resources; For each type of resource, the resource proportion of the available resource amount to the available resource amount of all resource nodes in the cluster is determined, and the performance indicator value of the resource node is determined according to the resource proportion of all types of resources and the corresponding degree of dependence.

3. The method according to claim 1, characterized in that The dynamically partitioning the multiple resource nodes according to the performance indicator value to obtain multiple resource node intervals includes: Pre-set multiple interval indicator value ranges; Dynamically partitioning the multiple resource nodes according to the matching relationship between the performance indicator value and the multiple interval indicator value ranges to obtain multiple resource node intervals; The determining of a target resource node interval from the plurality of resource node intervals comprises: From the multiple resource node intervals, determine the resource node interval with the largest interval index value range as the target resource node interval.

4. The method according to any one of claims 1 to 3, characterized in that: The generating a target task scheduling strategy according to the number of connections of resource nodes in the target resource node interval includes: Determine the number of connections of resource nodes in the target resource node interval; wherein the number of connections is positively correlated with the number of tasks being processed by the resource node; From the resource nodes in the target resource node interval, determine the resource node with the smallest number of connections, and establish a target task scheduling strategy between the task to be scheduled and the resource node with the smallest number of connections.

5. The method according to any one of claims 1 to 3, characterized in that: The preset particle swarm algorithm is used to determine the target task scheduling strategy from multiple task scheduling strategies, including: Use the preset particle swarm algorithm to initialize the particle swarm; Iteratively updating the particle swarm and calculating the evaluation value of the particle swarm updated each time; wherein each particle swarm updated each time corresponds to a task scheduling strategy; According to the evaluation value, a target particle group is determined from the particle group updated in each iteration, and the task scheduling strategy corresponding to the target particle group is used as the target task scheduling strategy.

6. The method according to claim 5, characterized in that The step of calculating the evaluation value of the particle swarm updated in each iteration includes: For each iteratively updated particle swarm, determine the time and cost required for all scheduled tasks to be executed under the task scheduling strategy corresponding to the particle swarm; An evaluation value of the particle swarm is determined according to the time and the cost.

7. The method according to claim 5, characterized in that The particle swarm includes a plurality of particles, each particle corresponds to a scheduling relationship between a task to be scheduled and a resource node in the task scheduling strategy, and the iterative updating of the particle swarm includes: The positions and velocities of the particles in the particle group are iteratively updated.

8. A cluster task scheduling device, characterized in that: The cluster includes a plurality of resource nodes, and the device includes: A resource interval partitioning module, used to obtain performance indicator values ​​of multiple resource nodes in the cluster, and dynamically partition the multiple resource nodes according to the performance indicator values ​​to obtain multiple resource node intervals; A resource occupancy rate determination module, used to determine whether the resource occupancy rate of the cluster is less than an occupancy rate threshold; According to the connection number scheduling module, it is used to determine the target resource node interval from the multiple resource node intervals when the resource occupancy rate is less than the occupancy rate threshold, and generate a target task scheduling strategy according to the number of connections of the resource nodes in the target resource node interval; According to the particle swarm scheduling module, when the resource occupancy rate is greater than or equal to the occupancy rate threshold, a preset particle swarm algorithm is used to determine a target task scheduling strategy from a plurality of task scheduling strategies; The task scheduling module is used to schedule tasks for the cluster according to the target task scheduling strategy.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images