Task allocation method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a task allocation method and device, equipment and a storage medium. The method comprises the following steps: for each preset task type, according to a real-time dynamic value of each associated load index of each service node in a distributed cluster and an index division interval of each associated load index under the preset task type, determining a real-time dynamic code of each associated load index of each service node under the preset task type; according to the real-time dynamic code of each associated load index of each service node under the preset task type, determining a real-time load balancing parameter of the service node under the preset task type; and when each to-be-executed task is received, determining a task allocation node of the to-be-executed task according to the real-time load balancing parameter of each service node under the target task type of the to-be-executed task. According to the embodiment of the invention, the load balance of each service node in the distributed cluster can be ensured, and the task execution efficiency of the whole distributed cluster is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically to a method, apparatus, device and storage medium for task allocation. Background Art

[0002] With the booming development of the global data economy and the rapid rise of technologies such as 5G, artificial intelligence, and the Internet of Things, the amount of global data is showing an explosive growth trend. In order to meet the high real-time processing of various data-related tasks, the big data computing engine (Apache Flink) came into being, using distributed clusters to perform corresponding stream processing on each task.

[0003] At present, the Apache Flink engine can usually use a polling scheduling algorithm to cyclically assign each task to the corresponding service node in the distributed cluster, so that each service node executes the assigned task respectively. However, when each service node and each task have certain heterogeneity, the polling scheduling algorithm is likely to cause uneven load on each service node after task assignment, which greatly reduces the task execution efficiency of the entire distributed cluster. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device and storage medium for task allocation, which realize reasonable task allocation within a distributed cluster, ensure load balancing of each service node within the distributed cluster after task allocation, and improve the task execution efficiency of the entire distributed cluster.

[0005] In a first aspect, an embodiment of the present application provides a method for task allocation, the method comprising:

[0006] For each preset task type, according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type, determine the real-time dynamic coding of each associated load indicator of each service node under the preset task type;

[0007] Determine the real-time load balancing parameter of the service node under the preset task type according to the real-time dynamic encoding of each associated load indicator of each service node under the preset task type;

[0008] When each task to be executed is received, a task allocation node for the task to be executed is determined according to the real-time load balancing parameters of each service node under the target task type of the task to be executed.

[0009] In a second aspect, an embodiment of the present application provides a device for task allocation, the device comprising:

[0010] A dynamic coding determination module is used to determine, for each preset task type, the real-time dynamic coding of each associated load indicator of each service node under the preset task type according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type;

[0011] A node load balancing module, configured to determine a real-time load balancing parameter of the service node under the preset task type according to the real-time dynamic encoding of each associated load indicator of each service node under the preset task type;

[0012] The task allocation module is used to determine the task allocation node of the task to be executed according to the real-time load balancing parameters of each service node under the target task type of the task to be executed each time a task to be executed is received.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0014] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the task allocation method provided in the first aspect of the present application.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the task allocation method provided in the first aspect of the present application.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method of task allocation provided in the first aspect of the present application.

[0017] The embodiment of the present application provides a method, device, equipment and storage medium for task allocation. For each task in a distributed cluster, the present application pre-configures multiple task types. For each preset task type, by analyzing the real-time dynamic value of each associated load index of each service node in the distributed cluster and the index division interval of each associated load index under the preset task type, the real-time dynamic encoding of each associated load index of each service node under the preset task type is determined, so as to dynamically maintain the real-time load balancing parameters of each service node under the preset task type. Then, when each task to be executed is received, the task allocation node of the task to be executed can be determined by analyzing the real-time load balancing parameters of each service node under the target task type of the task to be executed, thereby realizing reasonable task allocation in the distributed cluster, and by dynamically maintaining the real-time load balancing parameters of each service node under each preset task type, the heterogeneity problem existing in each service node and each task in the distributed cluster is eliminated, and the real-time load status of each service node is measured respectively under multiple preset task types, so as to ensure the load balancing of each service node in the distributed cluster after task allocation, and improve the task execution efficiency of the entire distributed cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flowchart of a method for task allocation shown in an embodiment of the present application;

[0020] Figure 2 A method flow chart of a specific dynamic maintenance process of real-time load balancing parameters of each service node under each preset task type shown in an embodiment of the present application;

[0021] Figure 3a and Figure 3b They are respectively exemplary principle diagrams of the indicator division intervals after each associated load indicator is divided under a certain preset task type as shown in the embodiments of the present application;

[0022] Figure 4 A principle block diagram of a task allocation device shown in an embodiment of the present application;

[0023] Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0026] In order to solve the problem that the polling scheduling algorithm used for task allocation in a distributed cluster easily leads to uneven load on each service node in the distributed cluster after task allocation, the embodiment of the present application designs a new task allocation scheme. By pre-configuring multiple task types, for each preset task type, the real-time dynamic value of each associated load index of each service node in the distributed cluster and the index division interval of each associated load index under the preset task type are analyzed, so as to determine the real-time dynamic encoding of each associated load index of each service node under the preset task type, and dynamically maintain the real-time load balancing parameters of each service node under the preset task type, thereby eliminating the heterogeneity problem of each service node and each task in the distributed cluster, and measuring the real-time load status of each service node from multiple preset task types. Then, when each task to be executed is received, the task allocation node of the task to be executed can be determined by analyzing the real-time load balancing parameters of each service node under the target task type of the task to be executed, thereby realizing the reasonable allocation of tasks in the distributed cluster, ensuring the load balancing of each service node in the distributed cluster after task allocation, and improving the task execution efficiency of the entire distributed cluster.

[0027] Figure 1 This is a flowchart of a method for task allocation shown in an embodiment of the present application. Figure 1 , the method may include the following steps:

[0028] S110, for each preset task type, determine the real-time dynamic encoding of each associated load indicator of each service node under the preset task type according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type.

[0029] Considering that there is a certain degree of heterogeneity among various tasks in a distributed cluster, it means that various service nodes in a distributed cluster have different processing performance requirements for different types of tasks. Then, in order to eliminate the influence of heterogeneity of different tasks, this application can pre-configure multiple task types as preset task types in this application by analyzing various tasks that need to be processed in a distributed cluster, so as to measure the real-time load status of each service node in the distributed cluster separately under multiple preset task types.

[0030] Among them, the tasks under the multiple preset task types may include but are not limited to memory-intensive tasks, network-intensive tasks, disk-intensive tasks, central processing unit (CPU)-intensive tasks, etc.

[0031] Moreover, considering that different service nodes in a distributed cluster also have certain heterogeneity, the performance of different service nodes is different (for example, the CPU performance, memory size, network bandwidth, disk input / output (IO) of different service nodes are different), which will make different service nodes have different processing efficiency for the same task. The processing performance of each service node for any task can be determined by the load state of the service node. For example, the smaller the load of a service node, the better the processing performance of the service node. The larger the load of a service node, the worse the processing performance of the service node.

[0032] It can be seen that in order to accurately analyze the reasonable allocation of each task to each service node in the distributed cluster, this application first needs to measure the real-time load status of each service node under each preset task type to analyze the processing performance of each service node in the distributed cluster for each task under different preset task types. Then, for each service node in the distributed cluster, this application can uniformly configure multiple associated load indicators for each service node, and each associated load indicator can represent the load status of the service node from one perspective.

[0033] The associated load indicators of each service node may include, but are not limited to, CPU utilization, memory utilization, disk utilization, network bandwidth utilization, etc. of each service node.

[0034] Generally, in order to accurately describe the load status of each service node, each associated load indicator can be divided into multiple different numerical intervals according to the data variation range of each associated load indicator. In this way, the specific data value of each associated load indicator of each service node can be used to determine in which numerical interval each associated load indicator of each service node is in real time, thereby analyzing the real-time load status of each service node.

[0035] However, since the various associated load indicators of any service node have different degrees of influence on the processing performance of different types of tasks, for example, when a service node is processing CPU-intensive tasks, the processing performance will be seriously affected by the CPU utilization of the service node, while other associated load indicators have less influence on its performance. Therefore, in order to accurately analyze the load status of each service node when processing different types of tasks, the present application can analyze the degree of influence of each associated load indicator on the task processing performance under the preset task type for each preset task type. Then, when dividing each associated load indicator into a plurality of different numerical intervals, each associated load indicator can be divided into a plurality of data intervals of different interval lengths under the preset task type according to the different degrees of influence of each associated load indicator on the task processing performance under the preset task type, so as to obtain the indicator division interval of each associated load indicator under the preset task type. For example, the greater the impact of a certain associated load indicator on the task processing performance under the preset task type, it means that it is necessary to focus on refining the real-time changes of the associated load indicator, and the shorter the interval length of the indicator division interval of the associated load indicator under the preset task type, that is, the more divisions of the indicator division interval of the associated load indicator under the preset task type. However, the smaller the impact of a certain associated load indicator on the task processing performance under the preset task type, it means that it is sufficient to pay attention to the real-time changes of the associated load indicator, and the longer the interval length of the indicator division interval of the associated load indicator under the preset task type, that is, the fewer divisions of the indicator division interval of the associated load indicator under the preset task type.

[0036] According to the above method, for each preset task type, each associated load indicator will be divided into different indicator division intervals under the same preset task type. For each associated load indicator, the same associated load indicator will also be divided into different indicator division intervals under each preset task type.

[0037] Then, for each preset task type, the present application can obtain the specific values ​​of each associated load indicator of each service node in the distributed cluster in real time as the real-time dynamic value of the associated load indicator. Then, for each associated load indicator, by judging the real-time dynamic value of the associated load indicator of each service node, in which indicator division interval the associated load indicator is located under the preset task type, and by performing corresponding encoding processing on the specific sorting number after the indicator division interval is divided, the real-time dynamic encoding of the associated load indicator of each service node under the preset task type is obtained.

[0038] Therefore, according to the same method as above, the real-time dynamic encoding of each associated load indicator of each service node under each preset task type can be determined, thereby representing the real-time load conditions of each associated load indicator of the service node when processing each task under the preset task type.

[0039] S120: Determine a real-time load balancing parameter of the service node under the preset task type according to the real-time dynamic coding of each associated load indicator of each service node under the preset task type.

[0040] For each preset task type, the real-time dynamic coding of each associated load indicator of each service node in the distributed cluster under the preset task type can accurately represent the real-time load situation of each associated load indicator of the service node when processing each task under the preset task type. Then, in order to accurately analyze the load status of each service node when processing different types of tasks, the present application can perform linear fusion processing on the real-time dynamic coding of each associated load indicator of each service node under the preset task type under each preset task type, and obtain a real-time comprehensive coding of the service node under the preset task type. Then, by performing corresponding inverse coding processing on the real-time comprehensive coding of each service node under the preset task type, the real-time comprehensive load information of each service node under the preset task type can be obtained, which is used as the real-time load balancing parameter of the service node under the preset task type to accurately describe the real-time load status of the service node when processing each task under the preset task type.

[0041] Under each preset task type, usually the same method as above can be used to obtain the real-time load balancing parameters of each service node under each preset task type.

[0042] It is understandable that in order to ensure real-time load analysis of each service node under each preset task type, the present application can repeat S110 and S120 once at a short interval (for example, every 10s) under each preset task type, that is, for each preset task type, the real-time dynamic value of each associated load indicator of each service node in the distributed cluster is obtained once at a short interval, and the real-time load balancing parameters of each service node under the preset task type are regularly updated in combination with the indicator division interval of each associated load indicator under the preset task type, so as to dynamically maintain the real-time load balancing parameters of each service node under the preset task type, thereby eliminating the heterogeneity problems existing among each service node and each task in the distributed cluster, and measuring the real-time load status of each service node under multiple preset task types.

[0043] S130, when each task to be executed is received, determining a task allocation node for the task to be executed according to the real-time load balancing parameters of each service node under the target task type of the task to be executed.

[0044] By dynamically maintaining the real-time load balancing parameters of each service node under each preset task type, the real-time load status of each service node under multiple preset task types can be measured timely and effectively.

[0045] Therefore, each time a task to be executed is received, in order to ensure the efficient execution of the task to be executed in the distributed cluster, the present application will first determine the target task type of the task to be executed. Then, for each service node in the distributed cluster, the real-time load balancing parameters of each service node under each preset task type can be found from the real-time load balancing parameters of each service node under the target task type that are dynamically maintained, thereby indicating the real-time load status of each service node when processing the task to be executed.

[0046] Furthermore, by comparing the real-time load balancing parameters of each service node under the target task type, a service node that meets the load balancing requirements can be determined from the various service nodes in the distributed cluster as the task allocation node for the task to be executed. The task to be executed is subsequently executed through the task allocation node to ensure the efficient execution of the task to be executed.

[0047] In some possible implementations, in order to ensure the reasonable distribution of tasks within a distributed cluster, the present application can determine the task allocation node for the task to be executed in the following manner: determine the target service node with the smallest real-time load balancing parameter based on the real-time load balancing parameters of each service node under the target task type of the task to be executed; assign the task to be executed to the target service node as the task allocation node for the task to be executed.

[0048] Specifically, by comparing the real-time load balancing parameters of each service node in the distributed cluster under the target task type of the task to be executed, the minimum real-time load balancing parameter can be screened out to find the target service node with the minimum real-time load balancing parameter from each service node. It can be seen that the real-time load of the target service node at the current processing moment of the task to be executed is the smallest among all service nodes. Then, in order to ensure the load balancing of each service node in the distributed cluster during task allocation, the present application can allocate the task to be executed to the target service node and use it as the task allocation node of the task to be executed to ensure the efficient execution of the task to be executed and improve the task execution efficiency of the entire distributed cluster.

[0049] The technical solution provided by the embodiment of the present application, for each task in the distributed cluster, the present application will pre-configure multiple task types. For each preset task type, by analyzing the real-time dynamic value of each associated load index of each service node in the distributed cluster and the index division interval of each associated load index under the preset task type, the real-time dynamic encoding of each associated load index of each service node under the preset task type is determined, so as to dynamically maintain the real-time load balancing parameters of each service node under the preset task type. Then, when each task to be executed is received, the task allocation node of the task to be executed can be determined by analyzing the real-time load balancing parameters of each service node under the target task type of the task to be executed, thereby realizing the reasonable allocation of tasks in the distributed cluster, and by dynamically maintaining the real-time load balancing parameters of each service node under each preset task type, the heterogeneity problems existing in each service node and each task in the distributed cluster are eliminated, and the real-time load status of each service node is measured respectively under multiple preset task types, so as to ensure the load balancing of each service node in the distributed cluster after task allocation, and improve the task execution efficiency of the entire distributed cluster.

[0050] As an optional implementation scheme in this application, this application mainly dynamically maintains the real-time load balancing parameters of each service node in the distributed cluster under each preset task type to eliminate the heterogeneity problems existing in each service node and each task in the distributed cluster, and measures the real-time load status of each service node under multiple preset task types, thereby accurately and reasonably allocating suitable service nodes for each task to be executed received, ensuring the load balancing of each service node in the distributed cluster after task allocation, and improving the task execution efficiency of the entire distributed cluster. Therefore, in order to achieve reasonable task allocation within a distributed cluster, this application first needs to ensure that the real-time load balancing parameters of each service node in the distributed cluster under each preset task type can be accurately and dynamically maintained.

[0051] Next, the present application may provide a detailed explanation of the specific dynamic maintenance process of the real-time load balancing parameters of each service node under each preset task type.

[0052] Figure 2 The method flow chart of the specific dynamic maintenance process of the real-time load balancing parameters of each service node under each preset task type shown in the embodiment of the present application. Figure 2 As shown, the method may specifically include the following steps:

[0053] S210, for each preset task type, according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type, determine the real-time indicator interval of each associated load indicator of each service node under the preset task type.

[0054] Taking into account that the various associated load indicators of any service node have different degrees of influence on the processing performance of different types of tasks, and each service node can divide each associated load indicator into multiple different numerical intervals, the specific data value of each associated load indicator of each service node can be used to accurately analyze the real-time load status of each service node. Then, for each preset task type, the present application can divide each associated load indicator into multiple data intervals with different interval lengths under the preset task type by analyzing the different degrees of influence of each associated load indicator on the task processing performance under the preset task type, and obtain the indicator division interval of each associated load indicator under the preset task type. Thus, for each preset task type, each associated load indicator will be divided into different indicator division intervals under the same preset task type. For each associated load indicator, the same associated load indicator will also be divided into different indicator division intervals under each preset task type.

[0055] In this application, in order to accurately analyze the real-time load status of each service node, first, the specific values ​​of each associated load indicator of each service node in the distributed cluster can be obtained in real time as the real-time dynamic value of the associated load indicator. Then, for each preset task type, by judging the real-time dynamic value of each associated load indicator of each service node, and specifically in which indicator division interval each associated load indicator is under the preset task type, the real-time indicator interval of each associated load indicator of each service node under the preset task type is determined, thereby indicating the real-time load status of each associated load indicator of the service node under the preset task type.

[0056] In some implementable methods, in order to ensure accurate analysis of the real-time load status of each service node, it is first necessary to ensure accurate division of the indicator division intervals of each associated load indicator under each preset task type. And because each associated load indicator has different degrees of influence on the processing performance of different types of tasks. Therefore, for the indicator division intervals of each associated load indicator under each preset task type, the present application can be determined in the following manner: for each preset task type, according to the sensitivity coefficient of the preset task type for each associated load indicator, determine the division granularity of each associated load indicator under the preset task type; according to the division granularity and indicator variation range of each associated load indicator under the preset task type, determine the indicator division interval of each associated load indicator under the preset task type.

[0057] Specifically, under each preset task type, the sensitivity of the preset task type to each associated load indicator can be judged by analyzing the specific impact of each associated load indicator on the task processing performance under the preset task type, thereby determining the sensitivity coefficient of the preset task type for each associated load indicator.

[0058] It is understandable that if a task under a preset task type is more sensitive to a certain associated load indicator, it means that for the task under the preset task type, it is necessary to focus on refining the real-time changes of the associated load indicator, and it is necessary to divide the associated load indicator into more fine-grained intervals. If a task under a preset task type is not sensitive to a certain associated load indicator, it means that for the task under the preset task type, it is sufficient to pay attention to the real-time changes of the associated load indicator, and it is necessary to divide the associated load indicator into coarse-grained intervals.

[0059] Therefore, under each preset task type, by analyzing the sensitivity coefficient of the preset task type to each associated load indicator, positive correlation can be used to determine the division granularity of each associated load indicator under the preset task type, so that the higher the sensitivity coefficient of the preset task type to a certain associated load indicator, the finer the division granularity of the associated load indicator under the preset task type.

[0060] Then, by analyzing the allowable parameter variation range of each associated load indicator, the indicator variation range of the associated load indicator is determined. Furthermore, under each preset task type, the indicator variation range of the associated load indicator can be divided into intervals of corresponding degrees using the division granularity of each associated load indicator under the preset task type, so as to obtain multiple indicator division intervals of the associated load indicator under the preset task type.

[0061] According to the above method, for each preset task type, each associated load indicator will be divided into different indicator division intervals under the same preset task type. For each associated load indicator, the same associated load indicator will also be divided into different indicator division intervals under each preset task type.

[0062] For example, assuming that a task of a certain preset task type is a CPU-intensive task, it can be known that the CPU-intensive task is more sensitive to the CPU utilization in each associated load indicator, but is not sensitive to the memory utilization. Figure 3a As shown in , assuming that the CPU utilization index range is [0, 100], then under this preset task type (that is, CPU-intensive task), the CPU utilization, an associated load index, can be divided into a finer granularity (for example, 3 levels). Figure 3b As shown, assuming that the memory utilization index variation range is also [0, 100], then under the preset task type (that is, CPU intensive task), the memory utilization, an associated load index, can be divided into coarse-grained (for example, 2-level division).

[0063] S220 , determining a real-time dynamic code of each associated load indicator of each service node under the preset task type according to the interval number of the indicator interval in which each associated load indicator of each service node is located in real time under the preset task type.

[0064] In order to accurately distinguish the indicator division intervals of each associated load indicator under each preset task type, after determining the indicator division intervals of each associated load indicator under each preset task type, this application will also, under each preset task type, number the indicator division intervals of each associated load indicator after it is divided under the preset task type in order from top to bottom and from low to high in order of the division granularity level of each indicator division interval after it is divided under the preset task type, so as to obtain the interval number of each indicator division interval after each associated load indicator is divided under the preset task type. Figure 3a and 3b As shown, the interval numbers of each indicator division interval after each associated load indicator is divided under the preset task type are associated with the load status of the indicator division interval, by the smaller the interval number of a certain indicator division interval after each associated load indicator is divided under the preset task type, the smaller the load of the indicator division interval is, and the larger the interval number of a certain indicator division interval is, the larger the load of the indicator division interval is. This allows the real-time load status of each associated load indicator of each service node to be represented subsequently according to the interval number of the specific indicator division interval where the real-time dynamic value of each associated load indicator of the service node is located.

[0065] Moreover, since the interval numbers of each indicator division interval after each associated load indicator of each service node is divided under any preset task type cannot be fused and processed later to comprehensively represent the real-time load status of each service node. To this end, under each preset task type, for each indicator division interval after each associated load indicator of each service node is divided under the preset task type, the present application can convert the interval numbers of each indicator division interval after each associated load indicator of each service node is divided under the preset task type into corresponding binary codes to obtain the load codes corresponding to each indicator division interval of each associated load indicator of each service node under the preset task type, so as to subsequently fuse the load codes corresponding to each indicator division interval of each associated load indicator of each service node under the preset task type to comprehensively analyze the real-time load status of each service node under the preset task type.

[0066] In some implementable methods, in order to ensure the overall consistency of the load coding corresponding to each indicator division interval after each associated load indicator is divided under each preset task type, it is required that under each preset task type, the number of bits of the load coding corresponding to each indicator division interval of each associated load indicator under the preset task type (that is, the coding length) is the same. Therefore, under each preset task type, for each indicator division interval of each associated load indicator under the preset task type, the maximum interval number of each indicator division interval of each associated load indicator under the preset task type can be determined, and based on the use of binary coding, the maximum interval number can be solved logarithmically with base 2, and the solution result can be rounded up to an integer, which can be used as the number of bits of the load coding corresponding to each indicator division interval of the associated load indicator under the preset task type. Based on this, if Figure 3a and 3b As shown, according to the number of load coding bits corresponding to each indicator division interval of each associated load indicator under the preset task type, the interval number of each indicator division interval of the associated load indicator under the preset task type can be converted into a corresponding binary code to obtain the load code corresponding to each indicator division interval of the associated load indicator under the preset task type.

[0067] It can be seen that for each preset task type, after determining the real-time indicator interval of each associated load indicator of each service node under the preset task type according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type, the present application will first determine the interval number of the real-time indicator interval of each associated load indicator of each service node under the preset task type. Then, from the load code corresponding to each indicator division interval of each associated load indicator under the preset task type, the load code corresponding to the interval number can be found according to the interval number of the real-time indicator interval of each associated load indicator of each service node under the preset task type, as the real-time dynamic code of each associated load indicator of each service node under the preset task type.

[0068] S230, for each service node, linearize the real-time dynamic coding of each associated load indicator of the service node under the preset task type to obtain the real-time load balancing coding of the service node under the preset task type.

[0069] For each preset task type, after determining the real-time dynamic encoding of each associated load indicator of each service node under the preset task type, the present application can, for each service node, analyze the sensitivity of each associated load indicator of the service node under the preset task type, and perform linear fusion processing on the real-time dynamic encoding of each associated load indicator of the service node under the preset task type according to the sensitivity level, so as to obtain the real-time load balancing encoding of the service node under the preset task type, thereby representing the real-time load status of the service node after comprehensive analysis under the preset task type.

[0070] In some implementable embodiments, under each preset task type, the real-time load balancing coding of each service node under the preset task type can be determined in the following manner: according to the coding length of the real-time dynamic coding of each associated load indicator of each service node under the preset task type, the real-time dynamic coding of each associated load indicator of the service node under the preset task type is interleaved in descending order to obtain the real-time load balancing coding of the service node under the preset task type.

[0071] Specifically, under each preset task type, the longer the encoding length of a certain associated load indicator of each service node under the preset task type is, the more sensitive the associated load indicator is to the task under the preset task type. Therefore, when comprehensively analyzing the real-time load status of each service node, the weight of the real-time dynamic encoding of the associated load indicator under the preset task type needs to be greater.

[0072] Therefore, for each preset task type, by analyzing the coding length of the real-time dynamic coding of each associated load indicator of each service node under the preset task type, the real-time dynamic coding of each associated load indicator of the service node under the preset task type is arranged in descending order. Then, for each service node, the present application can perform corresponding first-place zero padding processing on the real-time dynamic coding of each associated load indicator of the service node under the preset task type, so that the number of bits of the real-time dynamic coding of each associated load indicator of the service node under the preset task type can appear proportionally. It can be understood that if the number of bits of the real-time dynamic coding of each associated load indicator of a service node under the preset task type is originally proportional, then there is no need to perform first-place zero padding processing on the real-time dynamic coding of each associated load indicator of the service node under the preset task type.

[0073] Furthermore, by performing corresponding coding interleaving and fusion processing on the final real-time dynamic coding of each associated load indicator of each service node after being arranged in descending order of the preset task type, the final real-time dynamic coding of each associated load indicator of each service node after being arranged in descending order of the preset task type is interwoven together in descending order to obtain the real-time load balancing coding of the service node under the preset task type.

[0074] According to the above method, the real-time load balancing code of each service node under each preset task type can be obtained.

[0075] For example, assume that the associated load indicators of a service node are CPU utilization, memory utilization, disk utilization, and bandwidth utilization, and the real-time dynamic codes of these four associated load indicators under a preset task type are 010011, 100, 110, and 101. Then, by interleaving the real-time dynamic codes of these four associated load indicators under a preset task type in descending order (that is, fusing 01, 1, 1, 1, 00, 0, 1, 0, 11, 0, 0, 1), a linearized coding result of 011110001011001 can be obtained as the real-time load balancing code of the service node under the preset task type.

[0076] S240: Determine a real-time load balancing parameter of each service node under the preset task type according to the real-time load balancing code of the service node under the preset task type.

[0077] Under each preset task type, after obtaining the real-time load balancing code of each service node under the preset task type, the present application can perform binary reverse encoding processing on the real-time load balancing code of each service node under the preset task type to obtain the corresponding decimal result as the real-time load balancing parameter of the service node under the preset task type, so as to dynamically analyze the real-time load status of each service node under each preset task type.

[0078] The technical solution provided by the embodiment of the present application is that for each task in a distributed cluster, the present application will pre-configure multiple task types. For each preset task type, by analyzing the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type, the real-time dynamic encoding of each associated load indicator of each service node under the preset task type is determined, so as to dynamically maintain the real-time load balancing parameters of each service node under the preset task type, to eliminate the heterogeneity problems existing in each service node and each task in the distributed cluster, measure the real-time load status of each service node respectively under multiple preset task types, and ensure the load balancing of each service node in the distributed cluster after task allocation.

[0079] Figure 4 This is a principle block diagram of a task allocation device shown in an embodiment of the present application. Figure 4 As shown, the apparatus 400 may include:

[0080] The dynamic coding determination module 410 is used to determine, for each preset task type, the real-time dynamic coding of each associated load indicator of each service node under the preset task type according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type;

[0081] The node load balancing module 420 is used to determine the real-time load balancing parameters of the service node under the preset task type according to the real-time dynamic encoding of each associated load indicator of each service node under the preset task type;

[0082] The task allocation module 430 is used to determine the task allocation node of the task to be executed according to the real-time load balancing parameters of each service node under the target task type of the task to be executed each time a task to be executed is received.

[0083] In some implementations, the dynamic code determination module 410 may be specifically used to:

[0084] Determine the real-time indicator range of each associated load indicator of each service node under the preset task type according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division range of each associated load indicator under the preset task type;

[0085] According to the interval number of the indicator interval in which each associated load indicator of each service node is located in real time under the preset task type, the real-time dynamic coding of each associated load indicator of each service node under the preset task type is determined.

[0086] In some implementations, the task allocation device 400 may further include a load index partitioning module. The load index partitioning module may be used to:

[0087] For each preset task type, according to the sensitivity coefficient of the preset task type to each associated load indicator, determine the division granularity of each associated load indicator under the preset task type;

[0088] According to the division granularity and indicator variation range of each associated load indicator under the preset task type, the indicator division interval of each associated load indicator under the preset task type is determined.

[0089] In some implementations, the node load balancing module 420 may include:

[0090] A load balancing code determination unit, configured to perform linearization processing on the real-time dynamic code of each associated load indicator of the service node under the preset task type for each service node, so as to obtain the real-time load balancing code of the service node under the preset task type;

[0091] The load balancing parameter determination unit is used to determine the real-time load balancing parameter of each service node under the preset task type according to the real-time load balancing code of the service node under the preset task type.

[0092] In some implementations, the load balancing code determination unit may be specifically configured to:

[0093] According to the coding length of the real-time dynamic coding of each associated load indicator of each service node under the preset task type, the real-time dynamic coding of each associated load indicator of the service node under the preset task type is interleaved in descending order to obtain the real-time load balancing coding of the service node under the preset task type.

[0094] In some implementations, the task allocation module 430 may be specifically configured to:

[0095] Determine, according to the real-time load balancing parameter of each service node under the target task type of the task to be executed, the target service node with the minimum real-time load balancing parameter;

[0096] The task to be executed is assigned to the target service node as the task assignment node of the task to be executed.

[0097] In the embodiment of the present application, for each task in the distributed cluster, the present application will pre-configure multiple task types. For each preset task type, by analyzing the real-time dynamic value of each associated load index of each service node in the distributed cluster and the index division interval of each associated load index under the preset task type, the real-time dynamic encoding of each associated load index of each service node under the preset task type is determined, so as to dynamically maintain the real-time load balancing parameters of each service node under the preset task type. Then, when each task to be executed is received, the task allocation node of the task to be executed can be determined by analyzing the real-time load balancing parameters of each service node under the target task type of the task to be executed, thereby realizing the reasonable allocation of tasks in the distributed cluster, and by dynamically maintaining the real-time load balancing parameters of each service node under each preset task type, the heterogeneity problems existing in each service node and each task in the distributed cluster are eliminated, and the real-time load status of each service node is measured respectively under multiple preset task types, so as to ensure the load balancing of each service node in the distributed cluster after task allocation, and improve the task execution efficiency of the entire distributed cluster.

[0098] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 4 The device 400 shown can execute any method embodiment in the present application, and the aforementioned and other operations and / or functions of each module in the device 400 are respectively for implementing the corresponding processes in each method in the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0099] The above describes the device 400 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0100] Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present application.

[0101] like Figure 5 As shown, the electronic device 500 may include:

[0102] The memory 510 and the processor 520, the memory 510 is used to store the computer program and transmit the program code to the processor 520. In other words, the processor 520 can call and run the computer program from the memory 510 to implement the method in the embodiment of the present application.

[0103] For example, the processor 520 may be configured to execute the above method embodiments according to instructions in the computer program.

[0104] In some embodiments of the present application, the processor 520 may include but is not limited to:

[0105] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0106] In some embodiments of the present application, the memory 510 includes but is not limited to:

[0107] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0108] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 510 and executed by the processor 520 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0109] like Figure 5 As shown, the electronic device may also include:

[0110] The transceiver 530 may be connected to the processor 520 or the memory 510 .

[0111] The processor 520 may control the transceiver 530 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of antennas may be one or more.

[0112] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0113] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0114] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0115] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0116] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0117] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0118] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for task allocation, characterized in that: include: For each preset task type, according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type, determine the real-time dynamic coding of each associated load indicator of each service node under the preset task type; Determine the real-time load balancing parameter of the service node under the preset task type according to the real-time dynamic encoding of each associated load indicator of each service node under the preset task type; When each task to be executed is received, a task allocation node for the task to be executed is determined according to the real-time load balancing parameters of each service node under the target task type of the task to be executed.

2. The method according to claim 1, characterized in that The step of determining the real-time dynamic encoding of each associated load indicator of each service node under the preset task type according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type includes: Determine the real-time indicator range of each associated load indicator of each service node under the preset task type according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division range of each associated load indicator under the preset task type; According to the interval number of the indicator interval in which each associated load indicator of each service node is located in real time under the preset task type, the real-time dynamic coding of each associated load indicator of each service node under the preset task type is determined.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: For each preset task type, according to the sensitivity coefficient of the preset task type to each associated load indicator, determine the division granularity of each associated load indicator under the preset task type; According to the division granularity and indicator variation range of each associated load indicator under the preset task type, the indicator division interval of each associated load indicator under the preset task type is determined.

4. The method according to claim 1, characterized in that: The step of determining the real-time load balancing parameter of the service node under the preset task type according to the real-time dynamic encoding of each associated load indicator of each service node under the preset task type includes: For each service node, linearize the real-time dynamic coding of each associated load indicator of the service node under the preset task type to obtain the real-time load balancing coding of the service node under the preset task type; According to the real-time load balancing code of each service node under the preset task type, the real-time load balancing parameter of the service node under the preset task type is determined.

5. The method according to claim 4, characterized in that The linearizing of the real-time dynamic coding of each associated load indicator of the service node under the preset task type to obtain the real-time load balancing coding of the service node under the preset task type includes: According to the coding length of the real-time dynamic coding of each associated load indicator of each service node under the preset task type, the real-time dynamic coding of each associated load indicator of the service node under the preset task type is interleaved in descending order to obtain the real-time load balancing coding of the service node under the preset task type.

6. The method according to claim 1, characterized in that The determining, according to the real-time load balancing parameter of each service node under the target task type of the task to be executed, the task allocation node of the task to be executed comprises: Determine, according to the real-time load balancing parameter of each service node under the target task type of the task to be executed, the target service node with the minimum real-time load balancing parameter; The task to be executed is assigned to the target service node as the task assignment node of the task to be executed.

7. A task allocation device, characterized in that: include: A dynamic coding determination module is used to determine, for each preset task type, the real-time dynamic coding of each associated load indicator of each service node under the preset task type according to the real-time dynamic value of each associated load indicator of each service node in the distributed cluster and the indicator division interval of each associated load indicator under the preset task type; A node load balancing module, configured to determine a real-time load balancing parameter of the service node under the preset task type according to the real-time dynamic encoding of each associated load indicator of each service node under the preset task type; The task allocation module is used to determine the task allocation node of the task to be executed according to the real-time load balancing parameters of each service node under the target task type of the task to be executed each time a task to be executed is received.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the task allocation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the method for task allocation according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for task allocation as described in any one of claims 1 to 6 is implemented.

Images