Task allocation method and device, medium and program product

Abstract

The invention discloses a task allocation method and device, a medium and a program product in the technical field of computers. For one task, the task type of the current task can be judged in real time according to the historical tasks in the system, and the target node is selected for the current task in combination with the actual resource use condition of each physical node in the system, so that reasonable scheduling of the tasks is realized; the scheduling efficiency and the resource adaptability of multiple types of tasks in the same system are improved, the problems of low resource scheduling efficiency, serious resource fragmentation and the like can be solved to a certain extent, and the method can adapt to scenes with high-performance calculation, low delay and high dynamic expansion requirements.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task allocation method, device, medium, and program product. Background Art

[0002] At present, different types of tasks may need to be executed in the same system. Since the processing efficiency, bandwidth, storage and other types of resources required by each type of task are different, the scheduling difficulty of different types of tasks in the same system increases, and problems such as low resource scheduling efficiency and severe resource fragmentation are prone to occur. It is difficult to adapt to scenarios requiring high-performance computing, low latency and high dynamic expansion.

[0003] Therefore, how to improve the scheduling efficiency and resource adaptability of multiple types of tasks in the same system is a problem that those skilled in the art need to solve. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a task allocation method, device, medium and program product to improve the scheduling efficiency and resource adaptability of multiple types of tasks in the same system.

[0005] In a first aspect, the present application provides a task allocation method, which is applied to a scheduling node in a target processing system, comprising:

[0006] If the target task is obtained, the resource requirement thresholds corresponding to the preset multiple task types are calculated based on the historical task information between any historical moment and the current moment;

[0007] Using the resource requirement threshold to determine the target task type to which the target task belongs;

[0008] Performing weighted processing on the usage information of the target resources respectively included in each physical node in the target processing system according to the type weight corresponding to the target task type, thereby obtaining the weighted usage information of the target resources respectively included in each physical node;

[0009] Based on the relationship between the usage weighted information and the remaining information of the target resource in each physical node, the allocation fit corresponding to each physical node is determined, and the target task is processed in the physical node with the highest allocation fit.

[0010] In a second aspect, the present application provides a task allocation device, which is applied to a scheduling node in a target processing system, comprising:

[0011] The threshold calculation module is used to calculate the resource requirement thresholds corresponding to multiple preset task types based on the historical task information between any historical moment and the current moment if the target task is obtained;

[0012] A type determination module is used to determine the target task type to which the target task belongs by using a resource requirement threshold;

[0013] A type weighting module is used to perform weighted processing on the usage information of the target resources respectively included in each physical node in the target processing system according to the type weight corresponding to the target task type, so as to obtain the usage weighted information of the target resources respectively included in each physical node;

[0014] The task allocation module is used to determine the allocation fit corresponding to each physical node based on the relationship between the usage weighted information and the remaining information of the target resource in each physical node, and process the target task in the physical node with the highest allocation fit.

[0015] In a third aspect, the present application provides an electronic device, comprising:

[0016] memory for storing computer programs;

[0017] The processor is used to execute the computer program to implement the task allocation method disclosed above.

[0018] In a fourth aspect, the present application provides a non-volatile storage medium for storing a computer program, wherein the computer program implements the aforementioned disclosed task allocation method when executed by a processor.

[0019] In a fifth aspect, the present application provides a computer program product, comprising a computer program / instruction, which implements the steps of the aforementioned disclosed task allocation method when executed by a processor.

[0020] It can be seen from the above scheme that the present application provides a task allocation method, which is applied to the scheduling node in the target processing system, including: if the target task is obtained, then based on the historical task information between any historical moment and the current moment, the resource demand thresholds corresponding to the preset multiple task types are calculated; the target task type to which the target task belongs is determined using the resource demand threshold; according to the type weight corresponding to the target task type, the usage information of the target resources respectively included in each physical node in the target processing system is weighted to obtain the usage weighted information of the target resources respectively included in each physical node; according to the relationship between the usage weighted information and the remaining information of the target resources in each physical node, the allocation fit corresponding to each physical node is determined, and the target task is processed in the physical node with the largest allocation fit.

[0021] It can be seen that the beneficial effects of the present application are: based on the historical task information between any historical moment and the current moment, the target task type to which the current target task belongs is determined, and then based on the type weight corresponding to the target task type, the usage information of the target resources respectively included in each physical node in the target processing system is weighted, and based on the relationship between the usage information after weighted processing and the remaining information of the target resources in each physical node, the allocation fit corresponding to each physical node is determined, so that the target task is processed in the physical node with the largest allocation fit. For a task, this solution can determine the task type of the current task in real time based on the historical tasks in the system, and select the destination node for the current task based on the actual resource usage of each physical node in the system, thereby realizing the reasonable scheduling of tasks, improving the scheduling efficiency and resource adaptability of multiple types of tasks in the same system, and solving the problems of low resource scheduling efficiency and serious resource fragmentation to a certain extent, and can adapt to scenarios with high-performance computing, low latency and high dynamic expansion requirements.

[0022] Correspondingly, the task allocation device, equipment, medium and program product provided by this application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0024] Figure 1 This is a flow chart of the first task allocation method disclosed in this application; Figure 2 This is a flow chart of the second task allocation method disclosed in this application; Figure 3 A system framework and functional diagram disclosed in this application; Figure 4 This is a schematic diagram of the first task allocation device disclosed in this application; Figure 5 This is a schematic diagram of the second task allocation device disclosed in this application; Figure 6 A server structure diagram provided for this application; Figure 7 This is a terminal structure diagram provided for this application. DETAILED DESCRIPTION

[0025] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0027] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0028] Currently, different types of tasks may need to be executed in the same system. Because each type of task requires different amounts of resources, such as processing efficiency, bandwidth, and storage, scheduling different types of tasks in the same system becomes more difficult, leading to problems such as low resource scheduling efficiency and severe resource fragmentation. This makes it difficult to adapt to scenarios requiring high-performance computing, low latency, and high dynamic expansion. To this end, this application provides a task allocation solution that can improve the scheduling efficiency and resource adaptability of multiple types of tasks in the same system.

[0029] See also Figure 1 As shown, the embodiment of the present application discloses a first task allocation method, which is applied to a scheduling node in a target processing system, including: S101: If a target task is obtained, resource requirement thresholds corresponding to various preset task types are calculated based on historical task information between any historical moment and the current moment.

[0030] In this embodiment, the preset multiple task types may include: storage intensive (high storage performance requirements), computing intensive (high computing performance requirements), memory intensive (high memory performance requirements), bandwidth intensive (high bandwidth performance requirements) and ordinary (low performance requirements in terms of storage, computing, memory bandwidth, etc.). Computational intensive tasks usually require a large amount of CPU resources to perform computing operations, and compute-intensive tasks are modeled as high demands on CPU resources. Memory-intensive tasks involve a large amount of data operations and storage, so sufficient memory resources are required, and memory-intensive tasks are modeled as high demands on memory resources; network bandwidth intensive tasks require a large amount of network communication. The scheduler will consider the network bandwidth status of the physical node to ensure that network bandwidth intensive tasks are scheduled to nodes with sufficient network bandwidth; IO intensive tasks involve frequent read and write operations on the storage system, so sufficient disk IO resources are required. IO intensive tasks are modeled as high demands on disk IO resources.

[0031] Among them, each task type can correspond to at least one resource requirement threshold. For example: for storage-intensive task types with high storage performance requirements, a resource requirement threshold A corresponding to disk resources can be set. When the amount of disk resources required for a task does not exceed resource requirement threshold A, the task can be classified as a storage-intensive task type. Correspondingly, if each task type corresponds to two resource requirement thresholds, there can be the following example: for storage-intensive task types with high storage performance requirements, a resource requirement threshold A corresponding to disk resources (i.e., storage resources) and a resource requirement threshold B corresponding to processor resources (i.e., computing resources) are set. When the amount of disk resources required for a task does not exceed resource requirement threshold A, and the amount of processor resources required for the task does not exceed resource requirement threshold B, the task is classified as a storage-intensive task type. Among them, a resource type required for a task can correspond to a resource demand threshold. For example, the resource types required for a task include: processor resources, memory resources, bandwidth resources and / or disk resources; then the resource demand threshold corresponding to each task type may include: processor resource demand threshold, memory resource demand threshold, bandwidth resource demand threshold and / or disk resource demand threshold.

[0032] It should also be noted that the detection order of multiple task types can be set, and after a task determines the specific task type, the detection of other task types will no longer continue. For example: set the detection order of multiple task types to: storage intensive, computing intensive, memory intensive, bandwidth intensive and ordinary type, then for a certain task, first detect whether the task is storage intensive, if it is storage intensive, then end the task type detection step; if it is not storage intensive, then further detect whether the task is computing intensive, if it is computing intensive, then end the task type detection step; if it is not computing intensive, then further detect whether the task is memory intensive, if it is memory intensive, then end the task type detection step; if it is not memory intensive, then further detect whether the task is bandwidth intensive, if it is bandwidth intensive, then end the task type detection step; if it is not bandwidth intensive, then determine that the task is ordinary type.

[0033] In this embodiment, the process of determining the corresponding resource demand threshold for each task type can refer to the following: based on the historical task information between any historical moment and the current moment, the resource demand thresholds corresponding to the preset multiple task types are calculated, including: for any task type, based on the historical task information belonging to the current task type between any historical moment and the current moment, the first parameter corresponding to the current task type is determined; the historical task information may include multiple historical tasks, the total number of these historical tasks, and the actual resource usage information of each historical task; the actual resource usage information of each historical task is: the usage or usage rate of processor resources, memory resources, bandwidth resources and / or disk resources used during the actual operation of each historical task; based on the preset adjustment coefficient, the average of the usage information of the target resource in each physical node at the current moment, the average of the historical usage information of the target resource in each physical node, and the total number of resource categories in the target resource, the second parameter corresponding to the current task type is calculated; based on the first parameter and the second parameter, the resource demand threshold corresponding to the current task type is calculated.

[0034] Determining the first parameter corresponding to the current task type based on historical task information belonging to the current task type between any historical moment and the current moment includes: obtaining multiple historical tasks from the historical task information belonging to the current task type; calculating the first parameter corresponding to the current task type based on the total number of tasks of the multiple historical tasks and the actual resource usage information of each historical task. Correspondingly, calculating the first parameter corresponding to the current task type based on the total number of tasks of the multiple historical tasks and the actual resource usage information of each historical task includes: calculating the first parameter corresponding to the current task type according to the first parameter calculation formula; wherein the first parameter calculation formula is: ;BaseThreshold type is the first parameter corresponding to the current task type; Q is the total number of tasks, The actual resource usage information of the Zth task among multiple historical tasks, Z=1,2,3…Q.

[0035] In one embodiment, the calculation formula for the average value of the usage information of the target resource in each physical node at the current moment is: ; R j (t) is the average usage information (which can be usage amount or usage rate) of resource j in n physical nodes at the current time t; x i,j The usage information of resource j in any physical node i among n physical nodes is represented by n, where n is the total number of physical nodes in the target processing system, i = 1, 2, 3, ..., n; resource j is any type of resource in the target resources, j = 1, 2, 3, ..., m, where m is the total number of resource types in the target resources. Using this formula, you can calculate the mean usage information of any type of resource in n physical nodes. For example, the mean memory usage of memory resources in n physical nodes.

[0036] In one example, the target resources include: processor resources, memory resources, bandwidth resources and / or disk resources; accordingly, according to a preset adjustment coefficient, an average of the usage information of the target resources in each physical node at the current moment, an average of the historical usage information of the target resources in each physical node, and the total number of resource categories in the target resources, a second parameter corresponding to the current task type is calculated, including: calculating the second parameter corresponding to the current task type according to a second parameter calculation formula; wherein the second parameter calculation formula is: ; AdjustmentFactor(t) is the second parameter, α is the preset adjustment coefficient, R cpu (t) is the mean usage information of processor resources in each physical node at the current time t, R mem (t) is the average usage information of memory resources in each physical node at the current time t, R net (t) is the average usage information of bandwidth resources in each physical node at the current time t, R io (t) is the mean usage information of disk resources in each physical node at the current time t, R avg (t) is the mean value of historical usage information, and 4 is the total number of resource categories in the target resource. cpu (t), R mem (t), R net (t), R io (t) Refer to the above R jIf the target resources include a total of m types of resources, then the second parameter calculation formula is: ; R1(t), R2(t)…Rm(t) are the means of the usage information corresponding to the m types of resources at the current time t.

[0037] In one embodiment, the resource requirement threshold corresponding to the current task type is calculated based on the first parameter and the second parameter, including: adding the second parameter to the set value, and multiplying the result of the addition by the first parameter as the resource requirement threshold corresponding to the current task type. This process is expressed as follows: .

[0038] S102: Determine the target task type to which the target task belongs by using the resource requirement threshold.

[0039] In one embodiment, a resource requirement threshold is used to determine the target task type to which the target task belongs, including: determining the resource requirement of the target task; if the value corresponding to the resource requirement threshold in the resource requirement exceeds the resource requirement threshold, then determining that the target task type belongs to the task type corresponding to the resource requirement threshold. If each task type corresponds to two resource requirement thresholds, then there can be the following example: for storage-intensive task types with high storage performance requirements, set a resource requirement threshold A corresponding to disk resources (i.e., storage resources) and a resource requirement threshold B corresponding to processor resources (i.e., computing resources). When the amount of disk resources required for a task does not exceed the resource requirement threshold A, and the amount of processor resources required for the task does not exceed the resource requirement threshold B, then the task is classified as a storage-intensive task type, that is, the target task type to which the target task belongs is: storage intensive.

[0040] S103 . Perform weighted processing on usage information of target resources respectively included in each physical node in the target processing system according to the type weight corresponding to the target task type, to obtain weighted usage information of the target resources respectively included in the physical node.

[0041] It should be noted that the type weight corresponding to each task type includes: the weight value set for each type of resource. In one example, the type weight corresponding to various task types is: the type weight corresponding to the storage-intensive task type 0.1, 0.1, 0.1, and 0.7 are the weights corresponding to processor resources, memory resources, bandwidth resources, and disk resources, respectively. The weight of 0.7 for disk resources is the largest, indicating that this type of task focuses on data storage performance. Weights corresponding to compute-intensive tasks (Focus on computing performance.) Type weights corresponding to memory-intensive task types (Emphasis on memory performance.) Bandwidth-intensive task type corresponding type weight (Focus on network performance.) Type weights corresponding to common task types .

[0042] In one embodiment, based on the type weight corresponding to the target task type, weighted processing is performed on the usage information of the target resources respectively included in each physical node in the target processing system to obtain the weighted usage information of the target resources respectively included in each physical node, including: calculating the product of the type weight and the usage information to obtain the weighted usage information. For example, if the target resource is a bandwidth resource, then n physical nodes each include one piece of bandwidth usage information, for a total of n bandwidth usage information, and the type weight corresponding to the target task type includes: a weight value set for m types of resources, based on which n pieces of bandwidth usage information are multiplied by the weight value of the bandwidth resource to obtain the weighted usage information of the bandwidth resource. If the target resource includes multiple types of resources, then when calculating the product of the type weight and the usage information, the multiplication is performed according to the corresponding resource category.

[0043] The process of collecting usage information of target resources respectively included in each physical node in the target processing system includes: for each physical node, collecting the usage of the processor, memory, bandwidth and / or disk in the corresponding physical node as usage information; and standardizing the usage information.

[0044] In one embodiment, before determining the allocation fit corresponding to each physical node based on the relationship between usage weighted information and the remaining information of the target resource in each physical node, the method further includes: determining usage fluctuation information corresponding to the target resource based on the usage information; performing a secondary weighting process on the usage weighted information according to the usage fluctuation information; and using the usage weighted information after the secondary weighting process as the usage weighted information. The secondary weighting process is similar to the principle of the previous weighting process using the usage type weight, both of which add weight values ​​to the usage information. The method further includes calculating the ratio of the standardized usage information to the total usage information of the target resource in the target processing system; calculating the fluctuation coefficient of the target resource in each physical node based on the ratio and the total number of physical nodes in the target processing system; and using the fluctuation coefficient as the usage fluctuation information. If the usage or utilization rate of the processor resource is collected as the usage information of the target resource, then the ratio of the standardized usage information to the total usage information of the target resource in the target processing system is the ratio of the usage or utilization rate of the processor resource to the total usage or utilization rate of the processor resource in the entire system.

[0045] S104: Determine the allocation fit corresponding to each physical node according to the relationship between the usage weight information and the remaining information of the target resource in each physical node, and process the target task in the physical node with the highest allocation fit.

[0046] In one embodiment, the remaining information of the target resource in each physical node includes: the maximum remaining information and the minimum remaining information of the target resource in each physical node; accordingly, based on the relationship between the usage weighted information and the remaining information of the target resource in each physical node, the allocation fit corresponding to each physical node is determined, including: for each physical node, calculating the first similarity between the usage weighted information in the corresponding physical node and the maximum remaining information, and the second similarity between the usage weighted information in the corresponding physical node and the minimum remaining information; based on the first similarity and the second similarity, the allocation fit corresponding to each physical node is calculated.

[0047] In one embodiment, the first similarity between the usage weighted information and the maximum remaining information in the corresponding physical node, and the second similarity between the usage weighted information and the minimum remaining information in the corresponding physical node are calculated, including: constructing the usage weighted information in the current physical node as a target vector; constructing the maximum remaining information of the target resource in each physical node as a first vector; calculating the first distance between the target vector and the first vector, and using the first distance as the first similarity; constructing the minimum remaining information of the target resource in each physical node as a second vector; calculating the second distance between the target vector and the second vector, and using the second distance as the second similarity. According to the first similarity and the second similarity, the distribution fit corresponding to each physical node is calculated, including: for each physical node, calculating the sum of the first similarity and the second similarity corresponding to the corresponding physical node; calculating the ratio of the first similarity corresponding to the corresponding physical node to the sum; and using the ratio as the distribution fit corresponding to the corresponding physical node.

[0048] In one embodiment, processing the target task in the physical node with the greatest allocation fit includes: determining the resource requirements based on the task information of the target task; creating a container or virtual machine in the physical node with the greatest allocation fit as the target virtual node according to the resource requirements; and using the target virtual node to process the target task. If the load of any physical node in the target processing system is higher than the preset load upper limit, some of the tasks in the physical node are migrated to other physical nodes in the target processing system. Migrating some of the tasks in the physical node to other physical nodes in the target processing system includes: taking some of the tasks to be migrated in the physical node as the target task, and executing the step of obtaining the target task and other subsequent steps.

[0049] It should be noted that this embodiment can also correct the allocation fit corresponding to each physical node based on the node load and the overall resource usage of the system. In one embodiment, the node penalty factor of each physical node at the current moment is calculated based on the resource utilization of each physical node; the dynamic adjustment coefficient of each physical node at the current moment is calculated based on the resource utilization of the target resource in the target processing system; the node penalty factor and the dynamic adjustment coefficient are used to correct the allocation fit corresponding to each physical node; and the maximum allocation fit is determined from the corrected allocation fit. Among them, the node penalty factor can measure the node load at the current moment, and the dynamic adjustment coefficient can measure the overall resource usage of the system.

[0050] In one embodiment, the node penalty factor of each physical node at the current moment is calculated based on the resource utilization of each physical node, including: calculating the node penalty factor of each physical node at the current moment according to the penalty factor calculation formula; wherein the penalty factor calculation formula is: ;P i is the node penalty factor of physical node i, i=1,2,3…n, n is the total number of physical nodes in the target processing system; λ is the penalty coefficient; U avg is the average resource utilization of physical node i between any historical moment and the current moment; R i is the resource utilization of physical node i.

[0051] In one embodiment, the dynamic adjustment coefficient of each physical node at the current moment is calculated based on the resource utilization of the target resource in the target processing system, including: calculating the dynamic adjustment coefficient at the current moment according to the adjustment coefficient calculation formula; wherein the adjustment coefficient calculation formula is: ; is the dynamic adjustment coefficient of physical node i at the current time t, β is the preset adjustment factor, U j (t) is the resource utilization rate of resource j in the target processing system at the current time t, resource j is any type of resource in the target resources, j = 1, 2, 3…m, m is the total number of resource categories in the target resources.

[0052] In one embodiment, the node penalty factor and the dynamic adjustment coefficient are used to correct the distribution fit of each physical node, including: multiplying the inverse of the node penalty factor, the dynamic adjustment coefficient, and the distribution fit of the same physical node as the corrected distribution fit of the corresponding physical node. This process is expressed as: The corrected distribution fit S corresponding to the physical node i i " =S i ×1 / P i × , S i is the original allocation fit corresponding to physical node i.

[0053] It should be noted that the allocation fit before and after correction can be used to determine the destination node. At least the corrected allocation fit takes into account the node load at the current moment and the overall resource usage of the system more comprehensively, and the solution considerations are more comprehensive.

[0054] It can be seen that in this embodiment, based on the historical task information between any historical moment and the current moment, the target task type to which the current target task belongs is determined, and then based on the type weight corresponding to the target task type, the usage information of the target resources respectively included in each physical node in the target processing system is weighted, and based on the relationship between the usage information after weighted processing and the remaining information of the target resources in each physical node, the allocation fit corresponding to each physical node is determined, so that the target task is processed in the physical node with the largest allocation fit. For a task, this solution can determine the task type of the current task in real time based on the historical tasks in the system, and select the destination node for the current task based on the actual resource usage of each physical node in the system, thereby realizing the reasonable scheduling of tasks, improving the scheduling efficiency and resource adaptability of multiple types of tasks in the same system, and solving the problems of low resource scheduling efficiency and severe resource fragmentation to a certain extent, and can adapt to scenarios with high-performance computing, low latency and high dynamic expansion requirements.

[0055] It should be noted that the target processing system includes n physical nodes, each of which includes: processor resources, memory resources, bandwidth resources and / or disk resources; the same type of resources in all physical nodes can be virtualized into a resource pool for subsequent system calls. For a target task that needs to be processed, a container or virtual machine to run the task is created in the system based on the resource requirements of the task, and the destination node for the container or virtual machine deployment is determined. The destination node is the node that processes the target task. The scheduling node in the target processing system can be any physical node in the target processing system, or it can be a Figure 3 The resource scheduler shown may be a software program or a hardware device that combines software and hardware.

[0056] The following is a process for determining the destination node, including: collecting the usage of each type of resource in all physical nodes in the system, constructing a matrix X, and standardizing each element in the matrix X; calculating the ratio of each type of resource in each physical node to the total amount of resources of that type in the system; using the ratio to calculate the fluctuation coefficient of each type of resource. The larger the fluctuation coefficient, the greater the difference in the usage of this type of resource on each node, and the smaller its impact on the decision; calculating the usage weight of each type of resource based on the fluctuation coefficient, multiplying the usage weight by the matrix X to obtain the first weighted matrix Z; then performing secondary weighting, that is, using the task type of the current task The corresponding elements in matrix Z are reweighted using the type weights corresponding to the type. The distance between each physical node's resource usage and the corresponding maximum remaining amount is calculated, as well as the distance between each physical node's resource usage and the corresponding minimum remaining amount. The node penalty factor and dynamic adjustment coefficient are calculated for each physical node. The modified allocation fit is calculated for each physical node based on these two distance values, the node penalty factor, and the dynamic adjustment coefficient. The physical node with the highest allocation fit is selected as the destination node, the container or virtual machine is deployed on the destination node, and the current task is run in the container or virtual machine. In this implementation, the two weighted processes are performed in any order.

[0057] See also Figure 2 As shown, the embodiment of the present application discloses a second task allocation method, which is applied to a scheduling node in a target processing system, including: S201: If a target task is obtained, usage information of target resources respectively included in each physical node in the target processing system is collected.

[0058] In this embodiment, the target processing system includes n physical nodes, each of which contains processor resources, memory resources, bandwidth resources, and / or disk resources. Resources of the same type across all physical nodes can be virtualized into a resource pool for subsequent system calls. The target resources include processor resources (i.e., computing resources), memory resources, bandwidth resources, and / or disk resources (i.e., storage resources).

[0059] In one example, processor resources, memory resources, bandwidth resources, and disk resources are collected from each physical node in the target processing system as usage information of the target resources. That is, the target resources include at least one category, and may also include multiple categories. The total number of resource categories in the target resources is represented by m, and any one category of resources is represented by j, where j=1, 2, 3…m. In one embodiment, the target resources include: processor resources, memory resources, bandwidth resources, and / or disk resources; accordingly, the usage information of the target resources respectively included in each physical node in the target processing system is collected, including: for each physical node, the usage of the processor, memory, bandwidth, and / or disk in the corresponding physical node is collected as usage information.

[0060] S202: Determine usage fluctuation information corresponding to the target resource according to the usage information.

[0061] In one embodiment, determining usage fluctuation information corresponding to a target resource based on usage information includes: normalizing the usage information, calculating the ratio of the normalized usage information to the total usage information of the target resource in the target processing system; calculating the fluctuation coefficient of the target resource in each physical node based on the ratio and the total number of physical nodes in the target processing system; and using the fluctuation coefficient as the usage fluctuation information. If the usage amount or utilization rate of processor resources is collected as the usage information of the target resource, then the ratio of the normalized usage information to the total usage information of the target resource in the target processing system is the ratio of the usage amount or utilization rate of the processor resources to the total usage amount or utilization rate of the processor resources in the entire system.

[0062] In one embodiment, calculating the fluctuation coefficient of the target resource in each physical node based on the ratio and the total number of physical nodes in the target processing system includes: calculating the fluctuation coefficient of the target resource in each physical node according to a first formula; wherein the first formula is: ;e j is the fluctuation coefficient of resource j in each physical node, resource j is any type of resource in the target resources, j=1,2,3…m, m is the total number of resource types in the target resources; k=1 / ln(n), n is the total number of physical nodes in the target processing system; P ij is the usage information of resource j after normalization in physical node i, and its proportion to the total usage information of resource j in the target processing system. Physical node i is any one of the n physical nodes in the target processing system, i=1,2,3…n.

[0063] S203 : Perform weighted processing on the usage information according to the usage fluctuation information to obtain usage weighted information of the target resources included in each physical node.

[0064] In one embodiment, weighting the usage information according to the usage fluctuation information to obtain the usage weight information of the target resource included in each physical node includes: determining the usage weight corresponding to the target resource according to the usage fluctuation information; and calculating the product of the usage weight and the usage information to obtain the usage weight information. Determining the usage weight of the target resource according to the usage fluctuation information includes: calculating the usage weight of the target resource according to the usage fluctuation information according to a second formula; wherein the second formula is: ;W j is the usage weight corresponding to resource j; e j is the fluctuation coefficient of resource j in each physical node, resource j is any type of resource in the target resources, j=1,2,3…m, m is the total number of resource categories in the target resources.

[0065] S204 : Determine the allocation fit corresponding to each physical node based on the relationship between the usage weight information and the remaining information of the target resource in each physical node, and process the target task in the physical node with the highest allocation fit.

[0066] In one embodiment, the remaining information of the target resource in each physical node includes: the maximum remaining information and the minimum remaining information of the target resource in each physical node; accordingly, based on the relationship between the usage weighted information and the remaining information of the target resource in each physical node, the allocation fit corresponding to each physical node is determined, including: for each physical node, calculating the first similarity between the usage weighted information in the corresponding physical node and the maximum remaining information, and the second similarity between the usage weighted information in the corresponding physical node and the minimum remaining information; based on the first similarity and the second similarity, the allocation fit corresponding to each physical node is calculated.

[0067] In one embodiment, a first similarity between usage-weighted information and maximum remaining information in a corresponding physical node, and a second similarity between usage-weighted information and minimum remaining information in a corresponding physical node are calculated, including: constructing the usage-weighted information in the current physical node as a target vector; constructing the maximum remaining information of the target resource in each physical node as a first vector; calculating a first distance between the target vector and the first vector, and using the first distance as a first similarity; constructing the minimum remaining information of the target resource in each physical node as a second vector; calculating a second distance between the target vector and the second vector, and using the second distance as a second similarity.

[0068] In one embodiment, the distribution fit corresponding to each physical node is calculated based on the first similarity and the second similarity, including: for each physical node, calculating the sum of the first similarity and the second similarity corresponding to the corresponding physical node; calculating the ratio of the first similarity corresponding to the corresponding physical node to the sum; and using the ratio as the distribution fit corresponding to the corresponding physical node.

[0069] In one embodiment, processing the target task on the physical node with the highest allocation fit includes: determining resource requirements based on task information of the target task; creating a container or virtual machine in the physical node with the highest allocation fit as the target virtual node based on the resource requirements; and using the target virtual node to process the target task. If the load of any physical node in the target processing system exceeds a preset load limit, some tasks in that physical node are migrated to other physical nodes in the target processing system.

[0070] In one embodiment, migrating some tasks in the physical node to other physical nodes in the target processing system includes: taking some tasks to be migrated in the physical node as target tasks, and executing the step of acquiring the target tasks and subsequent other steps.

[0071] It can be seen that in this embodiment, based on the usage information of the target resources respectively included in each physical node in the target processing system, the usage fluctuation information corresponding to the target resource is determined, and after the usage information is weighted according to the usage fluctuation information, the relationship between the weighted usage information after weighted processing and the remaining information of the target resource in each physical node is used to determine the allocation fit corresponding to each physical node, and the target task is processed in the physical node with the largest allocation fit. For a task, this solution can select a destination node for the current task based on the resource usage and remaining situation of each physical node in the system, realize the reasonable scheduling of tasks, and can allocate the current task to the destination node that provides the most appropriate amount of resources, balance the load and resource availability on different physical nodes in the same system, improve the task scheduling efficiency and resource adaptability in the system, and can solve the problems of low resource scheduling efficiency and severe resource fragmentation to a certain extent. It can adapt to scenarios with high-performance computing, low latency and high dynamic expansion requirements.

[0072] Reference Figure 2The process shown can be illustrated as follows: Assume that the total amount of memory, CPU, bandwidth, and disk on each physical node is the same, all represented by 100, and there are three physical nodes in the system. After collecting usage information for each physical node, it is found that in physical node 1, the memory, CPU, bandwidth, and disk usage are 10, 10, 10, and 10, respectively; in physical node 2, the memory, CPU, bandwidth, and disk usage are 20, 20, 20, and 20, respectively; and in physical node 3, the memory, CPU, bandwidth, and disk usage are 30, 30, 30, and 30, respectively. Then, the proportion of the target resource usage information in each physical node to the total usage information of the corresponding category of resources in the system is as follows: in physical node 1, the corresponding proportions of memory, CPU, bandwidth and disk are 1 / 5, 1 / 5, 1 / 5, and 1 / 5 respectively; in physical node 2, the corresponding proportions of memory, CPU, bandwidth and disk are 2 / 5, 2 / 5, 2 / 5, and 2 / 5 respectively; in physical node 3, the corresponding proportions of memory, CPU, bandwidth and disk are 3 / 5, 3 / 5, 3 / 5, and 3 / 5 respectively; the standardization processing step is omitted here, and the usage weighted information of each physical node obtained after usage weighting is as follows: in physical node 1, the corresponding usage weighted information of memory, CPU, bandwidth and disk are 2, 2, 2, and 2 respectively; in physical node 2, the corresponding usage weighted information of memory, CPU, bandwidth and disk are 8, 8, 8, and 8 respectively; in physical node 3, the corresponding usage weighted information of memory, CPU, bandwidth and disk are 18, 18, 18, and 18 respectively. At this point, the physical node with the largest remaining memory, CPU, bandwidth, and disk is physical node 1, and the maximum remaining amount is 90. Then, the first vector is [90.90.90.90], and the second vector is [70.70.70.70]. For physical node 1, calculate the distance between the target vector [2.2.2.2] and [90.90.90.90] as the first similarity D1 of physical node 1. + , calculate the distance between the target vector [2.2.2.2] and [70.70.70.70] as the second similarity D1 of physical node 1 - For physical node 2, calculate the distance between the target vectors [8.8.8.8] and [90.90.90.90] as the first similarity D2 of physical node 2 + , calculate the distance between the target vector [8.8.8.8] and [70.70.70.70] as the second similarity D3 of physical node 2 - For physical node 3, calculate the distance between the target vector [18.18.18.18] and [90.90.90.90] as the first similarity D3 of physical node 3 + , calculate the distance between the target vector [18.18.18.18] and [70.70.70.70] as the second similarity D3 of physical node 3- ; Then calculate the degree of fit according to the distribution formula S i =D i - / (D i + +D i - ) calculates the allocation fit corresponding to the three physical nodes. In this example, i=1,2,3.

[0073] See Figure 3 ,The target processing system may include: infrastructure layer, hyper-converged resource pool, and resource scheduling layer.

[0074] The infrastructure layer encompasses the physical infrastructure for building a hyperconverged cloud platform, including numerous heterogeneous servers, or physical nodes. This heterogeneous servers and 10GbE networking form a hyperconverged computing and storage environment, supporting the construction of a hyperconverged virtualized resource pool on top of this infrastructure.

[0075] A hyperconverged resource pool uses virtualization and hyperconvergence technologies to virtualize and encapsulate the hardware facilities (such as computing units, storage, and network devices) and basic software (such as operating systems and databases) within physical nodes, providing information infrastructure services to users. Hyperconverged components are deployed on heterogeneous servers, and hyperconvergence technology is used to build a unified virtualized resource pool. Docker lightweight virtualization technology is used to build a heterogeneous computing resource pool, distributed storage virtualization technology is used to build a unified storage resource pool, and software-defined networking technology is used to build a virtualized network environment. Software-defined networking technology is used to create multiple, isolated virtual networks on the physical network, allowing different users to use independent network resource slices, improving network resource utilization and achieving network elasticity. Using software-defined networking makes network virtualization more flexible and efficient.

[0076] The resource scheduling layer provides the entire platform system with a lightweight virtualized flat resource environment that features rapid creation, on-demand scheduling, automatic scaling, and security and reliability. The resource scheduler monitors and manages compute, storage, and network resources within the virtual resource pool, enabling load-based adaptive scaling of containers and managing container network communications. The resource scheduling layer provides interfaces that support upper-layer microservices and applications.

[0077] In this system, the resource scheduling layer allocates tasks to the corresponding physical nodes reasonably based on the resource usage of each physical node and the resource requirements of the user-side tasks, ensuring that the workload is reasonably distributed to the physical nodes in the system to maximize resource utilization and system performance.

[0078] For n physical nodes in the system, monitor and collect the usage of four key resources: CPU, memory, disk, and bandwidth. The collected information can be: ,in, Indicates the CPU utilization on the i-th node; , represents the memory utilization on the i-th node; , Indicates the disk utilization on the i-th node; , Indicates the bandwidth utilization on the i-th node.

[0079] In a converged architecture server system, to effectively allocate and utilize various system resources, ensuring that each virtual node task and workload within the system fully utilizes available computing, storage, and network resources to achieve optimal performance and efficiency, this paper proposes a resource balance indicator to avoid resource over- or under-resource situations, enabling the system to optimally meet user needs and maintain scalability under varying workloads.

[0080] Assume that there are n physical nodes and m types of resources (such as CPU, memory, network bandwidth, disk IO, etc.) in the system. The real-time data of resource usage collected by the monitoring agent Proxy is used to construct the resource matrix X=(x ij ) n×m ,i=1,2,3…n,j=1,2,3…m,x ij represents the resource usage of the jth resource on the i-th node. Correspondingly, .

[0081] The next step is to perform standardization before resource calculation. The calculation formula for standardization is: ; These represent the maximum and minimum values ​​of the jth resource, respectively. This normalization process eliminates the deviations caused by numerical differences in different resource categories, such as CPU, memory, network bandwidth, and disk IO, and enables subsequent weighted operations to be performed on the same scale.

[0082] Next, we calculate the proportion P of the j-th resource usage of the i-th physical node to the total usage of the resource in the system. ij , ;in, The normalized value of the resource usage of the jth resource type on the i-th node. i = 1, 2, 3, ... n, represents the index of the physical node; j = 1, 2, 3, ... m, represents the index of the resource type.

[0083] Resource scheduling often involves multiple decision objectives, such as CPU, memory, network bandwidth, and disk I / O. By calculating a weight for each resource type and weighting them according to the relative importance of the decision objectives, this weighting approach ensures that important decision objectives receive higher weights in the weight calculation, more accurately reflecting their impact on resource scheduling decisions. Furthermore, by identifying and evaluating the relevance and importance of resources, redundant and duplicative resources can be effectively eliminated or reduced, improving decision efficiency and accuracy. This weighting also quantitatively describes which resource types are more scarce.

[0084] This embodiment uses the fluctuation coefficient to measure the uncertainty of resources such as CPU, memory, network bandwidth and disk IO in the system. The fluctuation coefficient can quantify the degree of dispersion of different types of resources. The fluctuation coefficient value of each type of resource is calculated. The larger the fluctuation coefficient value, the greater the difference in the usage of this type of resource on each physical node, and the smaller its impact on decision making. , k=1 / ln(n)>0, where n is the total number of physical nodes in the system. If the usage of a certain type of resource on each physical node is extremely uneven, the entropy is small, indicating that it is more critical to decision-making.

[0085] The next step is to use the fluctuation coefficient to calculate the corresponding usage weight. , the larger the volatility coefficient value, the smaller the weight.

[0086] Next, we use the usage weight to weight each element in the normalized matrix X. From each physical node dimension, we multiply the normalized resource usage by the corresponding usage weight to get the weighted resource usage. ;W j is the normalized resource usage The corresponding usage weight.

[0087] The Euclidean distance between each physical node and the ideal solution (optimal solution) and the anti-ideal solution (worst solution) is then calculated, and the physical node closest to the ideal solution is selected as the destination node. This calculation can reflect the degree to which various resources should be prioritized in scheduling at this moment, providing a benchmark for subsequent weighted decisions.

[0088] set up is the remaining maximum value of the first type of resource (such as CPU). This value is in physical node i. The remaining maximum values ​​of all types of resources and their physical nodes are summarized and recorded to obtain .

[0089] set up Indicates the remaining minimum value of the first type of resource (such as CPU). This value is in physical node i. The remaining minimum values ​​of all types of resources and their physical nodes are summarized and recorded to obtain

[0090] From each physical node dimension, calculate the weighted resource usage and and The Euclidean distance between the remaining quantities of corresponding resources in the optimal solution is expressed as: Worst solution . Represents the weighted resource usage z ij The corresponding resource j is The remaining maximum value recorded in , Represents the weighted resource usage z ij The corresponding resource j is The remaining minimum value recorded in .

[0091] Next, we calculate the distribution fit of each physical node based on the positive and negative ideal solutions. The greater the allocation fit, the closer the corresponding physical node is to the ideal solution. The physical node with the highest allocation fit is preferentially selected as the destination node.

[0092] In order to further integrate the processing status and task types of historical tasks, this embodiment establishes a historical task stack to record previous task execution information, and determines the type of the current task based on this information, thereby performing a secondary weighting on resource usage.

[0093] The following is the calculation of the resource requirement threshold used to determine the task type (there can be multiple under the same task type) , thereby being able to dynamically determine the type of each task at each moment based on historical tasks and real-time monitoring of resource usage.

[0094] For historical tasks, the collected data includes the task type, resource requirements, execution time, completion effect, etc. of each task before the current moment. Task types include: storage intensive, compute intensive, memory intensive, bandwidth intensive, and common.

[0095] For each type of task, a basic threshold is calculated , BaseThreshold type is the first parameter corresponding to the current task type; Q is the total number of tasks, The actual usage of any resource category for the Zth task in multiple historical tasks, where Z = 1, 2, 3... Q. For example, for computationally intensive tasks, additional attention is paid to CPU requirements. The average value of the CPU requirements of historical computationally intensive tasks is calculated as the base threshold. This is used to calculate the BaseThreshold in real time. type , which can achieve: the threshold is increased when the system is busy as a whole; the threshold is lowered when the system is idle as a whole, making the classification of current tasks more flexible.

[0096] AdjustmentFactor(t) is also a dynamic adjustment factor, which is calculated based on real-time data such as the current system load and resource usage. , , which represents the average CPU usage of all physical nodes at the current time t. , , . Indicates the CPU usage of the i-th physical node, Indicates the memory usage of the i-th physical node, represents the network bandwidth usage of the i-th physical node, Indicates the disk IO usage of the i-th physical node. avg AdjustmentFactor(t) is obtained by collecting the average usage of each resource type across all physical nodes over a period of time and then averaging the average across all resources. This means that the average usage of each resource type over time T is calculated and then these averages are averaged. α is a preset adjustment factor that controls the impact of load on the threshold, for example, α = 0.5. When the system's real-time load exceeds the historical average load, AdjustmentFactor(t) increases; otherwise, AdjustmentFactor(t) decreases.

[0097] This embodiment sets: Storage-intensive tasks have high memory requirements and low CPU requirements. Computation-intensive tasks have high CPU requirements and medium memory requirements. Memory-intensive tasks have high memory requirements and medium other requirements. Bandwidth-intensive tasks have high network bandwidth requirements and low other requirements. Based on the above, the demand thresholds corresponding to the four types of resources can be calculated under each task type. This embodiment sets 、 、 、 The corresponding thresholds are respectively for CPU, memory, bandwidth and disk IO. And the following task judgment conditions are set in order: , which records the resource requirements of various resources required for the task to run. , then the task is storage intensive; if , then the task is computationally intensive; if , then the task is memory-intensive; if , then the task is bandwidth-intensive: If none of the above conditions are met, then the task is ordinary.

[0098] Here AdjustmentFactor(t) and BaseThreshold type All tasks are dynamically calculated in real time, so the same task may be classified into different categories at different times. Task classification criteria require that the CPU requirement be below the compute threshold in addition to the resource requirement being above the corresponding threshold to avoid misclassifying CPU-intensive tasks as other tasks. However, compute-intensive tasks require that the CPU requirement be above the threshold and the memory requirement be below the threshold to avoid classifying CPU-intensive and memory-intensive tasks as memory-intensive.

[0099] After determining the task type of the current task, determine the preset type weight of the type. Among them, the type weights corresponding to various task types are: the type weight corresponding to the storage-intensive task type 0.1, 0.1, 0.1, and 0.7 are the weights corresponding to processor resources, memory resources, bandwidth resources, and disk resources, respectively. The weight of 0.7 for disk resources is the largest, indicating that this type of task focuses on data storage performance. Weights corresponding to compute-intensive tasks (Focus on computing performance.) Type weights corresponding to memory-intensive task types (Emphasis on memory performance.) Bandwidth-intensive task type corresponding type weight (Focus on network performance.) Type weights corresponding to common task types .

[0100] Next, the resource usage, after the initial weighting, is re-weighted using the type weight preset for the current task type. After the second weighting is complete, the degree of fit is calculated. Secondary weighting multiplies the resource usage after the initial weighting by the corresponding type weight. Multiplying resource usage by the resource usage weight first and then the type weight simultaneously considers both global importance and task preferences, enabling multi-objective coordinated scheduling that prioritizes the overall situation over the focus. Resource usage weights reflect the global importance and availability fluctuations of various resources within the system, ensuring fairness and balanced utilization across physically disparate nodes and resources. Type weights, on the other hand, emphasize the core resource bottlenecks required by the task type. For example, compute-intensive tasks are prioritized on the CPU, ensuring that critical tasks receive priority scheduling in the resource dimensions they most need. This two-level multiplication not only makes the algorithm modular and adjustable, but also significantly improves scheduling robustness, allowing the system to dynamically adapt to hardware load fluctuations while precisely meeting the specific needs of each task.

[0101] It should also be noted that in order to correct the allocation fit corresponding to each physical node according to the node load and the overall system resource usage, the node penalty factor and dynamic adjustment coefficient can also be calculated.

[0102] The penalty factor calculation formula is: ;P i is the node penalty factor of physical node i, i=1,2,3…n, n is the total number of physical nodes in the target processing system; λ is the penalty coefficient, which is used to control the intensity of the penalty; U avg is the average resource utilization of physical node i between any historical moment and the current moment; R i is the resource utilization of physical node i. The penalty factor is used to penalize nodes with too high or too low resource utilization, which can avoid system overload or resource waste and prevent overloaded or idle nodes from being selected too often or too little.

[0103] The adjustment coefficient calculation formula is: ; is the dynamic adjustment coefficient of physical node i at the current time t, β is the preset adjustment factor, U j (t) is the resource utilization of resource j in the target processing system at the current time t, where resource j is any resource type in the target resources, and j = 1, 2, 3, ..., m, where m is the total number of resource types in the target resources. The dynamic adjustment coefficient takes into account the dynamic changes in task resource requirements and the impact of system load, adjusting the fit of each physical node based on the real-time system status. When the system is busy, the overall fit is appropriately increased to accelerate response.

[0104] After that, the fit calculation formula is: This decision-making process not only considers resource distribution and task requirements, but also dynamically responds to system status, making it objective and flexible. Nodes with higher allocation fit are closer to the ideal resource allocation and receive priority in task allocation.

[0105] After determining the destination node, a Kubernetes-based container is created and started on that node to run the current task. Specifically, the node pulls the necessary container images and configures the container's runtime environment, including mounting volumes, configuring the network, and allocating required resources (CPU, memory, disk I / O, network bandwidth, etc.).

[0106] In one example, a specific solution process includes: Step 1: When a user submits a request to create a virtual node, the scheduler obtains the current system resource usage and constructs the matrix X.

[0107] Step 2: The scheduler calculates the weight of each resource type (CPU, memory, network bandwidth, and disk I / O) by calculating the fluctuation coefficient value to ensure that the relative importance of these resources is reasonably considered during the scheduling process.

[0108] Step 3: The scheduler calculates the distance between each physical node and the ideal solution and selects the optimal node based on the allocation fit.

[0109] Step 4: The scheduler deploys the created virtual node to the selected optimal node.

[0110] It can be seen that this solution not only considers CPU and memory, but also takes into account resources such as network bandwidth and disk IO, ensuring more balanced resource usage, reducing resource fragmentation and overload, and more intelligently evaluating the load of nodes to avoid overload or excessive idleness of single node resources; it improves system performance and resource utilization: through reasonable resource scheduling, it can improve the performance of the entire system and ensure that resources are maximized.

[0111] In another example, a specific solution process includes: Step 1: The user submits a computing task, which includes basic information of the computing task, resource requirements, constraints, etc.

[0112] Step 2: The scheduler obtains information about the system, including the current status of each physical node, resource utilization, network topology, etc.

[0113] Step 3: Create a container: Based on the task resource requirements, determine the container's resource requirements (CPU, memory, etc.) and possible constraints (such as label requirements for node selection, and some nodes are not involved in the current scheduling).

[0114] Step 4: Calculate and determine the optimal node.

[0115] Step 5: Based on the calculation results of Step 4, the physical node with the highest allocation fit is selected as the optimal node, which is used to deploy the newly created container.

[0116] Step 6: The scheduler marks some of the resources on the selected node as occupied by the newly created container and updates the system status. The computing task is then scheduled to that node.

[0117] Step 7: Create and start the container on the selected optimal node.

[0118] Step 8: Run the computing task in the newly created container on the optimal node.

[0119] During task execution, the scheduler regularly checks the resource utilization of each physical node. If a node's resources are insufficient or the resource requirements of a computing task change, the scheduler can reschedule the computing task to a physical node with more sufficient resources. For example, the computing task to be scheduled can be treated as a new task and the optimal node can be re-determined for it according to this embodiment.

[0120] In summary, this embodiment abstracts physical resources such as computing, storage, and network into virtualized resource pools, uses lightweight virtualization technologies such as container Docker to build a heterogeneous computing resource pool, adopts distributed storage virtualization technology to build a unified storage resource pool, and combines software-defined network technology to build a virtual network environment, so that the system can reasonably schedule multiple resources including CPU, memory, disk I / O, network bandwidth, etc., solves the complexity of heterogeneous computing resource scheduling, and realizes unified management and scheduling of resources.

[0121] When scheduling resources, the system comprehensively considers the usage of various resources, including CPU, memory, network bandwidth, and disk I / O, to ensure proper resource allocation during the scheduling process and avoid resource overload or excessive idleness, thereby improving overall system resource utilization and performance. Specifically, a fluctuation coefficient is introduced to quantify the uncertainty of various types of resources in the system and calculate resource usage weights accordingly. In scheduling decisions, resource usage weights reflect the importance and scarcity of each resource. The scheduler uses this information to optimize resource allocation decisions, ensuring that critical resources are prioritized, thereby improving scheduling accuracy and system performance. This embodiment also categorizes tasks into compute-intensive, memory-intensive, network bandwidth-intensive, and I / O-intensive tasks, and designs corresponding type weights for each task type based on their varying resource requirements. This allows the scheduler to further incorporate task type weights into its decision-making. The system then calculates the Euclidean distance between each node and the ideal solution (optimal resource solution) and the anti-ideal solution (worst resource solution), selecting the node closest to the ideal solution as the task scheduling target. This method can ensure that the nodes selected by the scheduler are most appropriate in terms of resource utilization and task load, further optimizing the execution efficiency of the task.

[0122] The following introduces a task allocation device provided in an embodiment of the present application. The task allocation device described below can be referenced with other embodiments described in this document.

[0123] See also Figure 4As shown, an embodiment of the present application discloses a first task allocation device, including: an information collection module, which is used to collect usage information of target resources respectively included in each physical node in the target processing system if a target task is obtained; a fluctuation determination module, which is used to determine usage fluctuation information corresponding to the target resource based on the usage information; a fluctuation weighting module, which is used to weight the usage information according to the usage fluctuation information to obtain usage weighted information of the target resource respectively included in each physical node; a task allocation module, which is used to determine the allocation fit corresponding to each physical node based on the relationship between the usage weighted information and the remaining information of the target resource in each physical node, and process the target task in the physical node with the largest allocation fit.

[0124] The task allocation device can balance the load and resource availability on different physical nodes in the same system.

[0125] See also Figure 5 As shown, an embodiment of the present application discloses a second task allocation device, which is applied to a scheduling node in a target processing system, including: a threshold calculation module, which is used to calculate the resource demand thresholds corresponding to a plurality of preset task types based on historical task information between any historical moment and the current moment if a target task is obtained; a type determination module, which is used to use the resource demand threshold to determine the target task type to which the target task belongs; a type weighting module, which is used to perform weighted processing on the usage information of the target resources respectively included in each physical node in the target processing system according to the type weight corresponding to the target task type, and obtain the usage weighted information of the target resources respectively included in each physical node; a task allocation module, which is used to determine the allocation fit corresponding to each physical node according to the relationship between the usage weighted information and the remaining information of the target resource in each physical node, and process the target task in the physical node with the largest allocation fit.

[0126] Among them, the task allocation device can improve the scheduling efficiency and resource adaptability of multiple types of tasks in the same system.

[0127] Among them, for more specific working processes of each module and unit in this embodiment, reference can be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0128] An electronic device provided in an embodiment of the present application is introduced below. The electronic device described below can be referenced with other embodiments described herein.

[0129] An embodiment of the present application discloses an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the method disclosed in any of the above embodiments.

[0130] Furthermore, the embodiment of the present application also provides an electronic device. The electronic device can be Figure 6 The server shown can also be Figure 7 The terminal shown. Figure 6 and Figure 7 Each of the diagrams is a structural diagram of an electronic device according to an exemplary embodiment, and the contents in the diagrams cannot be considered as any limitation on the scope of use of the present application.

[0131] Figure 6 This is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory is used to store a computer program, which is loaded and executed by the processor to implement the relevant steps of task allocation disclosed in any of the aforementioned embodiments.

[0132] In this embodiment, the power supply is used to provide operating voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external devices. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface is used to obtain external input data or output data to the outside world. The specific interface type can be selected according to specific application needs and is not specifically limited here.

[0133] In addition, the memory as a carrier for resource storage can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include operating system, computer programs and data, etc. The storage method can be temporary storage or permanent storage.

[0134] The operating system is used to manage and control the hardware devices and computer programs on the server to enable the processor to operate and process data in the memory. It can be Windows Server, NetWare, Unix, Linux, etc. In addition to computer programs capable of implementing the task allocation method disclosed in any of the aforementioned embodiments, computer programs can further include computer programs capable of performing other specific tasks. Data can include data such as application update information and other data such as application developer information.

[0135] Figure 7 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may specifically include but is not limited to a smartphone, tablet computer, laptop computer or desktop computer.

[0136] Generally, the terminal in this embodiment includes: a processor and a memory.

[0137] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor may be implemented in at least one of the following hardware forms: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content required to be displayed on the display. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0138] The memory may include one or more computer non-volatile storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory is used to store at least the following computer program, wherein, after the computer program is loaded and executed by the processor, it can implement the relevant steps in the task allocation method performed by the terminal side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include but is not limited to update information of the application.

[0139] In some embodiments, the terminal may further include a display screen, an input and output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0140] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation to the terminal, and may include more or fewer components than shown in the figure.

[0141] A non-volatile storage medium provided in an embodiment of the present application is introduced below. The non-volatile storage medium described below can be referenced with other embodiments described herein.

[0142] A non-volatile storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the task allocation method disclosed in the aforementioned embodiment. The non-volatile storage medium is a computer-readable non-volatile storage medium that, as a carrier for resource storage, may be a read-only memory, random access memory, a magnetic disk, or an optical disk. The resources stored thereon include an operating system, a computer program, and data, and the storage method may be either temporary or permanent.

[0143] A computer program product provided in an embodiment of the present application is introduced below. The computer program product described below can be referenced with other embodiments described herein.

[0144] A computer program product includes a computer program / instruction, which implements the steps of the aforementioned task allocation method when executed by a processor.

[0145] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the steps in any of the above embodiments are implemented.

[0146] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0147] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of non-volatile storage medium known in the art.

[0148] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A task allocation method, characterized in that: The scheduling nodes used in the target processing system include: If the target task is obtained, the resource requirement thresholds corresponding to the preset multiple task types are calculated based on the historical task information between any historical moment and the current moment; Determining the target task type to which the target task belongs using the resource requirement threshold; performing weighted processing on usage information of target resources respectively included in each physical node in the target processing system according to the type weight corresponding to the target task type, to obtain weighted usage information of the target resources respectively included in each physical node; According to the relationship between the usage weight information and the remaining information of the target resource in each physical node, the allocation fit corresponding to each physical node is determined, and the target task is processed in the physical node with the largest allocation fit.

2. The method according to claim 1, characterized in that Based on the historical task information between any historical moment and the current moment, the resource requirement thresholds corresponding to various preset task types are calculated, including: For any task type, determine the first parameter corresponding to the current task type based on historical task information belonging to the current task type between any historical moment and the current moment; Calculate a second parameter corresponding to the current task type based on a preset adjustment coefficient, an average of usage information of the target resource in each physical node at the current moment, an average of historical usage information of the target resource in each physical node, and the total number of resource categories in the target resource; A resource requirement threshold corresponding to the current task type is calculated based on the first parameter and the second parameter.

3. The method according to claim 2, characterized in that Determine the first parameter corresponding to the current task type based on historical task information belonging to the current task type between any historical moment and the current moment, including: Get multiple historical tasks in the historical task information of the current task type; A first parameter corresponding to the current task type is calculated based on the total number of the multiple historical tasks and actual resource usage information of each historical task.

4. The method according to claim 3, characterized in that Calculating a first parameter corresponding to the current task type based on the total number of the multiple historical tasks and actual resource usage information of each historical task includes: Calculate the first parameter corresponding to the current task type according to the first parameter calculation formula; The first parameter calculation formula is: ;BaseThreshold type is the first parameter corresponding to the current task type; Q is the total number of tasks, The actual resource usage information of the Zth task among the multiple historical tasks, where Z=1, 2, 3…Q.

5. The method according to claim 2, characterized in that The calculation formula for the average value of the usage information of the target resource in each physical node at the current moment is: ; R j (t) is the mean value of the usage information of resource j in n physical nodes at the current time t; x i,j is the usage information of resource j in any physical node i included in n physical nodes, n is the total number of physical nodes in the target processing system, i=1,2,3…n; resource j is any type of resource in the target resources, j=1,2,3…m, m is the total number of resource categories in the target resources.

6. The method according to claim 5, characterized in that The target resources include: processor resources, memory resources, bandwidth resources and / or disk resources; Accordingly, the second parameter corresponding to the current task type is calculated based on the preset adjustment coefficient, the average usage information of the target resource in each physical node at the current moment, the average historical usage information of the target resource in each physical node, and the total number of resource categories in the target resource, including: Calculate the second parameter corresponding to the current task type according to the second parameter calculation formula; The calculation formula of the second parameter is: ； AdjustmentFactor(t) is the second parameter, α is the preset adjustment coefficient, R cpu (t) is the average usage information of processor resources in each physical node at the current time t, R mem (t) is the average usage information of memory resources in each physical node at the current time t, R net (t) is the average usage information of bandwidth resources in each physical node at the current time t, R io (t) is the average usage information of disk resources in each physical node at the current time t, R avg (t) is the mean value of the historical usage information, and 4 is the total number of resource categories in the target resource.

7. The method according to claim 1, characterized in that The remaining information of the target resource in each physical node includes: the maximum remaining information and the minimum remaining information of the target resource in each physical node; Accordingly, determining the allocation fit corresponding to each physical node according to the relationship between the usage weight information and the remaining information of the target resource in each physical node includes: For each physical node, calculating a first similarity between the usage weighted information in the corresponding physical node and the maximum remaining information, and a second similarity between the usage weighted information in the corresponding physical node and the minimum remaining information; The allocation fit corresponding to each physical node is calculated based on the first similarity and the second similarity.

8. The method according to claim 7, characterized in that Calculating a first similarity between the usage weighted information in the corresponding physical node and the maximum remaining information, and a second similarity between the usage weighted information in the corresponding physical node and the minimum remaining information, comprising: Constructing the usage weight information in the current physical node into a target vector; constructing the maximum remaining information of the target resource in each physical node into a first vector; calculating a first distance between the target vector and the first vector, and using the first distance as the first similarity; The minimum remaining information of the target resource in each physical node is constructed as a second vector; a second distance between the target vector and the second vector is calculated, and the second distance is used as the second similarity.

9. The method according to any one of claims 1 to 8, characterized in that Before determining the allocation fit corresponding to each physical node based on the relationship between the usage weight information and the remaining information of the target resource in each physical node, the method further includes: determining usage fluctuation information corresponding to the target resource according to the usage information; performing secondary weighting processing on the usage weighted information according to the usage fluctuation information; The usage weighted information after the secondary weighting process is used as the usage weighted information.

10. The method according to any one of claims 1 to 8, characterized in that Also includes: Calculate the node penalty factor of each physical node at the current moment based on the resource utilization rate of each physical node; Calculating a dynamic adjustment coefficient of each physical node at a current moment according to a resource utilization rate of the target resource in the target processing system; Using the node penalty factor and the dynamic adjustment coefficient, correcting the allocation fit corresponding to each physical node; The maximum distribution fit is determined among the corrected distribution fits.

11. The method according to claim 10, characterized in that Calculate the node penalty factor of each physical node at the current moment based on the resource utilization of each physical node, including: Calculate the node penalty factor of each physical node at the current moment according to the penalty factor calculation formula; The penalty factor calculation formula is: ;P i is the node penalty factor of physical node i, i=1,2,3…n, n is the total number of physical nodes in the target processing system; λ is the penalty coefficient; U avg is the average resource utilization of physical node i between any historical moment and the current moment; R i is the resource utilization of physical node i.

12. The method according to claim 10, characterized in that Calculating the dynamic adjustment coefficient of each physical node at the current moment according to the resource utilization rate of the target resource in the target processing system includes: The dynamic adjustment coefficient at the current moment is calculated according to the adjustment coefficient calculation formula; The calculation formula of the adjustment coefficient is: ; is the dynamic adjustment coefficient of physical node i at the current time t, β is the preset adjustment factor, U j (t) is the resource utilization rate of resource j in the target processing system at the current time t, resource j is any type of resource in the target resources, j=1,2,3…m, m is the total number of resource categories in the target resources.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 12.

14. A non-volatile storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images