A multi-model service-oriented aggregated computing power pool construction and channel polling distribution method

Abstract

The application discloses a kind of aggregated computing power pool construction and channel polling distribution method for multi-model service, it is related to computing power distribution technical field.Method includes: computing power pool reconstruction, classification computing power unit, reconstruct computing power base unit, the computing power of corresponding computing power unit is summarized and divided, form new computing power base unit, improve computing power resource utilization efficiency;Channel polling distribution, algorithm dynamic analysis, add dynamic analysis algorithm, preset task quantity grouping, polling queuing task is divided into several groups, task grouping sorting, according to the amount of calculation time target group task is sorted, task dynamic scheduling, preset time threshold, when target task use time exceeds threshold, reduce its computing power, other tasks with shorter operation calculation time.The application can improve the efficiency of computing power resource utilization, avoid low computing power demand task queuing and waiting, high computing power task demand occupies computing power resource, avoid causing high resource idle rate.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of computing power allocation, in particular to an aggregated computing power pool construction and channel polling allocation method for multi-model services. BACKGROUND

[0002] The construction of the aggregated computing power pool is mainly to integrate multiple computing resources to form a powerful computing power set to support the high-performance computing requirements of multi-model services. This resource integration method can not only effectively improve the computing power, but also can fully exert the potential of each computing resource through reasonable configuration and management. The channel polling allocation method is a resource scheduling algorithm used to allocate computing tasks in the aggregated computing power pool. The core idea is to allocate tasks to computing resources in the computing power pool in a fixed order to ensure that each resource has equal opportunities to process tasks.

[0003] The application discloses a computing power scheduling management method and system based on an algorithm, which comprises a cloud platform provided with a plurality of project splitting templates; real-time computing power of a first computing power pool of the cloud platform and expected computing power of a second computing power pool are counted; the cloud platform receives a computing request and allocates the project splitting templates; the computing task is preprocessed by splitting; the cloud platform splits the computing task according to the splitting label; and the cloud platform allocates real-time computing power and expected computing power to each sub-module. The application can more reasonably split the computing task according to the characteristics of the computing task, and allocate computing power according to the real-time computing power and the expected computing power of the cloud platform, so that more computing tasks can be more timely, efficient and reasonable allocated to the required computing power.

[0004] The existing aggregated computing power pool construction and channel polling allocation method for multi-model services has significant defects. The traditional channel polling allocation adopts a fixed order or equal weight strategy, lacks dynamic perception ability of real-time load, task priority and model size difference, causes resource waste phenomena such as queuing of high-priority tasks and forced occupation of whole-card resources by small model tasks, and is difficult to realize fine-grained dynamic adjustment and matching of resources such as display memory, computing power and bandwidth, resulting in high resource idle rate. SUMMARY

[0005] The application aims to provide an aggregated computing power pool construction and channel polling allocation method for multi-model services to solve the problems in the background art.

[0006] To achieve the above-mentioned purpose, the application provides the following technical scheme: an aggregated computing power pool construction and channel polling allocation method for multi-model services, which comprises:

[0007] Computing power pool reconstruction, cleaning of invalid computing power resources, and pooling of effective computing power resources;

[0008] The computing power units are classified into training computing power units, inference computing power units and edge computing power units according to the characteristics and application scenarios of the computing power units and the task requirements;

[0009] The computing power basic units are reconstructed, the computing power of the corresponding computing power units is collected and divided, a new computing power basic unit is formed, and the utilization efficiency of the computing power resources is improved;

[0010] The channel polling allocation sorts the request tasks according to the order of the request arrival;

[0011] The algorithm dynamic analysis adds a dynamic analysis algorithm, groups the polling queued tasks into a plurality of groups, analyzes the task requirements of each group, and improves the resource allocation efficiency;

[0012] The task grouping sorting sorts the target group tasks according to the calculation time, preferentially processes the tasks with low computing power requirements, and improves the overall fluency and resource utilization efficiency of the system;

[0013] The task dynamic scheduling reduces the computing power of the target task when the use time exceeds the threshold, and calculates other tasks with shorter calculation time, thereby improving the utilization efficiency of the computing power.

[0014] The invalid computing power resources are cleaned, the invalid computing power resource standard is preset, the invalid computing power resources are identified through monitoring and log analysis, and the invalid computing power resources are released and isolated.

[0015] The effective computing power resource pooling collects and pools the remaining effective computing power resources after cleaning the invalid computing power resources, calculates the total effective computing power of the computing power resource pool, and the calculation formula is as follows:

[0016]

[0017] C pool total effective computing power of the effective computing power resource pool, n represents the total number of computing power resources participating in the pooling, i is an index variable for traversing each computing power resource participating in the pooling, the value range of i is from 1 to n, a i is the available coefficient of the i th computing power resource, which reflects the current available proportion of the computing power resource, and the value range is [0, 1], C i is the original computing power value of the i th computing power resource, b i is the invalid computing power proportion of the i th computing power resource, g i is the performance coefficient of the i th computing power resource, d i is the weight coefficient of the i th computing power resource, and j n j=1 d i represents the sum of the weight coefficients of all computing power resources.​

[0018] The classification computing power unit, according to the unique characteristics and application scenarios of the computing power unit, classifies the computing power unit, and the classification formula is as follows:

[0019]

[0020] Wherein, R is the computing power resource classification function, used to determine the most suitable computing power unit type, d i represents the i-th data sample in the data set D, D is the data set, containing multiple data samples, ω i is the weight of the data sample d i , used to adjust the importance of different samples in the classification process, E is the error measurement function, used to calculate the error between the operation result of the computing power unit and the true value, f k (d i , θ k ) is the computing power unit type, k is the operation result of the data sample d k under the parameter θ i , ti is the true value or expected result of the data sample d i , λ is the weight coefficient of the cost term, used to balance the trade-off between error and cost, C k is the cost measurement function, used to calculate the cost of the computing power unit type k under the specific parameter and model constraint, θ k is the parameter configuration of the computing power unit type k, M is the model constraint, indicating the restrictions or conditions that the computing power unit needs to follow in the operation process, k∈K represents that the computing power unit type k belongs to the set K.

[0021] The reconstructed computing power basic unit will collect and evenly divide the computing power of the corresponding computing power unit, and the reconstruction computing power formula is as follows:

[0022]

[0023] Wherein, U is a real number, representing the new computing power basic unit after collection and even division, N is a positive integer, representing the total number of computing power units, C i is a real number, representing the computing power of the i-th computing power unit, ∑ N i=1 C i is a summation symbol, representing the sum of all C i from i=1 to i=N, C is a real number, representing the average value of the computing power of all computing power units, (C / C i ) α is a power operation, representing the α-th power of the ratio of the computing power of the i-th computing power unit to the average computing power, α is a real number, which is an adjustable parameter, used to control the strength of the adjustment factor.

[0024] The algorithm dynamically analyzes the preset task quantity grouping, and divides the polling queued tasks into several groups through a dynamic analysis algorithm. The dynamic analysis formula is:

[0025]

[0026] where k i represents the group number to which the i-th task belongs, i represents the index of the task, from 1 to N, Δ(N, α, λ) represents an adjustment function, used to fine-tune the starting point of task allocation, N is the total number of tasks, α is a dynamic adjustment factor, λ is a difference control parameter, used to control the task quantity difference between groups, Q(N, G, α) represents a basic calculation function of the number of tasks in each group, N is the total number of tasks, G is the preset number of groups, and α is a dynamic adjustment factor.

[0027] The task grouping sorting sorts the target group tasks according to the calculation time, and the task grouping sorting formula is as follows:

[0028]

[0029] where P i represents the priority factor of the i-th task, C i represents the estimated calculation time of the i-th task, D i represents the computing power requirement of the i-th task, β represents the weight index of the calculation time, used to adjust the influence of the calculation time on the priority, γ represents the weight index of the computing power requirement, used to adjust the influence of the computing power requirement on the priority, ω represents the relative weight of the computing power requirement, E i represents the urgency or other additional attributes of the i-th task, and δ represents the weight factor of the urgency or other additional attributes, used to adjust the influence of E i in calculating the priority, K represents a constant factor, used to adjust the overall scale of the priority factor, and ε represents a very small positive number, used to avoid the case that the denominator is zero, and ensure the mathematical stability of the formula.

[0030] The task dynamic scheduling presets a time threshold, and reduces the computing power of the target task when the target task uses time exceeds the threshold, and calculates other tasks with shorter calculation time. The calculation formula is as follows:

[0031]

[0032] where A i (t) represents the computing power adjustment factor of the i-th task at time t, D i represents the initial computing power requirement of the i-th task, t represents the current time, θ i represents the time threshold of the i-th task, and α idenotes the rate of the computing power of the ith task, when the task time exceeds the time threshold, the computing power will decrease at this rate, C remain j denotes the remaining computing time of the jth task, C remain i denotes the remaining computing time of the ith task, β ij denotes the computing power transfer weight of the jth task relative to the ith task, when C remain j <C remain i , the jth task can obtain additional computing power from the ith task, I denotes the indicator function, which takes the value 1 when the condition in the parentheses is true, and 0 otherwise.

[0033] Compared with the prior art, the beneficial effects of the present application are:

[0034] The multi-model service-oriented aggregated computing power pool construction and channel polling allocation method cleans up invalid computing power resources, classifies computing power units, and according to the unique characteristics and application scenarios of the computing power units, divides them into training-type computing power units, inference-type computing power units and edge computing power units according to task requirements, reconstructs the basic unit of computing power, improves the utilization efficiency of computing power resources, and avoids resource waste phenomena such as high computing power task demand occupying whole card resources.

[0035] By adding a dynamic analysis algorithm, the polling queued tasks are divided into several groups, so as to analyze the task demand of each group, improve the resource allocation efficiency, sort the target group tasks according to the computing time, preferentially process the tasks with low computing power demand, preset a time threshold, and when the target task time exceeds the threshold, the computing power is reduced, other tasks with shorter computing time are calculated, the utilization efficiency of computing power is improved, and the phenomenon of low computing power demand task queuing and waiting is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a schematic diagram of the principle structure of the present application;

[0037] Figure 2 is a schematic diagram of the principle structure of the resource pool reconstruction in the present application;

[0038] Figure 3 is a schematic diagram of the principle structure of the computing power classification in the present application;

[0039] Figure 4 is a schematic diagram of the principle structure of the task dynamic grouping in the present application;

[0040] Figure 5 is a schematic diagram of the principle structure of the task dynamic scheduling in the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0042] In the present application, the method steps used for the convenience of understanding are not required to be performed according to the sequence of the steps in the embodiments in actual operation, and in other embodiments, the steps can be performed synchronously or in a changed sequence.

[0043] As shown in the drawings, Figures 1-5 The present application provides a technical solution: a method for constructing and polling and distributing a channel of an aggregated computing power pool for a multi-model service, the method comprising:

[0044] Reconfiguring the computing power pool, cleaning up invalid computing power resources, and pooling effective computing power resources;

[0045] Classifying computing power units, according to the unique characteristics and application scenarios of the computing power units, and according to task requirements, into training-type computing power units, inference-type computing power units, and edge computing power units;

[0046] Reconfiguring the computing power basic unit, aggregating and dividing the computing power of the corresponding computing power units to form a new computing power basic unit, and improving the utilization efficiency of computing power resources;

[0047] Polling and distributing a channel, sorting the request tasks according to the order of their arrival;

[0048] Dynamic algorithm analysis, adding a dynamic analysis algorithm, grouping a preset number of tasks, dividing the polling and queuing tasks into several groups, and analyzing the task requirements of each group to improve the resource allocation efficiency;

[0049] Task grouping and sorting, sorting the target group tasks according to the calculation time, preferentially processing tasks with low computing power requirements, and improving the overall smoothness and resource utilization efficiency of the system;

[0050] Dynamic task scheduling, presetting a time threshold, reducing the computing power of a target task when its use time exceeds the threshold, and calculating other tasks with shorter calculation time to improve the utilization efficiency of computing power.

[0051] Cleaning up invalid computing power resources, presetting invalid computing power resource standards, identifying invalid computing power resources through monitoring and log analysis, and releasing and isolating the invalid computing power resources.

[0052] It should be noted that a reasonable resource utilization threshold is set, and resources below the threshold can be considered as invalid computing power resources. For tasks that require fast response, if the response time of the resource exceeds the preset time limit, it can also be considered as invalid computing power resource. If the frequency of errors of the resource in executing tasks is too high, it may indicate that the resource has a problem and needs to be cleaned up.

[0053] After cleaning up the invalid computing power resources, the remaining valid computing power resources are pooled, the total effective computing power of the computing power resource pool is calculated, and the calculation formula is as follows:

[0054]

[0055] Among them, C pool represents the total effective computing power of the valid computing power resource pool, n represents the total number of computing power resources participating in the pooling, i is an index variable for traversing each computing power resource participating in the pooling, the value range of i is from 1 to n, indicating that each computing power resource is calculated and considered separately, a i is the availability coefficient of the i-th computing power resource, which reflects the proportion of the computing power resource that can be used at present, and the value range is between [0, 1], for example, if a server has only 80% of the computing power that can be normally used due to hardware failure, then a i = 0.8, C i is the original computing power value of the i-th computing power resource, which represents the computing power of the computing power resource in the ideal state, b i is the invalid computing power proportion of the i-th computing power resource, which represents the proportion of computing power that cannot be effectively utilized due to hardware aging, software conflicts, etc., and the value range is between [0, 1], for example, if a server has 10% of the computing power that cannot work normally due to software problems, then b i = 0.1, g i is the performance coefficient of the i-th computing power resource, which considers the performance difference of the computing power resource, such as CPU performance, memory bandwidth, etc. of different servers, d i is the weight coefficient of the i-th computing power resource, and j = n j=1 d i represents the sum of the weight coefficients of all computing power resources, which represents the importance or priority of the resource in the pooling.

[0056] It should be noted that C iThe original computing power value of the i-th computing power resource, the hardware specification, directly query the hardware specification of each computing power resource, such as the number of CPU cores, the frequency, the cache size, and the number of GPU stream processors, the size of the video memory, etc., these specifications can usually reflect the original computing power of the computing power resource, the performance test measures the actual computing power of the computing power resource by running benchmark test programs (such as Linpack test of CPU, CUDA benchmark test of GPU, etc.), and serves as a reference for the original computing power value, α i The available coefficient of the i-th computing power resource, the running state of the computing power resource is monitored in real time by using monitoring tools (such as Prometheus, Grafana, etc.), including CPU usage, memory occupancy, disk I / O, etc., so as to judge the availability of the computing power resource, analyze the running log of the computing power resource, find out the events that may cause the computing power to decrease, such as hardware failure, software exception, etc., and adjust the available coefficient accordingly, β i The invalid computing power proportion of the i-th computing power resource, the acquisition method is to periodically troubleshoot the computing power resource, identify and repair problems such as hardware aging and software conflict, so as to reduce the invalid computing power proportion, by comparing the performance test results of the computing power resource in normal running state and in fault state, to estimate the invalid computing power proportion, γ i The performance coefficient of the i-th computing power resource, the performance of different computing power resources is tested by using professional benchmark test tools to obtain its performance coefficient, the performance of the computing power resource in the past task execution is referred to, and the performance coefficient is evaluated in combination with the changes of current hardware and software environment, δ i The weight coefficient of the i-th computing power resource, the importance of different computing power resources is evaluated according to business requirements, so as to determine its weight coefficient, for example, the computing power resource processing key business or high priority task can be given higher weight coefficient, by analyzing the utilization rate and load of the computing power resource in the past period of time, to evaluate its importance in the resource pool, and adjust the weight coefficient accordingly, ∑ j = n j=1 δ i The sum of the weight coefficients of all computing power resources, after determining the weight coefficient of each computing power resource, directly adding it can obtain the sum of the weight coefficients of all computing power resources.

[0057] Substitute the simulation data,

[0058] Computing power resource 1:

[0059] The original computing power value C1=1000;

[0060] The available coefficient α1=0.9;

[0061] The invalid computing power proportion β1=0.05;

[0062] Performance coefficient γ1 = 1.0;

[0063] Weight coefficient δ1 = 0.4;

[0064] Computing power resource 2:

[0065] Original computing power value C2 = 1500;

[0066] Available coefficient α2 = 0.85;

[0067] Invalid computing power proportion β2 = 0.1;

[0068] Performance coefficient γ2 = 0.9;

[0069] Weight coefficient δ2 = 0.3;

[0070] Computing power resource 3:

[0071] Original computing power value C3 = 1200;

[0072] Available coefficient α3 = 0.95;

[0073] Invalid computing power proportion β3 = 0.02;

[0074] Performance coefficient γ3 = 1.1;

[0075] Weight coefficient δ3 = 0.3;

[0076] Calculate the effective computing power of each computing power resource:

[0077] Effective computing power of computing power resource 1:

[0078] C1,effective = C1 × α1 × (1− β1) × γ1;

[0079] = 1000 × 0.9 × 0.95 × 1.0;

[0080] = 855;

[0081] Effective computing power of computing power resource 2:

[0082] C2,effective = C2 × α2 × (1− β2) × γ2;

[0083] = 1500 × 0.85 × 0.9 × 0.9;

[0084] = 1039.5;

[0085] Effective computing power of computing power resource 3:

[0086] C3,effective = C3 × α3 × (1− β3) × γ3;

[0087] = 1200 x 0.95 x 0.98 x 1.1;

[0088] = 1256.04;

[0089] Calculate the sum of all computing power resource weight coefficients:

[0090] ∑ n i=1 δ i = δ1+ δ2+ δ3;

[0091] = 0.4 + 0.3 + 0.3;

[0092] = 1.0;

[0093] Calculate the total effective computing power of the effective computing power resource pool:

[0094] C pool = (855 x 0.4 + 1039.5 x 0.3 + 1256.04 x 0.3) / 1.0;

[0095] = (342 + 311.85 + 376.812) / 1.0;

[0096] = 1030.662;

[0097] Therefore, the total effective computing power of the effective computing power resource pool is 1030.662.

[0098] The classification computing power unit classifies the computing power unit according to the unique characteristics and application scenarios of the computing power unit, and the classification formula is as follows:

[0099]

[0100] Wherein, R is a computing power resource classification function, used to determine the most suitable computing power unit type, d i represents the i-th data sample in the data set D, D is the data set, containing multiple data samples, ω i is the weight of the data sample d i , used to adjust the importance of different samples in the classification process, E is an error measurement function, used to calculate the error between the operation result of the computing power unit and the true value, f k (d i , θ k ) is the operation result of the computing power unit type k on the data sample d k under the parameter θ i , ti is the true value or expected result of the data sample d i , λ is the weight coefficient of the cost term, used to balance the trade-off between error and cost, C k is a cost measurement function, used to calculate the cost of computing power unit type k under certain parameters and model constraints, θk For parameter configuration of computing unit type k, M is the model constraint, representing the restrictions or conditions that the computing unit needs to follow during operation, and k ∈ K represents that the computing unit type k belongs to the set K.

[0101] It should be noted that D is a data set containing multiple data samples, which can be obtained by collecting historical computing power usage records, user feedback, expert evaluation, etc. These data should cover various computing power demands and application scenarios to ensure the accuracy and generalization ability of the classification function, ω i is the weight of data sample d i , obtained by expert scoring, statistical methods (such as determining the weight according to the frequency or distribution of data samples), or machine learning algorithms (such as learning the weight through a trained model), E is the error measurement function, selected according to actual needs, such as mean squared error (MSE), absolute error (MAE), etc. In practical applications, the most suitable function can be selected by comparing the effects of different error measurement functions, f k (d i , θ k ) is the computing unit type, obtained by actually running the computing unit and collecting the output results, multiple experiments and tests need to be conducted for each computing unit type k to ensure the accuracy and reliability of the results, t i is the true value or expected result of data sample d i , obtained by expert evaluation, historical data records, or simulation experiments, etc. to ensure the accuracy and reliability of the true value to avoid misleading the classification function R, λ is the weight coefficient of the cost term, determined by statistical methods or machine learning algorithms, considering the importance of error and cost to determine the appropriate λ value, C k is the cost measurement function, selected according to actual needs, such as considering the price, energy consumption, maintenance cost, etc. of the computing unit, detailed cost analysis is conducted for each computing unit type k to determine the most suitable cost measurement function, θ k is the parameter configuration of computing unit type k, determined by experimental testing or machine learning algorithms, adjusted according to specific computing power requirements and application scenarios to obtain the best computing performance.

[0102] Enter data;

[0103] Data set D: contains 3 data samples d1, d2, d3.

[0104] Computing unit type K: contains two types k1, k2.

[0105] Data sample weight: ω1=0.5, ω2=0.3, ω3=0.2.

[0106] Error measurement function E: uses mean squared error (MSE).

[0107] Computing result of computing unit:

[0108] k1: fk1(d1, θk1) = 0.8;

[0109] fk1(d2, θk1) = 0.6;

[0110] fk1(d3, θk1) = 0.7;

[0111] k2: fk2(d1, θk2) = 0.9;

[0112] fk2(d2, θk2) = 0.5;

[0113] fk2(d3, θk2) = 0.8;

[0114] True value of data sample: t1 = 1;

[0115] t2 = 0.5;

[0116] t3 = 0.9;

[0117] Cost measure function C: preset as a simple function of the number of parameters.

[0118] For k1: Ck1(θk1) = 10;

[0119] For k2: Ck2(θk2) = 15;

[0120] Cost term weight coefficient: λ = 0.1.

[0121] Computing process;

[0122] Calculate the total error of each computing unit type:

[0123] For k1:

[0124] Ek1 = ω1 × (0.8 - 1) 2 + ω2 × (0.6 - 0.5) 2 + ω3 × (0.7 - 0.9) 2 ;

[0125] = 0.5 × 0.04 + 0.3 × 0.01 + 0.2 × 0.04;

[0126] = 0.02 + 0.003 + 0.008;

[0127] = 0.031;

[0128] For k2:

[0129] Ek2 = ω1 × (0.9 - 1) 2 + ω2 × (0.5 - 0.5)2 + ω3 x (0.8 - 0.9) 2 ;

[0130] = 0.5 x 0.01 + 0.3 x 0 + 0.2 x 0.01;

[0131] = 0.005 + 0 + 0.002;

[0132] = 0.007;

[0133] Calculate the total cost of each type of computing power unit:

[0134] For k1: Ck1=10;

[0135] For k2: Ck2=15;

[0136] Calculate the comprehensive score of each type of computing power unit:

[0137] For k1:

[0138] Rk1=Ek1+λ×Ck1;

[0139] =0.031+0.1×10;

[0140] =0.031+1;

[0141] =1.031;

[0142] For k2:

[0143] Rk2=Ek2+λ×Ck2

[0144] =0.007+0.1×15;

[0145] =0.007+1.5;

[0146] =1.507;

[0147] Select the optimal computing power unit type;

[0148] Compare Rk1 and Rk2, and select the type with the lower score;

[0149] In this example, Rk1=1.031 < Rk2=1.507, so k1 is selected.

[0150] The reconstruction computing power base unit will collect and evenly divide the computing power of the corresponding computing power unit. The reconstruction computing power formula is as follows:

[0151]

[0152] Where U is a real number representing the new computing power base unit after collection and even division, N is a positive integer representing the total number of computing power units, Ci is a real number representing the computing power of the i-th computing unit, and N i=1 C i is a summation symbol representing the sum of all C i from i=1 to i=N, C is a real number representing the average value of all computing units' computing power, calculated by dividing the total sum by the number of units N, and (C / C i ) α is a power operation representing the i-th computing unit's computing power raised to the power of α, used as an adjustment factor, adjusted according to the relative size of each computing unit compared to the average computing power, and α is a real number, a tunable parameter used to control the strength of the adjustment factor, which can be any real number, but may need to be adjusted according to specific circumstances in practical applications.

[0153] It should be noted that N is a positive integer determined by the total number of computing units. For example, in a data center or computing cluster, the number of available computing units (such as CPUs, GPUs, etc.) can be counted, and C i is a real number calculated by measurement or estimation. For different types of computing units (such as CPUs, GPUs, etc.), different methods can be used to measure their computing power. For example, for CPUs, floating-point operation capability (FLOPS) or instruction processing capability (IPS) can be used as indicators to measure, and α is a real number set according to actual needs, which will affect the degree of influence of the difference in computing power between computing units on the final new computing power base unit U, and the best α value is determined through experiments or optimization algorithms.

[0154] Enter the data;

[0155] Total number of computing units N: 3;

[0156] Computing power of each computing unit Ci:

[0157] C1=100;

[0158] C2=150;

[0159] C3=200;

[0160] Adjustment parameter α: 2 (this value can be adjusted according to actual conditions);

[0161] Calculate the average value of all computing units' computing power C:

[0162] C=(100+150+200) / 3;

[0163] =450 / 3;

[0164] =150;

[0165] Calculate the α power of the ratio of each computing power unit computing power to the average computing power:

[0166] For C1=(150 / 100) 2 ;

[0167] =1.5 2 ;

[0168] =2.25;

[0169] For C2=(150 / 150) 2 ;

[0170] =1 2 ;

[0171] =1;

[0172] For C3(150 / 200) 2 ;

[0173] =0.75 2 ;

[0174] =0.5625;

[0175] Calculate the new computing power base unit U after the summary and equal division:

[0176] U=(100*2.25+150*1+200*0.5625) / 3;

[0177] =(225+150+112.5) / 3;

[0178] =487.5 / 3;

[0179] =162.5;

[0180] According to the given data and parameters, the new computing power base unit U after the summary and equal division is 162.5.

[0181] The algorithm dynamically analyzes the preset task quantity grouping, and divides the polling queued tasks into several groups through a dynamic analysis algorithm. The dynamic analysis formula is:

[0182]

[0183] Where k i represents the group number to which the i-th task belongs, i represents the index of the task, from 1 to N, Δ(N, α, λ) represents an adjustment function, used to fine-tune the starting point of task allocation, N is the total number of tasks, α is a dynamic adjustment factor, λ is a difference control parameter used to control the task quantity difference between groups, Q(N, G, α) represents a basic calculation function of the number of tasks in each group, N is the total number of tasks, and G is the preset grouping number.

[0184] It should be noted that α is a dynamic adjustment factor, which is determined by experiments, simulations or empirical rules to adjust the value of α according to the actual situation and requirements of task allocation to achieve the best grouping effect, and λ is a difference control parameter, which is determined by experiments, simulations or empirical rules to adjust the value of λ according to the requirements and restrictions of task allocation uniformity to ensure that the task quantity difference between groups is within an acceptable range.

[0185] Suppose we have the following parameters:

[0186] Total task quantity N = 20;

[0187] Default group quantity G = 4;

[0188] Dynamic adjustment factor α = 0.5;

[0189] Difference control parameter λ = 0.2;

[0190] Calculate the base task quantity:

[0191] Q(20, 4, 0.5) = 20 / 4 + 0.5;

[0192] = 5 + 0.5;

[0193] = 5.5;

[0194] Calculate the adjustment quantity:

[0195] Δ(20, 0.5, 0.2) = 0.2 × (20 mod 4);

[0196] = 0.2 × 0;

[0197] = 0;

[0198] Calculate the group number of each task:

[0199] For i = 1:

[0200] k1 = (1 - 1 + 0) / 5 + 1;

[0201] = 0 + 1;

[0202] = 1;

[0203] For i = 2 to i = 5:

[0204] k2 = k3 = k4 = k5 = (i - 1 + 0) / 5 + 1 = 1;

[0205] For i = 6:

[0206] k6 = (6 - 1 + 0) / 5 + 1;

[0207] = 1 + 1;

[0208] =2;

[0209] And so on, until i=20.

[0210] The final result will be that the tasks are divided into 4 groups, each with 5 tasks, and since Δ(N,α,λ)=0, the adjustment amount has no effect on the grouping in this example.

[0211] By this method, we can dynamically allocate tasks to different groups, and can fine-tune the grouping strategy by adjusting α and λ.

[0212] The task grouping order is sorted according to the amount of calculation time for the target group task, and the task grouping order formula is as follows:

[0213]

[0214] Where P i represents the priority factor of the i-th task, C i represents the estimated calculation time of the i-th task, D i represents the computing power demand of the i-th task, β represents the weight index of calculation time, used to adjust the influence of calculation time on priority, γ represents the weight index of computing power demand, used to adjust the influence of computing power demand on priority, ω represents the relative weight of computing power demand, E i represents the urgency or other additional attributes of the i-th task, δ represents the weight factor of urgency or other additional attributes, used to adjust the influence of E i in calculating the priority, K represents a constant factor used to adjust the overall scale of the priority factor, and ϵ represents a very small positive number used to avoid the case where the denominator is zero, ensuring the mathematical stability of the formula.

[0215] It should be noted that β represents the weight index of calculation time, which is estimated by historical data, expert experience or project management software, γ represents the weight index of computing power demand, which analyzes the influence of calculation time and computing power demand on task priority in historical projects, and the value of β is determined by data analysis, ω represents the relative weight of computing power demand, which is determined according to the characteristics of the project and organizational strategy. For example, in a project with tight computing power resources, the value of ω may be relatively high to emphasize the importance of computing power demand, while in a project with abundant computing power resources, the value of ω may be relatively low, E i represents the urgency or other additional attributes of the i-th task, which is determined by extracting key information from the task description, and K represents a constant factor, which is determined by trial and error or comprehensive consideration of other parameters.

[0216] Plug in the data;

[0217] Number of tasks: 3;

[0218] Task data:

[0219] Task 1: C1=10 (compute time), D1=5 (compute power demand), E1=3 (urgency level);

[0220] Task 2: C2=5 (compute time), D2=8 (compute power demand), E2=1 (urgency level);

[0221] Task 3: C3=8 (compute time), D3=3 (compute power demand), E3=2 (urgency level);

[0222] Weights and constants:

[0223] β=2 (weight exponent for compute time);

[0224] γ=1 (weight exponent for compute power demand);

[0225] ω=0.5 (relative weight for compute power demand);

[0226] δ=0.1 (weight factor for urgency level);

[0227] K=10 (constant factor);

[0228] ε=0.0001 (very small positive number);

[0229] For each task, we calculate its priority factor:

[0230] Task 1:

[0231] P1=10×[10 2 ×5 1×0.5 × (1+0.1×3)] / [0.0001+10 2 ×5 1×0.5 × (1+0.1×3)];

[0232] Task 2:

[0233] P2=10×[5 2 ×8 1×0.5 × (1+0.1×1)] / [0.0001+5 2 ×8 1×0.5 × (1+0.1×1)];

[0234] Task 3:

[0235] P3=10×[8 2 ×3 1×0.5 × (1+0.1×2)] / [0.0001+8 2 ×3 1×0.5× (1 + 0.1 x 2) ] ;

[0236] The tasks are sorted in descending order according to the calculated priority factor Pi.

[0237] The task dynamic scheduling, preset time threshold, when the target task use time exceeds the threshold, reduce its computing power, operation calculation time shorter other tasks, calculation formula as follows:

[0238]

[0239] Where, A i (t) represents the power adjustment factor of the i-th task at time t, D i represents the initial computing power requirement of the i-th task, t represents the current time, θ i represents the time threshold of the i-th task, α i represents the computing power reduction rate of the i-th task, when the task use time exceeds the time threshold, the computing power will be reduced at this rate, C remain j represents the remaining calculation time of the j-th task, C remain i represents the remaining calculation time of the i-th task, β ij represents the computing power transfer weight of the j-th task relative to the i-th task, when C remain j < C remain i When the j-th task may obtain additional computing power from the i-th task, I represents the indicator function, which takes the value of 1 when the condition in the parentheses is true, otherwise it takes the value of 0.

[0240] It should be noted that α i represents the computing power reduction rate of the i-th task, the value of αi is determined by experiment, simulation or empirical rule, for example, an initial computing power reduction rate can be set first, then the task execution and system resource utilization are observed, and αi is adjusted according to the observation results, C remain j represents the remaining calculation time of the j-th task, C remain i represents the remaining calculation time of the i-th task, which is obtained by task progress monitoring, time estimation or project management software, etc. Task progress monitoring is to track the execution of the task in real time, so as to calculate the remaining calculation time, time estimation is to predict the remaining calculation time based on the nature and size of the task, and project management software provides the function of task time management and tracking, which can easily obtain the information of the remaining calculation time, β ijβij represents the computing power transfer weight of the jth task relative to the ith task, and the value of βij is determined by methods such as task importance evaluation or system resource optimization algorithm. For example, for an urgent and important task, a higher computing power transfer weight can be given to ensure that it can obtain sufficient computing power resources.

[0241] Substitute the data,

[0242] Number of tasks: 2;

[0243] Task data:

[0244] Task 1: D1=10 (initial computing power demand), Θ1=5 (time threshold), α1=0.2 (computing power reduction rate);

[0245] Task 2: D2=8 (initial computing power demand), Θ2=6 (time threshold), α2=0.1 (computing power reduction rate);

[0246] Computing power transfer weight:

[0247] β12=0.5 (computing power transfer weight of task 2 relative to task 1);

[0248] β21=0.3 (computing power transfer weight of task 1 relative to task 2);

[0249] Current time: t=6;

[0250] Remaining calculation time:

[0251] Cremain1=4;

[0252] Cremain2=2;

[0253] Calculate the computing power adjustment factor Ai(t) of each task;

[0254] Calculate its computing power adjustment factor:

[0255] Task 1:

[0256] A1(6)=10×(1−0.2×5max(0,6−5))+0.3×8×I(2<4);

[0257] Task 2:

[0258] A2(6)=8×(1−0.1×6max(0,6−6))+0.5×10×I(4<2);

[0259] Calculate the indicator function I:

[0260] For I(2<4) in task 1, because the condition is true, I=1.

[0261] For I(4<2) in task 2, because the condition is false, I = 0.

[0262] Substitute the calculation:

[0263] Task 1:

[0264] A1(6) = 10 x (1 - 0.2 x 51) + 0.3 x 8 x 1;

[0265] = 10 x 0.96 + 2.4;

[0266] = 9.6 + 2.4;

[0267] = 12;

[0268] Task 2:

[0269] A2(6) = 8 x (1 - 0.1 x 0) + 0.5 x 10 x 0;

[0270] = 8 x 1 + 0;

[0271] = 8;

[0272] In conclusion, the computational power adjustment factor for task 1 at time t = 6 is 12, while the computational power adjustment factor for task 2 is 8. This means that task 1 has its computational power reduced due to exceeding the time threshold, but a portion of the computational power of task 1 is transferred to task 2 due to the shorter remaining computation time of task 2.

[0273] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for constructing an aggregated computing power pool and allocating channels in a round-robin fashion for multi-model services, characterized in that: The method includes: The computing power pool is restructured, ineffective computing power resources are cleaned up, and effective computing power resources are pooled. The computing power units are classified into training computing power units, inference computing power units, and edge computing power units according to their unique characteristics and application scenarios and task requirements. Reconstruct the basic unit of computing power, aggregate and distribute the computing power of the corresponding computing power units to form a new basic unit of computing power, and improve the utilization efficiency of computing power resources; Channel round-robin allocation sorts the request tasks according to the order in which the requests arrive; The algorithm features dynamic analysis, with a preset task quantity grouping. The polling queue tasks are divided into several groups to analyze the task requirements of each group and improve resource allocation efficiency. Tasks are grouped and sorted according to computation time, prioritizing tasks with low computational requirements to improve overall system smoothness and resource utilization efficiency. Dynamic task scheduling with preset time thresholds: when the time taken by the target task exceeds the threshold, its computing power is reduced, and other tasks with shorter computing time are processed to improve computing power utilization efficiency. The effective computing power resource pooling involves cleaning up ineffective computing power resources, then pooling the remaining effective computing power resources to calculate the total effective computing power of the pool. The calculation formula is as follows: Among them, C pool α represents the total effective computing power of the effective computing power resource pool, n represents the total number of computing power resources participating in the pooling, and i is an index variable used to iterate through each computing power resource participating in the pooling. The value of i ranges from 1 to n. i C is the availability coefficient of the i-th computing resource, reflecting the proportion of the computing resource that is currently available for use. Its value ranges from [0,1]. i Let β be the original computing power value of the i-th computing power resource. i γ represents the percentage of invalid computing power for the i-th computing resource. i Let δ be the performance coefficient of the i-th computing resource. i Let ∑ be the weight coefficient of the i-th computing resource. j = n j=1 δ i This represents the sum of the weight coefficients of all computing resources; The aforementioned computing power units are classified according to their unique characteristics and application scenarios. The classification formula is as follows: Where R is the computing power resource classification function, used to determine the most suitable type of computing power unit, d i Let ω represent the i-th data sample in dataset D, where D is a dataset containing multiple data samples. i For data sample d i The weights are used to adjust the importance of different samples in the classification process, and E is the error metric function used to calculate the error between the computational unit's result and the true value. k (d i ,θ k ) represents the computing unit type k in parameter θ k Below is a sample of data d i The result of the calculation, ti, is the data sample d. i The true value or expected result, λ is the weighting coefficient of the cost item, used to balance the trade-off between error and cost, C k Here, θ is a cost metric function used to calculate the cost of computing unit type k under specific parameters and model constraints. k M is the parameter configuration for computing unit type k, M is the model constraint, which represents the restrictions or conditions that the computing unit needs to follow during the operation, and k∈K means that computing unit type k belongs to set K.

2. The method for constructing an aggregated computing pool and allocating channels for multi-model services according to claim 1, characterized in that: The process of cleaning up invalid computing resources involves setting up invalid computing resource standards, identifying invalid computing resources through monitoring and log analysis, and releasing or isolating invalid computing resources.

3. The method for constructing an aggregated computing pool and allocating channels for multi-model services according to claim 1, characterized in that: The reconstructed computing power basic unit is to summarize and evenly distribute the computing power of the corresponding computing power unit. The reconstructed computing power formula is as follows: Where U is a real number representing the new basic unit of computing power after aggregation and equal distribution, N is a positive integer representing the total number of computing power units, and C i It is a real number representing the computing power of the i-th computing unit. It is a summation symbol, representing all C from i=1 to i=N. i The sum of, It is a real number representing the average computing power of all computing units. It is a power operation, representing the ratio of the computing power of the i-th computing unit to the average computing power raised to the power of α. α is a real number, an adjustable parameter used to control the strength of the adjustment factor.

4. The method for constructing an aggregated computing pool and allocating channels in a round-robin fashion for multi-model services according to claim 1, characterized in that: The algorithm dynamically analyzes the preset number of task groups and divides the polling queue tasks into several groups through dynamic analysis. The dynamic analysis formula is as follows: Where, k i Let represent the group number to which the i-th task belongs, where i represents the index of the task, from 1 to N. Δ(N,α,λ) represents an adjustment function used to fine-tune the starting point of task allocation, where N is the total number of tasks, α is the dynamic adjustment factor, and λ is a difference control parameter used to control the difference in the number of tasks between groups. Q(N,G,α) represents the basic calculation function for the number of tasks in each group, where N is the total number of tasks, G is the preset number of groups, and α is the dynamic adjustment factor.

5. The method for constructing an aggregated computing pool and allocating channels in a round-robin fashion for multi-model services according to claim 1, characterized in that: The task grouping and sorting is based on the computation time required to sort the target group of tasks. The task grouping and sorting formula is as follows: Among them, P i C represents the priority factor of the i-th task. i D represents the estimated computation time for the i-th task. i Let E represent the computational power requirement of the i-th task, used to adjust the impact of computation time on priority; γ represent the weighting index of computational power requirement, used to adjust the impact of computational power requirement on priority; ω represent the relative weight of computational power requirement; and E represents the weighting index of computational power requirement. i Let δ represent the urgency or other additional attributes of the i-th task, and let δ represent the weighting factor of the urgency or other additional attributes, used to adjust E. i In calculating the impact of priority, K represents a constant factor used to adjust the overall size of the priority factor, and ϵ represents a very small positive number used to avoid the case where the denominator is zero, ensuring the mathematical stability of the formula.

6. The method for constructing an aggregated computing pool and allocating channels for multi-model services according to claim 1, characterized in that: The task dynamic scheduling uses a preset time threshold. When the time taken by the target task exceeds the threshold, its computing power is reduced, and other tasks with shorter computation times are processed instead. The calculation formula is as follows: Among them, A i (t) represents the computing power adjustment factor for the i-th task at time t, D i Let θ represent the initial computing power requirement of the i-th task, t represent the current time, and θ represent the current computing power requirement. i Let α represent the time threshold for the i-th task. i This represents the rate at which the computing power decreases for the i-th task. When the task takes longer than a time threshold, the computing power will decrease at this rate. C remain j C represents the remaining computation time for the j-th task. remain i β represents the remaining computation time for the i-th task. ij This represents the computational power transfer weight of the j-th task relative to the i-th task, when C remain j <C remain i When the j-th task may obtain additional computing power from the i-th task, I represents an indicator function, which takes the value 1 when the condition in parentheses is true, and takes the value 0 otherwise.

Images