Machine learning GPU (Graphics Processing Unit) computing power distribution method and device and computer program product

Abstract

The invention relates to the technical field of machine learning, in particular to a machine learning GPU computing power distribution method and device and a computer program product, and the method comprises the steps: constructing a multi-dimensional spectrum feature vector of each GPU; calculating the weight vector of each dimension of the multi-dimensional spectrum feature vector according to the current machine learning task characteristics and the current machine learning task state; calculating the effective computing power score of the GPU according to the weight vector and the multi-dimensional spectrum feature vector; and performing GPU computing power distribution based on the GPU effective computing power score, performing iterative updating on the weight vector according to feedback information of the machine learning task after GPU computing power distribution, and performing iterative correction on GPU computing power distribution. According to the technical scheme provided by the invention, the batch size allocation method based on dynamic weight spectrum mapping can realize accurate evaluation and efficient allocation of heterogeneous GPU resources, and improve the overall efficiency of deep learning training.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a method, apparatus and computer program product for allocating GPU computing power for machine learning. Background Technology

[0002] As the scale of AI training continues to expand, the management efficiency of heterogeneous GPU resource pools has become a key bottleneck restricting computing performance. Traditional GPU resource allocation algorithms are usually based on static rules or simple dynamic strategies, which are difficult to adapt to the differentiated hardware requirements of complex AI tasks. For example, computationally intensive tasks rely on the number of GPU cores and computing power, but have lower requirements for memory bandwidth; while memory-sensitive tasks require higher memory bandwidth and video memory capacity, but have lower priority for computing power.

[0003] The limitations of the relevant technical algorithms are mainly reflected in the following aspects: 1. Static weighting of hardware characteristics: Most solutions only use computing power as the basis for allocation, ignoring key indicators such as memory bandwidth and video memory capacity, resulting in low resource utilization.

[0004] 2. Insufficient task adaptability: It cannot dynamically adjust the allocation strategy according to the task type. For example, in mixed load scenarios, the GPU may be idle or overloaded.

[0005] 3. Uneven load distribution: Fixed distribution strategies may cause some GPUs to complete tasks ahead of time, while other GPUs still have to wait, making it difficult to optimize the overall completion time. Summary of the Invention

[0006] This invention aims to at least partially solve one of the technical problems in the aforementioned technologies, and to this end proposes a method for allocating GPU computing power for machine learning, comprising: Construct a multidimensional spectral feature vector for each GPU; Based on the characteristics and status of the current machine learning task, calculate the weight vector of each dimension of the multidimensional spectral feature vector; The effective computing power score of the GPU is calculated based on the weight vector and the multidimensional spectral feature vector. GPU computing power is allocated based on the effective computing power score of the GPU. The weight vector is iteratively updated according to the feedback information of the machine learning task after the GPU computing power allocation, and the GPU computing power allocation is iteratively corrected.

[0007] Furthermore, the step of calculating the weight vectors of each dimension of the multidimensional spectral feature vector based on the characteristics of the machine learning task and the current state of the machine learning task includes: Generate task feature vectors based on the characteristics of the machine learning task; Based on the task feature vector and the current machine learning task state, calculate the weight vector of each dimension of the multidimensional spectral feature vector.

[0008] Furthermore, the expression corresponding to the weight vector is: W=[w1,w2,w3,...,w j ] Where W represents the weight vector; w j This represents the j-th sub-weight vector; wj=α•t j +β•h j +γ•f j Where α, β, and γ are balance coefficients, and α + β + γ = 1; h j f represents the historical weight value of the j-th dimension of the multidimensional spectral feature vector; j This represents the feedback adjustment factor of the current machine learning task on the j-th dimension of the multidimensional spectral feature vector.

[0009] Furthermore, the expression corresponding to the feedback adjustment factor is: f j =f j +λ·(E target -E current )·▽ j E Where λ represents the learning rate of the current machine learning task; E target E represents the objective balance of the current machine learning task; current This indicates the current balance of the current machine learning task; ▽ j E represents the degree of equilibrium E relative to w. j The gradient; The expression corresponding to the balance degree E is: E=1-[σ(T1,T2,…T m ) / μ(T1,T2,…T m )] Among them, T i σ(·) represents the time required for the i-th GPU to process its batch; σ(·) represents the standard deviation of the sample ·; μ(·) represents the average of the sample ·.

[0010] Furthermore, the expression corresponding to the GPU effective computing power score is:

[0011] Among them, S i v represents the effective computing power score of the i-th GPU; ij This represents the performance value of the i-th GPU in the j-th dimension of the multidimensional spectral feature vector; The expression for allocating GPU computing power based on the GPU's effective computing power score is:

[0012] Among them, B i B represents the batch size allocated to the i-th GPU; total This indicates the total batch size; m indicates the total number of GPUs.

[0013] This application also proposes a machine learning GPU computing power allocation system, including: The spectrum vector construction module is used to construct a multidimensional spectrum feature vector for each GPU; The weight calculation module is used to calculate the weight vector of each dimension of the multidimensional spectral feature vector based on the characteristics and state of the current machine learning task. The batch mapping module is used to calculate the GPU effective computing power score based on the weight vector and the multi-dimensional spectral feature vector, and to allocate GPU computing power based on the GPU effective computing power score. The feedback collection module is used to collect feedback information from machine learning tasks after GPU computing power allocation; An adaptive adjustment module is used to iteratively update the weight vector based on the feedback information and iteratively correct the GPU computing power allocation.

[0014] Furthermore, the system also includes: The task characteristic analysis module is used to generate task feature vectors based on the characteristics of the machine learning task. The weight calculation module is used to calculate the weight vector of each dimension of the multidimensional spectral feature vector based on the task feature vector and the current machine learning task state.

[0015] This application also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program or instructions, and when the computer program or instructions are executed by the processor, they are used to at least implement the above-described machine learning GPU computing power allocation method.

[0016] This application also proposes a computer-readable storage medium storing a computer program or instructions, which, when executed by a processor, are at least used to implement the above-described machine learning GPU computing power allocation method.

[0017] This application also proposes a computer program product stored in a computer-readable storage medium, which, when executed by a processor, is used to at least implement the above-described machine learning GPU computing power allocation method.

[0018] Compared with the prior art, the beneficial effects of the present invention are: The technical solution presented in this application is based on a batch size allocation method using dynamic weighted spectrum mapping. By representing GPU performance as a multi-dimensional spectrum feature vector and combining dynamic weight adjustment and real-time feedback mechanisms, it achieves accurate evaluation and efficient allocation of heterogeneous GPU resources, improves the overall efficiency of distributed deep learning training, reduces the idle rate of computing resources, and ensures that the completion time of each GPU tends to be consistent.

[0019] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. The technical solutions of the invention will be further described below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram illustrating a machine learning GPU computing power allocation method as shown in the embodiment. Figure 2 This is a schematic diagram illustrating the principle of GPU computing power allocation for machine learning, provided as an example. Figure 3 This is a schematic diagram of the GPU computing power allocation iteration process given in the example; Figure 4 This is a schematic diagram of a machine learning GPU computing power allocation system provided in an embodiment. Figure 5 The working principle diagram of the machine learning GPU computing power allocation system is given in the example. Figure 6 A schematic diagram of an electronic device provided for an embodiment; Figure 7 This is a schematic diagram of a computer-readable storage medium provided for an embodiment. Detailed Implementation

[0021] The present invention will be described below with reference to the accompanying drawings. The preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0022] Figure 1 The machine learning GPU computing power allocation method provided in this invention includes: Construct a multidimensional spectral feature vector for each GPU; Based on the characteristics and status of the current machine learning task, calculate the weight vector of each dimension of the multidimensional spectral feature vector; The effective computing power score of the GPU is calculated based on the weight vector and the multidimensional spectral feature vector. GPU computing power is allocated based on the effective computing power score of the GPU. The weight vector is iteratively updated according to the feedback information of the machine learning task after the GPU computing power allocation, and the GPU computing power allocation is iteratively corrected.

[0023] According to some embodiments of this application, such as Figure 2 As shown, the core idea of ​​the Dynamic Weighted Spectrum Mapping (DWSM) algorithm proposed in this invention is to extract the GPU performance features of the heterogeneous GPU pool, represent the computing power of each GPU as a multi-dimensional spectrum feature vector, dynamically adjust the weights of each dimension through an adaptive filter, and adjust the spectrum mapping function in combination with real-time execution feedback, so as to finally achieve accurate batch size allocation.

[0024] Furthermore, based on the characteristics of the machine learning task and the current state of the machine learning task, the weight vectors of each dimension of the multidimensional spectral feature vector are calculated, including: Generate task feature vectors based on the characteristics of machine learning tasks; Based on the task feature vector and the current machine learning task state, calculate the weight vector of each dimension of the multidimensional spectral feature vector.

[0025] Furthermore, the expression corresponding to the multidimensional spectral feature vector is: V i =[v i1 ,v i2 ,v i3 ,...,v in ] Among them, V i Represents a multidimensional spectral eigenvector; v ij This represents the performance value of the i-th GPU in the j-th dimension of the multidimensional spectral feature vector.

[0026] According to some embodiments of this application, the GPU spectral feature vector construction process is as follows: For each GPU, a multidimensional spectral feature vector Vi is constructed, including but not limited to the following dimensions: computing power (FLOPS), memory bandwidth (GB / s), number of cores, video memory capacity (GB), architecture generation index, and dedicated computing unit capabilities (such as Tensor Cores).

[0027] For the i-th GPU, its spectral feature vector can be represented as: V i =[v i1 ,v i2 ,v i3 ,...,v in ] Among them, v ij This represents the performance value of the i-th GPU in dimension j.

[0028] For example, for a heterogeneous GPU pool containing 1 NVIDIA A100 (80GB), 2 NVIDIA V100 (32GB), and 3 NVIDIA T4 (16GB), its spectral feature vectors V_A100, V_V100, and V_T4 can be represented as: V_A100 = [19.5, 1935, 6912, 80, 1.0, 312] V_V100 = [14.0, 900, 5120, 32, 0.8, 112] V_T4 = [8.1, 320, 2560, 16, 0.6, 65] Furthermore, the expression corresponding to the task feature vector is: T=[t1,t2,t3,...,t j ] Where T represents the task feature vector; t j This indicates the degree of dependence of the current machine learning task on the j-th dimension of the multidimensional spectral feature vector; According to some embodiments of this application, a task feature vector T is generated by analyzing the computational characteristics of the current deep learning task: T=[t1,t2,t3,...,t j ] Among them, t j This indicates the degree of dependency of the task on dimension j, ranging from [0,1].

[0029] For example, for a large Transformer model training task that is highly dependent on memory bandwidth (0.85), GPU memory capacity (0.9), and Tensor Cores (0.95), its multidimensional spectral feature vector T can be represented as: T = [0.7, 0.85, 0.5, 0.9, 0.6, 0.95] Furthermore, the expression corresponding to the weight vector is: W=[w1,w2,w3,...,w j ] Where W represents the weight vector; w j This represents the j-th sub-weight vector; wj=α•t j +β•h j +γ•f j Where α, β, and γ are balance coefficients, and α + β + γ = 1; h j f represents the historical weight value of the j-th dimension of the multidimensional spectral eigenvector; j This represents the feedback adjustment factor of the current machine learning task on the j-th dimension of the multidimensional spectral feature vector.

[0030] Furthermore, the expression for the feedback adjustment factor is: f j =f j +λ·(E target -E current )·▽ j E Where λ represents the learning rate of the current machine learning task; E target E represents the objective balance of the current machine learning task; current This indicates the current balance of the current machine learning task; ▽ j E represents the degree of equilibrium E relative to w. j The gradient; The expression for the balance degree E is: E=1-[σ(T1,T2,…T m ) / μ(T1,T2,…T m )] Among them, T i σ(·) represents the time required for the i-th GPU to process its batch; σ(·) represents the standard deviation of the sample ·; μ(·) represents the average of the sample ·.

[0031] According to some embodiments of this application, during the execution of a machine learning task, in order to achieve the feedback factor f j The following feedback information needs to be collected in real time for any adjustments: 1. Actual completion time per GPU 2. Resource utilization (computing power, memory bandwidth, etc.) 3. Computational bottleneck identification Based on the collected feedback information, the feedback adjustment factor is adjusted, and the expression is: f j =f j +λ·(E target -E current )·▽ j E Where λ represents the learning rate of the current machine learning task; E target E represents the objective balance of the current machine learning task; current This indicates the current balance of the current machine learning task; ▽ j E represents the degree of equilibrium E relative to w. j The gradient; It should be noted that the "=" in the above formula indicates assignment.

[0032] In the above formula, the expression corresponding to the balance degree E is: E=1-[σ(T1,T2,…T m ) / μ(T1,T2,…T m )] Among them, T i σ(·) represents the time required for the i-th GPU to process its batch; σ(·) represents the standard deviation of ·; μ(·) represents the average of ·.

[0033] Ideally, E should be close to 1, indicating that the completion time of each GPU is basically the same.

[0034] Furthermore, the expression corresponding to the GPU effective computing power score is:

[0035] Among them, S i v represents the effective computing power score of the i-th GPU; ij This represents the performance value of the i-th GPU in the j-th dimension of the multidimensional spectral feature vector; The expression for allocating GPU computing power based on the GPU's effective computing power score is:

[0036] Among them, B i B represents the batch size allocated to the i-th GPU; total This indicates the total batch size; m indicates the total number of GPUs.

[0037] According to some embodiments of this application, based on the above formula, the effective computing power score of each GPU is calculated according to the spectral feature vector and the weight vector, and then the batch size of each GPU is determined based on the effective computing power score.

[0038] According to some embodiments of this application, for V_A100, V_V100, and V_T4 mentioned above, it is assumed that in the initial stage α=0.7, β=0.2, γ=0.1, the historical weights and feedback factors are all initial values, the feature vector T = [0.65, 0.8, 0.55, 0.85, 0.7, 0.9], and the weight vector W = [0.455, 0.56, 0.385, 0.595, 0.49, 0.63].

[0039] Based on the above data, the effective computing power score is calculated as follows:

[0040] If the total batch size is 1024, then the initial allocation is:

[0041] According to calculations, the actual allocation results are as follows: 260 samples / batch per A100 unit, 128 samples / batch per V100 unit, and 56 samples / batch per T4 unit.

[0042] After a certain number of rounds, feedback data was collected, as shown in Table 1: Table 1

[0043] Based on the data shown in Table 1, calculate the current computing power balance: E current =1-(σ(245,278,263) / μ(245,278,263))=1-(16.72 / 262)=0.936 Assume target equilibrium E target = 0.98, learning rate λ = 0.1. Based on the above data, recalculate the weight vector W_new: W_new = [0.432, 0.585, 0.378, 0.612, 0.475, 0.648] Recalculate scores and batch assignments based on the new weights: B_A100_new = 252 B_V100_new = 134 B_T4_new = 58 After multiple rounds of iterative optimization, the computing power balance E has increased to 0.975, which is close to the target value.

[0044] The above iterative process is as follows: Figure 3 As shown.

[0045] Based on the same technical concept, this application also proposes a machine learning GPU computing power allocation system, which includes: The spectrum vector construction module is used to construct a multidimensional spectrum feature vector for each GPU; The weight calculation module is used to calculate the weight vectors of each dimension of the multidimensional spectral feature vector based on the characteristics and state of the current machine learning task. The batch mapping module is used to calculate the GPU effective computing power score based on the weight vector and the multi-dimensional spectral feature vector, and to allocate GPU computing power based on the GPU effective computing power score. The feedback collection module is used to collect feedback information from machine learning tasks after GPU computing power allocation; The adaptive adjustment module is used to iteratively update the weight vector based on feedback information and iteratively correct the GPU computing power allocation.

[0046] Furthermore, the computing power allocation system also includes: The task characteristic analysis module is used to generate task feature vectors based on the characteristics of machine learning tasks. The weight calculation module is used to calculate the weight vectors of each dimension of the multidimensional spectral feature vector based on the task feature vector and the current machine learning task state.

[0047] According to some embodiments of this application, such as Figure 4 As shown, the machine learning GPU computing power allocation system includes: a spectrum vector construction module, a weight calculation module, a batch mapping module, an allocation execution module, and a task characteristic analysis module.

[0048] The working principle of the above machine learning GPU computing power allocation system is as follows: Figure 5 As shown: First, the controller retrieves GPU features from the GPU pool. The GPU pool returns raw performance data. The controller constructs a spectral feature vector based on the returned information and then submits the current machine learning task information to the task analyzer. The task analyzer returns a task feature vector. The controller then calculates initial weights and batch assignments through a mapper and sends the batch assignment scheme returned by the mapper to the executor for execution. The executor collects execution feedback and sends it to a feedback collector. The feedback collector returns the feedback performance data to the controller, enabling the controller to adaptively adjust the weights based on the feedback performance data, recalculate the batch assignments through the mapper, and send the optimized assignment scheme returned by the mapper to the executor for execution.

[0049] like Figure 6 As shown, this application provides an electronic device, which includes a memory and a processor. The memory stores computer programs or instructions, and when the computer programs or instructions are executed by the processor, they are used to implement at least the above-described machine learning GPU computing power allocation method.

[0050] like Figure 7 As shown, this application provides a computer-readable storage medium storing a computer program or instructions, which, when executed by a processor, are used to implement at least the above-described machine learning GPU computing power allocation method.

[0051] This application also provides a computer program product stored in a computer-readable storage medium, which, when executed by a processor, is used to implement at least the above-described machine learning GPU computing power allocation method.

[0052] It is obvious that those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for allocating GPU computing power in machine learning, characterized in that, include: Construct a multidimensional spectral feature vector for each GPU; Based on the characteristics and status of the current machine learning task, calculate the weight vector of each dimension of the multidimensional spectral feature vector; The effective computing power score of the GPU is calculated based on the weight vector and the multidimensional spectral feature vector. GPU computing power is allocated based on the effective computing power score of the GPU. The weight vector is iteratively updated according to the feedback information of the machine learning task after the GPU computing power allocation, and the GPU computing power allocation is iteratively corrected.

2. The machine learning GPU computing power allocation method as described in claim 1, characterized in that, The step of calculating the weight vectors of each dimension of the multidimensional spectral feature vector based on the characteristics of the machine learning task and the current state of the machine learning task includes: Generate task feature vectors based on the characteristics of the machine learning task; Based on the task feature vector and the current machine learning task state, calculate the weight vector of each dimension of the multidimensional spectral feature vector.

3. The machine learning GPU computing power allocation method as described in claim 2, characterized in that, The expression corresponding to the weight vector is: W=[w1,w2,w3,...,w j ] Where W represents the weight vector; w j This represents the j-th sub-weight vector; wj=α•t j +β•h j +γ•f j Where α, β, and γ are balance coefficients, and α + β + γ = 1; h j f represents the historical weight value of the j-th dimension of the multidimensional spectral feature vector; j This represents the feedback adjustment factor of the current machine learning task on the j-th dimension of the multidimensional spectral feature vector.

4. The machine learning GPU computing power allocation method as described in claim 3, characterized in that, The expression corresponding to the feedback adjustment factor is: f j =f j +λ·(E target -E current )·▽ j E Where λ represents the learning rate of the current machine learning task; E target E represents the objective balance of the current machine learning task; current This indicates the current balance of the current machine learning task; ▽ j E represents the degree of equilibrium E relative to w. j The gradient; The expression corresponding to the balance degree E is: E=1-[σ(T1,T2,…T m ) / μ(T1,T2,…T m )] Among them, T i σ(·) represents the time required for the i-th GPU to process its batch; σ(·) represents the standard deviation of the sample ·; μ(·) represents the average of the sample ·.

5. The machine learning GPU computing power allocation method as described in claim 4, characterized in that, The expression corresponding to the GPU effective computing power score is: Among them, S i v represents the effective computing power score of the i-th GPU; ij This represents the performance value of the i-th GPU in the j-th dimension of the multidimensional spectral feature vector; The expression for allocating GPU computing power based on the GPU's effective computing power score is: Among them, B i B represents the batch size allocated to the i-th GPU; total This indicates the total batch size; m indicates the total number of GPUs.

6. A machine learning GPU computing power allocation system, characterized in that, include: The spectrum vector construction module is used to construct a multidimensional spectrum feature vector for each GPU; The weight calculation module is used to calculate the weight vector of each dimension of the multidimensional spectral feature vector based on the characteristics and state of the current machine learning task. The batch mapping module is used to calculate the GPU effective computing power score based on the weight vector and the multi-dimensional spectral feature vector, and to allocate GPU computing power based on the GPU effective computing power score. The feedback collection module is used to collect feedback information from machine learning tasks after GPU computing power allocation; An adaptive adjustment module is used to iteratively update the weight vector based on the feedback information and iteratively correct the GPU computing power allocation.

7. The machine learning GPU computing power allocation system as described in claim 6, characterized in that, The system also includes: The task characteristic analysis module is used to generate task feature vectors based on the characteristics of the machine learning task. The weight calculation module is used to calculate the weight vector of each dimension of the multidimensional spectral feature vector based on the task feature vector and the current machine learning task state.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program or instructions, which, when executed by the processor, are used to implement at least the method described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions, which, when executed by a processor, are used to implement at least the method described in any one of claims 1-5.

10. A computer program product, said computer program product being stored in a computer-readable storage medium, characterized in that, When the computer program product is executed by a processor, it is used to implement at least the method described in any one of claims 1-5.

Images