A method for elastic scheduling and optimization of GPU computing power

Abstract

This invention relates to the field of GPU resource management technology and discloses a method for elastic scheduling and optimization of GPU computing power. The method first collects real-time operational status data of the GPU cluster, including multi-dimensional metrics such as core utilization and memory usage, to form a status sequence. Then, it divides the cluster into dynamic units based on load distribution and groups them to obtain scheduling subsequences. Next, it acquires the load fluctuation index, computing power allocation differences, and timing differences of each scheduling subsequence to calculate the resource fluctuation coefficient. Then, it constructs and decomposes an association matrix to obtain the core component proportion and association index of the eigenvalue sequence, thereby calculating the scheduling stability index. Finally, based on this index, it determines the dynamic allocation benchmark adjustment amount to achieve elastic scheduling and optimization of GPU computing power. This method integrates multi-dimensional data and multiple indicators, accurately assesses the cluster status, improves computing power resource utilization and cluster performance, and is applicable to various application scenarios.

Description

Technical Field

[0001] This invention relates to the field of GPU resource management technology, specifically a method for elastic scheduling and optimization of GPU computing power. Background Technology

[0002] In today's technological environment, with the rapid development of fields such as artificial intelligence, big data processing, and scientific computing, GPU clusters, as a crucial infrastructure providing powerful computing capabilities, are finding increasingly widespread and complex applications. However, in practical use, GPU clusters face numerous challenges.

[0003] The load on GPU clusters exhibits significant dynamic changes. Different application tasks have vastly different demands on GPU computing power, and even the same task can experience drastic fluctuations in computing power requirements at different stages of execution. For example, during the training of deep learning models, the requirements for GPU core utilization and memory usage differ drastically between the data preprocessing stage and the model iterative training stage. This dynamic variation makes traditional static computing power scheduling methods difficult to adapt to, easily leading to some GPU nodes experiencing performance bottlenecks under high load, while other nodes remain idle under low load, thereby reducing the overall utilization of computing resources in the cluster.

[0004] Existing GPU computing power scheduling methods have shortcomings when handling multi-dimensional data. Real-time operational status data of GPU clusters includes multiple dimensions such as GPU core utilization, memory usage, task queue wait time, and process priority information. These dimensions are interconnected and influence each other. Traditional methods often focus only on one or a few dimensions, ignoring the complex relationships between the data, and thus cannot comprehensively and accurately reflect the actual operational status of the GPU cluster, leading to inaccurate scheduling decisions.

[0005] The temporal nature of computing power allocation has also not been adequately considered. In practical applications, the allocation trends and patterns of computing power at different points in time have a significant impact on task execution efficiency and the overall performance of the cluster. Traditional scheduling methods typically do not conduct in-depth analysis of the temporal changes in computing power allocation, making it difficult to reasonably predict and dynamically adjust future computing power needs based on historical computing power allocation and current task requirements. This results in a lack of flexibility and foresight in computing power allocation.

[0006] Existing technologies also have shortcomings in handling resource fluctuations and scheduling stability in GPU clusters. Due to the uncertainty of task load and the influence of the external environment, the resource status of GPU clusters is prone to fluctuations. Traditional methods lack effective mechanisms to quantify and assess the degree of resource fluctuations and scheduling stability, and cannot adjust scheduling strategies in a timely manner according to fluctuations, thus affecting the overall performance and reliability of GPU clusters. Summary of the Invention

[0007] The purpose of this invention is to provide a method for elastic scheduling and optimization of GPU computing power to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides a method for elastic scheduling and optimization of GPU computing power, the method comprising:

[0009] A status sequence composed of real-time running status data of the GPU cluster;

[0010] The dynamic units are divided according to the load distribution in the state sequence, and the dynamic units in the state sequence are grouped according to the distribution of the dynamic units to obtain each scheduling subsequence;

[0011] Based on the dynamic fluctuations in unit load changes within the scheduling subsequence and the dynamic correlations between units, load fluctuation indicators for each scheduling subsequence are obtained.

[0012] Based on the differences in the computing power allocation trends of the scheduling subsequences and the differences in the timing changes of computing power in the scheduling subsequences, we obtain the differences in computing power allocation and computing power timing.

[0013] Based on the differences in computing power allocation, differences in computing power timing, the average load fluctuation index of all scheduling subsequences, and the average coordination among scheduling subsequences, the resource fluctuation coefficient of the state sequence is obtained.

[0014] Construct the correlation matrix of the state sequence, perform eigenvalue decomposition on the correlation matrix to obtain each eigenvalue, and obtain the core component proportion and correlation index of the eigenvalue sequence based on the distribution, average and difference of the eigenvalues.

[0015] The scheduling stability index of the state sequence is obtained based on the correlation between data in each row of the correlation matrix, the resource fluctuation coefficient, the proportion of core components, and the correlation index; the GPU computing power is then elastically scheduled and optimized based on the scheduling stability index.

[0016] Preferably, the method for obtaining the scheduling subsequence is as follows:

[0017] The load threshold detection algorithm is used to obtain all high-load and low-load nodes in the state sequence. The state sequence is divided into subsequences from each low-load node. The load span of the unit in each subsequence is calculated. The load span of all subsequences is used as the input of the grouping algorithm to output each group cluster.

[0018] Calculate the mean value of the internal units of each group cluster, take the group cluster with the largest mean value of internal units as the scheduling group cluster, and take the state subsequence corresponding to the internal units of the scheduling group cluster as the scheduling subsequence.

[0019] Preferably, the method for obtaining the load fluctuation index is as follows:

[0020] Based on the fluctuations in the dynamic unit load changes in the scheduling subsequence and the dynamic correlation between units, the load oscillation parameters and load change parameters of each scheduling subsequence are obtained.

[0021] The load fluctuation index of each scheduling subsequence is obtained by combining the load oscillation parameters and load change parameters of each scheduling subsequence.

[0022] Preferably, the method for obtaining the load change parameters is as follows:

[0023] For each scheduling subsequence, obtain the maximum load value in each scheduling subsequence, and take the subsequence consisting of the maximum load value and all units to the left of the maximum load value as the left subsequence of each scheduling subsequence, and take the subsequence consisting of all units to the right of the maximum load value as the right subsequence of each scheduling subsequence.

[0024] Obtain the load change sequence of the left subsequence of each scheduling subsequence, and use a smoothing function to process all units in the load change sequence to obtain a smoothed change sequence;

[0025] The absolute value of the difference between the absolute value of the unit sum in the change smooth sequence and the length of the load change sequence of the left subsequence is calculated as the load oscillation value of the left subsequence of each scheduling subsequence.

[0026] Using the same method as the left subsequence, the load oscillation value of the right subsequence of each scheduling subsequence is obtained; the product of the absolute value of the difference between the load oscillation values ​​of the left subsequence and the right subsequence and the mean value is calculated as the load oscillation parameter of each scheduling subsequence.

[0027] Calculate the dispersion of units in the left subsequence and the dispersion of units in the right subsequence of each scheduling subsequence, and use the mean of the dispersion of the left subsequence and the right subsequence as the load change parameter of each scheduling subsequence.

[0028] Preferably, the method for obtaining the computing power timing difference is as follows:

[0029] Sort all computing power allocations in all scheduling subsequences according to their temporal positions in the state sequence to construct a computing power sequence; sort the temporal order of all computing power allocations in the scheduling subsequences according to their magnitude to construct a computing power time sequence.

[0030] Obtain the load change sequence of the computing power sequence and the computing power time series sequence. Use the dispersion of all units in the load change sequence of the computing power sequence as the computing power allocation difference; use the dispersion of all units in the load change sequence of the computing power time series sequence as the computing power time series difference.

[0031] Preferably, the method for obtaining the resource fluctuation coefficient is as follows:

[0032] Calculate the mean of the load fluctuation index for all scheduling subsequences as the average fluctuation index.

[0033] The mean of the coordination coefficients between each scheduling subsequence and all other scheduling subsequences is obtained as the sequence coordination of each scheduling subsequence;

[0034] The method for obtaining the resource fluctuation coefficient is as follows: the resource fluctuation coefficient of the state sequence is obtained based on the differences in computing power allocation, differences in computing power timing, average fluctuation index, the mean of the sequence coordination of all scheduling subsequences, and a preset adjustment coefficient.

[0035] Preferably, the method for obtaining the core component proportion and correlation index is as follows:

[0036] Sort all feature values ​​in descending order to construct a feature value sequence;

[0037] The feature value sequence is used as the input of the key threshold method, and the output of the key threshold method is the threshold. All feature values ​​greater than or equal to the threshold are used as core feature values, and all feature values ​​less than the threshold are used as secondary feature values. The ratio of the number of core feature values ​​to the total number of all feature values ​​is used as the proportion of core components in the feature value sequence.

[0038] Calculate the mean and dispersion of all core feature values ​​in the feature value sequence, and use the product of the mean and dispersion as the correlation fluctuation value of the core feature value; use the same method as the correlation fluctuation value of the core feature value to calculate the correlation fluctuation value of the secondary feature value, and use the absolute value of the difference between the correlation fluctuation value of the core feature value and the correlation fluctuation value of the secondary feature value as the correlation index of the feature value sequence.

[0039] Preferably, the method for obtaining the scheduling stability index is as follows:

[0040] Calculate the absolute value of the coordination coefficient between each row of data in the association matrix and all other rows of data, and take the mean of the absolute values ​​of all the coordination coefficients of each row of data as the coordination mean of each row of data; take the mean of the coordination mean of all rows of data in the association matrix as the coordination coefficient of the association matrix itself.

[0041] The method for obtaining the scheduling stability index is as follows: the scheduling stability index of the state sequence is obtained based on the proportion of core components of the feature value sequence, the correlation index, the self-coordination coefficient of the correlation matrix, the resource fluctuation coefficient, and the preset adjustment factor.

[0042] Preferably, the elastic scheduling and optimization of GPU computing power based on the scheduling stability index includes:

[0043] The adjustment amount of the dynamic allocation benchmark is obtained based on the scheduling stability index of the state sequence. The method for determining the adjustment amount is as follows: the adjustment amount of the dynamic allocation benchmark is determined according to the number of units in the state sequence, the scheduling stability index, and the preset adjustment index.

[0044] The computing power of the GPU cluster is dynamically allocated based on the adjustment amount of the dynamically allocated baseline, and the optimal allocation scheme of computing power in the GPU cluster is calculated using the elastic scheduling method.

[0045] Preferably, the real-time running status data of the GPU cluster includes a multi-dimensional data sequence composed of GPU computing core utilization, video memory usage, task queue waiting time, and process priority information.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] 1. This invention comprehensively and accurately grasps the cluster's operating status by collecting and analyzing real-time data from multiple dimensions, such as GPU computing core utilization, memory usage, task waiting time, and process priority, providing a rich and accurate data foundation for scheduling decisions.

[0048] 2. Based on the dynamic division of load distribution into units and the generation of scheduling sub-sequences, the load change patterns can be effectively identified, enabling fine-grained management and targeted scheduling of cluster resources.

[0049] 3. By combining multiple indicators such as load fluctuation, computing power allocation difference, and timing difference, the resource fluctuation coefficient is calculated to quantitatively assess the degree of cluster resource fluctuation and provide a reliable basis for scheduling adjustment.

[0050] 4. By constructing an association matrix and performing feature decomposition, the proportion of core components and association indices are extracted to deeply explore the inherent association characteristics of the cluster, further calculate the scheduling stability index, and comprehensively evaluate the system stability.

[0051] 5. Introduce computing power sequence and time series analysis to enhance the understanding of historical and real-time allocation trends, improve the ability to predict future computing power demand, and achieve forward-looking elastic resource allocation and optimization.

[0052] 6. This method can adapt to different application scenarios and task requirements, and has good processing capabilities for various dynamic load conditions. It has wide applicability and strong practicality. Attached Figure Description

[0053] Figure 1 This is a schematic diagram illustrating the working principle of the GPU computing power elastic scheduling and optimization method described in this invention.

[0054] Figure 2 A flowchart for obtaining the scheduling subsequence;

[0055] Figure 3 Flowchart for obtaining load change parameters;

[0056] Figure 4 A flowchart for obtaining the proportion of core components and related indices. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please see Figures 1-4 This invention provides a method for elastic scheduling and optimization of GPU computing power, the method comprising:

[0059] A status sequence composed of real-time running status data of the GPU cluster is collected. The real-time running status data of the GPU cluster includes a multi-dimensional data sequence composed of GPU computing core utilization, video memory usage, task queue waiting time, and process priority information.

[0060] Dynamic units are divided based on the load distribution in the state sequence, and the dynamic units in the state sequence are grouped according to the distribution of the dynamic units to obtain each scheduling subsequence.

[0061] Based on the dynamic fluctuations in unit load changes and the dynamic correlations between units in the scheduling subsequence, the load fluctuation index of each scheduling subsequence is obtained.

[0062] Based on the differences in the computing power allocation trends of the scheduling subsequences and the differences in the timing changes of computing power in the scheduling subsequences, we can obtain the differences in computing power allocation and computing power timing.

[0063] Based on the differences in computing power allocation, differences in computing power timing, the average load fluctuation index of all scheduling subsequences, and the average coordination among scheduling subsequences, the resource fluctuation coefficient of the state sequence is obtained.

[0064] Construct the correlation matrix of the state sequence, perform eigenvalue decomposition on the correlation matrix to obtain each eigenvalue, and obtain the core component proportion and correlation index of the eigenvalue sequence based on the distribution, average and difference of the eigenvalues.

[0065] The scheduling stability index of the state sequence is obtained based on the correlation between data in each row of the correlation matrix, the resource fluctuation coefficient, the proportion of core components, and the correlation index; the GPU computing power is then elastically scheduled and optimized based on the scheduling stability index.

[0066] Example 1

[0067] During the acquisition of the scheduling subsequence, the state sequence needs to be processed. The state sequence consists of real-time running status data of the GPU cluster, including multi-dimensional data such as GPU core utilization, memory usage, task queue wait time, and process priority information. The load threshold detection algorithm mentioned here detects nodes in the state sequence to distinguish between high-load and low-load nodes. This algorithm sets a load threshold; when a node's load exceeds this threshold, it is considered a high-load node; when a node's load is below this threshold, it is considered a low-load node.

[0068] After obtaining all high-load and low-load nodes, the state sequence needs to be segmented starting from each low-load node. The purpose of this is to divide the state sequence into multiple subsequences. For example, assuming there are multiple low-load nodes in the state sequence, the sequence can be segmented at the locations of these low-load nodes to obtain several independent subsequences.

[0069] After segmenting the data into subsequences, it is necessary to calculate the load span of each unit in each subsequence. The load span here refers to the difference between the maximum and minimum load values ​​in the subsequence. By calculating the load span, the range of load variation in each subsequence can be determined. For example, for a subsequence with a maximum load value of A and a minimum load value of B, the load span of that subsequence is AB.

[0070] The load spans of all subsequences are used as input to the grouping algorithm. Common clustering algorithms, such as K-means clustering, can be used for this purpose. The grouping algorithm then groups subsequences with similar load spans into the same cluster based on the input load span data, ultimately outputting multiple clusters.

[0071] After obtaining the clusters, it is necessary to calculate the internal unit mean of each cluster. The internal unit mean refers to the average load value of all units in the cluster. By calculating the internal unit mean, the overall load level of each cluster can be understood. For example, if a cluster has n units, and the load values ​​of each unit are C1, C2, ..., Cn, then the internal unit mean of that cluster is (C1 + C2 + ... + Cn) / n.

[0072] The cluster with the highest internal unit mean among all clusters is identified as the scheduling cluster. This scheduling cluster is considered to be a relatively high-load and representative cluster. The state subsequences corresponding to the internal units of the scheduling cluster are extracted as the final scheduling subsequences. These scheduling subsequences will play an important role in the subsequent GPU computing power elastic scheduling and optimization process, and will be used to further analyze load fluctuations, computing power allocation, and other factors.

[0073] Throughout the process, each step must be strictly performed according to the above method to ensure that the obtained scheduling subsequences accurately reflect the load characteristics of the state sequence. For example, when using the load threshold detection algorithm, the threshold setting needs to be reasonably adjusted according to the actual GPU cluster operation to ensure accurate partitioning of high-load and low-load nodes. When performing subsequence segmentation, it is crucial to ensure accurate segmentation to avoid missing important data. When calculating parameters such as load span and internal unit mean, the accuracy of the calculations must be guaranteed to ensure reasonable grouping and selection of scheduling groups.

[0074] Example 2

[0075] When obtaining load fluctuation indicators, it is necessary to acquire load oscillation parameters and load change parameters separately, considering the load change fluctuations and inter-unit correlations within the scheduling subsequences. For each scheduling subsequence, the maximum load value must first be determined, and this value is used as the dividing point to divide the subsequence into left and right parts. The left subsequence consists of the maximum load value and all units to its left, while the right subsequence consists of all units to its right. This division is intended to analyze the load variation characteristics before and after the peak value.

[0076] Process the left subsequence to obtain its load change sequence. Here, the load change sequence refers to the sequence formed by the differences in load values ​​between adjacent cells. For example, if the cell load values ​​in the left subsequence are L1, L2, L3, ..., Ln, then the load change sequence is L2-L1, L3-L2, ..., Ln-Ln-1. Then, use a smoothing function to process this load change sequence. Common smoothing tools such as Gaussian smoothing can be used to reduce noise interference in the data and make the load change trend clearer. After smoothing, a smoothed change sequence is obtained.

[0077] Calculate the absolute value of the sum of all unit values ​​in the smoothed sequence, then calculate the absolute value of the difference between this absolute value and the length of the load change sequence in the left subsequence. Use this difference as the load oscillation value of the left subsequence. For example, if the absolute value of the unit sum in the smoothed sequence is S and the length of the load change sequence is m, then the load oscillation value is |Sm|. The same method is used to process the right subsequence: first, obtain the load change sequence of the right subsequence, process it with a smoothing function to obtain the smoothed sequence, and then calculate the absolute value of the difference between the absolute value of the unit sum and the length of the load change sequence to obtain the load oscillation value of the right subsequence.

[0078] Calculate the absolute value of the difference between the load oscillation values ​​on the left and right sides, and then multiply this difference by the mean of the load oscillation values ​​on both sides. The result is the load oscillation parameter. Assuming the left oscillation value is A, the right oscillation value is B, the absolute value of the difference is |AB|, and the mean is (A+B) / 2, then the load oscillation parameter is |AB|×(A+B) / 2.

[0079] When obtaining load change parameters, it is necessary to calculate the dispersion of unit load values ​​in the left subsequence and the right subsequence. Dispersion can be calculated using methods such as standard deviation, reflecting the degree of data dispersion. For example, for the unit load values ​​in the left subsequence, calculate its standard deviation; this standard deviation is the dispersion of the left subsequence. Similarly, calculate the dispersion of the right subsequence. Finally, the mean of the dispersions on both sides is used as the load change parameter, i.e., (left dispersion + right dispersion) / 2.

[0080] After obtaining the load oscillation parameter and the load variation parameter, these two parameters need to be combined to obtain the load fluctuation index. This combination can be achieved through a certain calculation rule, such as weighting the two parameters. The specific weighting settings need to be determined based on the actual situation, but no specific formula is involved here; only the logical process is described. For example, different weights can be assigned to the load oscillation parameter and the load variation parameter according to their influence on load fluctuation, and then the two can be weighted and summed to obtain the load fluctuation index.

[0081] Throughout the process, each step requires meticulous execution. For example, determining the maximum load value necessitates accurate positioning to avoid misalignment of the left and right subsequences due to data reading errors. When using smoothing functions, the choice of parameters affects the smoothing effect and must be set appropriately based on the data characteristics. Calculating dispersion requires accurate application of statistical methods to ensure the results accurately reflect the degree of data dispersion.

[0082] By obtaining the load fluctuation index in this way, it is possible to comprehensively reflect the characteristics of load changes in the scheduling subsequence. The load oscillation parameter reflects the oscillation amplitude and symmetry of the load before and after the peak, and the load change parameter reflects the distribution dispersion of the load values on both the left and right sides. The combination of the two can more comprehensively describe the complex situation of load fluctuations. These indicators will be used in subsequent processes such as calculating the computing resource fluctuation coefficient and the scheduling stability index, and further affect the elastic scheduling strategy of GPU computing power. For example, a scheduling subsequence with a large load fluctuation index may require a more flexible computing power allocation method to cope with the drastic changes in load, ensuring that the GPU resources can be dynamically adjusted according to the actual load situation and improving the resource utilization efficiency.

[0083] Embodiment III

[0084] When obtaining the computing power timing difference, it is necessary to process the computing power allocation amounts and timing information in all scheduling subsequences. Extract all the computing power allocation amounts included in all scheduling subsequences, and then sort them according to the timing positions of these computing power allocation amounts in the original state sequence. Here, the timing position refers to the order in which the computing power allocation amounts appear in the state sequence. For example, the first computing power allocation amount appears at the front, the second appears immediately after it, and so on, and finally a computing power sequence is constructed. This process needs to ensure the accuracy of the timing position of each computing power allocation amount to avoid deviations in subsequent analysis caused by sorting errors.

[0085] Process the computing power allocation timing in the scheduling subsequence. Sort the timings of all computing power allocations in each scheduling subsequence according to their magnitudes. Here, the magnitude of the timing can be understood as the numerical value of the timestamp or other quantitative values representing the time sequence. For example, in a scheduling subsequence, the timings of computing power allocations are t1, t2, t3, where t1 < t2 < t3. After sorting, it is t1, t2, t3, thus constructing a computing power timing sequence. During the sorting process, the accuracy of each timing value should be ensured, and they should be strictly arranged in ascending or descending order to ensure that the computing power timing sequence can correctly reflect the time sequence characteristics of computing power allocation.

[0086] After constructing the computing power sequence and the computing power timing sequence, it is necessary to obtain their load change sequences respectively. For the computing power sequence, the load change sequence is obtained by calculating the difference between adjacent two computing power allocation amounts. For example, if the values in the computing power sequence are F1, F2, F3,..., Fn in sequence, then the load change sequence is F2 - F1, F3 - F2,..., Fn - Fn - 1. Similarly, for the computing power timing sequence, the load change sequence is calculated by taking the difference between adjacent two timing values to obtain the difference sequence of timing changes.

[0087] After obtaining the load change sequence of the computing power sequence, it is necessary to calculate the dispersion of all units in the sequence. Dispersion is a statistic used to measure the degree of data dispersion, which can be achieved by calculating variance, standard deviation, etc. For example, for each difference data point in the load change sequence of the computing power sequence, their variance is calculated. This variance reflects the degree of dispersion of the computing power allocation over time, and is used as the computing power allocation difference. Here, the computing power allocation difference reflects the magnitude and fluctuation of the computing power allocation at different points in time. The larger the value, the more drastic the change in computing power allocation and the more dispersed the distribution.

[0088] Similarly, for the load change sequence of the computing power time series, the dispersion of all units is calculated and used as the computing power time series difference. The computing power time series difference reflects the degree of dispersion of the changes in computing power allocation time series. For example, when the dispersion of the time series difference in computing power allocation is large, it indicates that the time interval of computing power allocation is uneven and there are large fluctuations; conversely, when the dispersion is small, it indicates that the time interval of computing power allocation is relatively uniform and the time series changes are relatively stable.

[0089] When extracting computing power allocation and timing information, it is crucial to ensure data integrity and accuracy, avoiding the omission of any critical data. During sorting operations, strict adherence to rules regarding timing position and numerical magnitude is essential to guarantee the correctness of the computing power sequence and its timing sequence. When calculating the load change sequence, the differences between adjacent data points must be accurately calculated to avoid errors. Furthermore, when calculating dispersion, appropriate statistical methods must be selected to ensure that the results accurately reflect the data's dispersion characteristics.

[0090] Obtaining computing power allocation differences and computing power timing differences in this way can reflect the distribution characteristics and temporal variation characteristics of computing power in the scheduling subsequence from different perspectives. Computing power allocation differences focus on the fluctuations in the amount of computing power allocated, while computing power timing differences focus on the changes in the time interval between allocations. These two difference parameters are used in subsequent calculations of resource fluctuation coefficients, and they work together with other parameters to influence the assessment of resource fluctuations in the state sequence. For example, when both computing power allocation differences and computing power timing differences are large, it indicates significant fluctuations in both the amount and time of computing power allocation. A more flexible scheduling strategy may be needed to cope with this instability, enabling elastic scheduling and optimization of GPU computing power. This allows computing resources to be dynamically adjusted according to actual needs, improving resource utilization efficiency and system stability.

[0091] Example 4

[0092] Obtaining resource fluctuation coefficients requires completing several key steps sequentially. The first step is to calculate the average load fluctuation index of all scheduled subsequences; this average is called the average fluctuation index. Assume there are n scheduled subsequences, and the load fluctuation indices for each subsequence are f1, f2, and f3, respectively. 2、 ...、f n Then the average volatility index The calculation method is to add up all load fluctuation indicators and then divide by the number of scheduling subsequences, that is... The load fluctuation index here is obtained by the method described in Example 2. Each index reflects the comprehensive characteristics of load changes in the corresponding scheduling subsequence, while the average fluctuation index reflects the average level of load fluctuation across all scheduling subsequences.

[0093] We need to obtain the mean of the coordination coefficients between each scheduled subsequence and all other scheduled subsequences, and use this as the sequence coordination of each scheduled subsequence. For each scheduled subsequence i, we need to calculate its coordination coefficient c with the other n-1 scheduled subsequences. i1 c i2 ... c i(n-1) Then calculate the mean of these synergy coefficients. This mean is the sequence coherence of scheduling subsequence i. The coherence coefficient can be determined by calculating the correlation between two scheduling subsequences, such as using the Pearson correlation coefficient. It reflects the degree of correlation between the two scheduling subsequences in terms of load changes, etc., while sequence coherence represents the average degree of correlation between a certain scheduling subsequence and all other scheduling subsequences.

[0094] After obtaining the average fluctuation index, the difference in computing power allocation, the difference in computing power timing, and the mean of the sequence coordination of all scheduled subsequences, we can begin calculating the resource fluctuation coefficient. Here, the formula for calculating the resource fluctuation coefficient is set as follows: Where λ represents the resource fluctuation coefficient, which is an indicator that comprehensively reflects the resource fluctuation situation in the state sequence. f This represents the difference in computing power allocation, calculated using the method described in Example 3, and reflects the degree of dispersion of the temporal variation in computing power allocation. (D) t The difference in computing power timing, also obtained by the method in Example 3, reflects the degree of dispersion of changes in computing power allocation timing. The average fluctuation index calculated above represents the average level of load fluctuation across all scheduled subsequences. This is the mean of the sequence coordination of all scheduling subsequences. In other words, it's the average of the sequence coordination of each scheduling subsequence, resulting in the overall average level of coordination among the scheduling subsequences in the entire state sequence. α, β, γ, and δ are preset adjustment coefficients. These coefficients are pre-set based on actual application scenarios and experience, used to adjust the weighting of different parameters on the resource fluctuation coefficient. ε is also a preset adjustment coefficient used to fine-tune the entire calculation result to adapt to different system requirements.

[0095] During the calculation process, the acquisition of each parameter must strictly adhere to the corresponding methodology. For example, the calculation of computing power allocation differences and computing power timing differences must ensure the accuracy of the construction of the computing power sequence and computing power timing sequence, the error-free calculation of the load change sequence, and the correct solution for the dispersion. The calculation of the average fluctuation index must ensure the accuracy of the load fluctuation index for each scheduling subsequence, and that there are no calculation errors in the summation and averaging processes. Furthermore, the calculation of sequence synergy requires the accurate calculation of the synergy coefficient between each pair of scheduling subsequences, followed by the correct calculation of the mean.

[0096] Setting the preset adjustment coefficients requires comprehensive consideration of multiple factors. For example, when the system is more concerned with changes in computing power allocation, the value of α can be appropriately increased; if the temporal stability of computing power allocation is more important, the weight of β can be increased. The specific values ​​of these coefficients can be determined through extensive testing and simulation before system deployment to ensure that the resource fluctuation coefficients accurately reflect the actual resource fluctuation situation.

[0097] The resource fluctuation coefficient is a key intermediate parameter that integrates information from various aspects, including differences in computing power allocation, timing differences in computing power, average load fluctuations, and the coordination between scheduling subsequences. In subsequent processing, it will be used to calculate the scheduling stability index of the state sequence, thus influencing the elastic scheduling strategy for GPU computing power. For example, a large resource fluctuation coefficient indicates that resource fluctuations in the state sequence are relatively severe, potentially requiring more aggressive elastic scheduling measures, such as dynamically adjusting the baseline value of computing power allocation to adapt to rapid resource changes and avoid waste or shortage of computing power resources.

[0098] Example 5

[0099] When obtaining the proportion of core components and the correlation index, the first step is to construct a correlation matrix for the state sequence. The elements in the correlation matrix reflect the correlation between different units in the state sequence. After constructing the correlation matrix, eigenvalue decomposition is performed to obtain multiple eigenvalues. Next, all eigenvalues ​​are sorted in descending order to form an eigenvalue sequence. For example, assuming the eigenvalues ​​obtained through eigenvalue decomposition are λ1, λ2, λ3, λ4, λ5, and λ1 > λ2 > λ3 > λ4 > λ5, then the sorted eigenvalue sequence is λ1, λ2, λ3, λ4, λ5.

[0100] After sorting, the feature value sequence is used as input for the key thresholding method. The key thresholding method outputs a threshold based on the input feature value sequence; this threshold distinguishes between core and secondary feature values. All feature values ​​greater than or equal to the threshold are considered core feature values, while those less than the threshold are classified as secondary feature values. For example, if the threshold is set to λ3, then in the previous example, λ1, λ2, and λ3 are core feature values, while λ4 and λ5 are secondary feature values. The key thresholding method can employ common methods such as the cumulative contribution rate method, for instance, setting the feature value corresponding to a cumulative contribution rate of 80% as the threshold.

[0101] After determining the core and secondary eigenvalues, the proportion of core components is calculated. The proportion of core components is the ratio of the number of core eigenvalues ​​to the total number of eigenvalues. Continuing with the example above, there are 3 core eigenvalues ​​and 5 total eigenvalues, so the proportion of core components is 3 ÷ 5 = 0.6. This value reflects the proportion of core eigenvalues ​​in the entire eigenvalue sequence, demonstrating the degree to which core eigenvalues ​​contribute to the features of the correlation matrix.

[0102] Next, we calculate the correlation index of the eigenvalue sequence. First, we process the core eigenvalues, calculating the mean and dispersion of all core eigenvalues. The mean is calculated by summing all core eigenvalues ​​and dividing by the number of core eigenvalues. For example, the mean of core eigenvalues ​​λ1, λ2, and λ3 is (λ1 + λ2 + λ3) / 3. The dispersion can be represented by calculating the standard deviation. The standard deviation is calculated by first finding the square of the difference between each core eigenvalue and the mean, summing these squares, dividing by the number of core eigenvalues, and then taking the square root of the result. Assuming the calculated standard deviation is σ1, the product of the mean and dispersion of the core eigenvalues ​​is (λ1 + λ2 + λ3) / 3 × σ1. This result is called the correlation fluctuation value of the core eigenvalues.

[0103] The same method is used to process secondary eigenvalues, calculating the mean and dispersion of all secondary eigenvalues. Assuming the mean of secondary eigenvalues ​​λ4 and λ5 is (λ4 + λ5) / 2 and the standard deviation is σ², then the correlation fluctuation value of the secondary eigenvalues ​​is (λ4 + λ5) / 2 × σ². Finally, the absolute value of the difference between the correlation fluctuation value of the core eigenvalues ​​and the correlation fluctuation value of the secondary eigenvalues ​​is calculated; this absolute value is the correlation index of the eigenvalue sequence. For example, if the correlation fluctuation value of the core eigenvalues ​​is A and the correlation fluctuation value of the secondary eigenvalues ​​is B, then the correlation index is |AB|.

[0104] When obtaining the scheduling stability index, the first step is to calculate the absolute value of the coordination coefficient between each row of data in the association matrix and all other rows. Taking the i-th row of the association matrix as an example, we need to calculate its coordination coefficient with the 1st, 2nd, ..., nth rows (i≠n), and then take the absolute value of these coordination coefficients. These coordination coefficients can be obtained by calculating the correlation between two rows of data, such as using the Pearson correlation coefficient. After calculation, the mean of the absolute values ​​of all coordination coefficients in the i-th row is taken to obtain the coordination mean of that row.

[0105] After calculating the collaborative mean of all rows in the association matrix, the average of these collaborative means is taken again. The result is the self-collision coefficient of the association matrix. For example, if the association matrix has m rows, and the collaborative mean of each row is c1, c2, c3, c4, c5, c6, c7, c8, c9, c1, c1, c1, c2 ...2, c1, c1, c2, c1, c2, c1, c2, c1, c2, c1, c1, c2, c1, c2, c1, c2, c1, c2, c1, c2, c1, c2, c 2、 ...、c m Then its own coordination coefficient is (c1+c2+…+c m ) / m. This coefficient reflects the average degree of coordination among the data in each row of the correlation matrix.

[0106] Subsequently, the scheduling stability index is obtained based on the proportion of core components in the eigenvalue sequence, the correlation index, the self-coordination coefficient of the correlation matrix, the resource fluctuation coefficient, and a preset adjustment factor. The preset adjustment factor is a parameter pre-set according to the actual application scenario, used to adjust the impact of each parameter on the scheduling stability index. For example, a higher proportion of core components may indicate a greater impact of core eigenvalues ​​on the system, and the scheduling stability index may be affected accordingly; the correlation index reflects the fluctuation difference between core and secondary eigenvalues, the self-coordination coefficient reflects the synergy within the correlation matrix, and the resource fluctuation coefficient integrates various factors of resource fluctuation. These parameters work together in the calculation of the scheduling stability index through certain logical relationships.

[0107] When performing elastic scheduling and optimization of GPU computing power based on the scheduling stability index, the first step is to obtain the baseline adjustment amount for dynamic allocation. The determination of the adjustment amount is related to the number of units in the state sequence, the scheduling stability index, and the preset adjustment index. For example, assuming there are k units in the state sequence, the scheduling stability index is S, and the preset adjustment index is τ, the calculation of the adjustment amount might consider some relationship between k, S, and τ, such as performing corresponding calculations based on their numerical values. The specific calculation method does not involve formulas, but logically, the magnitude and direction of the adjustment amount are determined through these parameters.

[0108] After obtaining the adjustment amount, the computing power of the GPU cluster is dynamically allocated based on this adjustment. For example, if the adjustment amount is positive, it may be necessary to increase the computing power allocation to some GPU nodes; if it is negative, it may be necessary to decrease the computing power allocation. Then, the elastic scheduling method is used to calculate an optimized computing power allocation scheme in the GPU cluster. The elastic scheduling method combines the adjustment amount with the current operating status of the GPU cluster, such as the load of each node and the computing power utilization rate, to reallocate computing power resources, making the computing power allocation more reasonable and adapting to real-time changes in the system.

[0109] For example, suppose a GPU cluster contains 5 nodes, with 100 units in the state sequence. The scheduling stability index is calculated to a specific value, and the preset adjustment index is set to a certain value. Using these parameters, the adjustment amount is calculated to be +5%. During dynamic allocation, the overall computing power allocation baseline might be increased by 5%. Then, based on the actual load of each node, the increased computing power is allocated to nodes with lower loads to balance the overall cluster load. When using elastic scheduling, data such as the current core utilization and memory usage of each node are considered to avoid allocating computing power to already heavily loaded nodes, thereby achieving optimized computing power allocation and improving the cluster's resource utilization and operating efficiency.

[0110] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0111] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for elastic scheduling and optimization of GPU computing power, characterized in that, The method includes the following steps: A status sequence composed of real-time running status data of the GPU cluster; The dynamic units are divided according to the load distribution in the state sequence, and the dynamic units in the state sequence are grouped according to the distribution of the dynamic units to obtain each scheduling subsequence; Based on the dynamic fluctuations in unit load changes within the scheduling subsequence and the dynamic correlations between units, load fluctuation indicators for each scheduling subsequence are obtained. Based on the differences in the computing power allocation trends of the scheduling subsequences and the differences in the timing changes of computing power in the scheduling subsequences, we obtain the differences in computing power allocation and computing power timing. Based on the differences in computing power allocation, differences in computing power timing, the average load fluctuation index of all scheduling subsequences, and the average coordination among scheduling subsequences, the resource fluctuation coefficient of the state sequence is obtained. Construct the correlation matrix of the state sequence, perform eigenvalue decomposition on the correlation matrix to obtain each eigenvalue, and obtain the core component proportion and correlation index of the eigenvalue sequence based on the distribution, average and difference of the eigenvalues. The scheduling stability index of the state sequence is obtained based on the correlation between data in each row of the correlation matrix, the resource fluctuation coefficient, the proportion of core components, and the correlation index; the GPU computing power is then elastically scheduled and optimized based on the scheduling stability index.

2. The GPU computing power elastic scheduling and optimization method as described in claim 1, characterized in that, The method for obtaining the scheduling subsequence is as follows: The load threshold detection algorithm is used to obtain all high-load and low-load nodes in the state sequence. The state sequence is divided into subsequences from each low-load node. The load span of the unit in each subsequence is calculated. The load span of all subsequences is used as the input of the grouping algorithm to output each group cluster. Calculate the mean value of the internal units of each group cluster, take the group cluster with the largest mean value of internal units as the scheduling group cluster, and take the state subsequence corresponding to the internal units of the scheduling group cluster as the scheduling subsequence.

3. The GPU computing power elastic scheduling and optimization method as described in claim 1, characterized in that, The method for obtaining the load fluctuation index is as follows: Based on the fluctuations in the dynamic unit load changes in the scheduling subsequence and the dynamic correlation between units, the load oscillation parameters and load change parameters of each scheduling subsequence are obtained. The load fluctuation index of each scheduling subsequence is obtained by combining the load oscillation parameters and load change parameters of each scheduling subsequence.

4. The GPU computing power elastic scheduling and optimization method as described in claim 3, characterized in that, The method for obtaining the load change parameters is as follows: For each scheduling subsequence, obtain the maximum load value in each scheduling subsequence, and take the subsequence consisting of the maximum load value and all units to the left of the maximum load value as the left subsequence of each scheduling subsequence, and take the subsequence consisting of all units to the right of the maximum load value as the right subsequence of each scheduling subsequence. Obtain the load change sequence of the left subsequence of each scheduling subsequence, and use a smoothing function to process all units in the load change sequence to obtain a smoothed change sequence; The absolute value of the difference between the absolute value of the unit sum in the change smooth sequence and the length of the load change sequence of the left subsequence is calculated as the load oscillation value of the left subsequence of each scheduling subsequence. Using the same method as the left subsequence, the load oscillation value of the right subsequence of each scheduling subsequence is obtained; The product of the absolute value of the difference between the load oscillation values ​​of the left subsequence and the right subsequence and the mean value is calculated as the load oscillation parameter of each scheduling subsequence; Calculate the dispersion of units in the left subsequence and the dispersion of units in the right subsequence of each scheduling subsequence, and use the mean of the dispersion of the left subsequence and the right subsequence as the load change parameter of each scheduling subsequence.

5. The GPU computing power elastic scheduling and optimization method as described in claim 2, characterized in that, The method for obtaining the computing power timing difference is as follows: Sort all computing power allocations in all scheduling subsequences according to their temporal positions in the state sequence to construct a computing power sequence; sort the temporal order of all computing power allocations in the scheduling subsequences according to their magnitude to construct a computing power time sequence. Obtain the load change sequence of the computing power sequence and the computing power time series sequence. Use the dispersion of all units in the load change sequence of the computing power sequence as the computing power allocation difference; use the dispersion of all units in the load change sequence of the computing power time series sequence as the computing power time series difference.

6. The GPU computing power elastic scheduling and optimization method as described in claim 1, characterized in that, The method for obtaining the resource fluctuation coefficient is as follows: Calculate the mean of the load fluctuation index for all scheduling subsequences as the average fluctuation index. The mean of the coordination coefficients between each scheduling subsequence and all other scheduling subsequences is obtained as the sequence coordination of each scheduling subsequence; The method for obtaining the resource fluctuation coefficient is as follows: the resource fluctuation coefficient of the state sequence is obtained based on the differences in computing power allocation, differences in computing power timing, average fluctuation index, the mean of the sequence coordination of all scheduling subsequences, and a preset adjustment coefficient.

7. The GPU computing power elastic scheduling and optimization method as described in claim 1, characterized in that, The method for obtaining the core component proportion and correlation index is as follows: Sort all feature values ​​in descending order to construct a feature value sequence; The feature value sequence is used as the input of the key threshold method, and the output of the key threshold method is the threshold. All feature values ​​greater than or equal to the threshold are used as core feature values, and all feature values ​​less than the threshold are used as secondary feature values. The ratio of the number of core feature values ​​to the total number of all feature values ​​is taken as the proportion of core components in the feature value sequence; Calculate the mean and dispersion of all core feature values ​​in the feature value sequence, and use the product of the mean and dispersion as the correlation fluctuation value of the core feature value; use the same method as the correlation fluctuation value of the core feature value to calculate the correlation fluctuation value of the secondary feature value, and use the absolute value of the difference between the correlation fluctuation value of the core feature value and the correlation fluctuation value of the secondary feature value as the correlation index of the feature value sequence.

8. The GPU computing power elastic scheduling and optimization method as described in claim 1, characterized in that, The method for obtaining the scheduling stability index is as follows: Calculate the absolute value of the coordination coefficient between each row of data in the association matrix and all other rows of data, and take the mean of the absolute values ​​of all the coordination coefficients of each row of data as the coordination mean of each row of data; take the mean of the coordination mean of all rows of data in the association matrix as the coordination coefficient of the association matrix itself. The method for obtaining the scheduling stability index is as follows: the scheduling stability index of the state sequence is obtained based on the proportion of core components of the feature value sequence, the correlation index, the self-coordination coefficient of the correlation matrix, the resource fluctuation coefficient, and the preset adjustment factor.

9. The GPU computing power elastic scheduling and optimization method as described in claim 1, characterized in that, The elastic scheduling and optimization of GPU computing power based on the scheduling stability index includes: The adjustment amount of the dynamic allocation benchmark is obtained based on the scheduling stability index of the state sequence. The method for determining the adjustment amount is as follows: the adjustment amount of the dynamic allocation benchmark is determined according to the number of units in the state sequence, the scheduling stability index, and the preset adjustment index. The computing power of the GPU cluster is dynamically allocated based on the adjustment amount of the dynamically allocated baseline, and the optimal allocation scheme of computing power in the GPU cluster is calculated using the elastic scheduling method.

10. The GPU computing power elastic scheduling and optimization method as described in claim 1, characterized in that, The real-time running status data of the GPU cluster includes a multi-dimensional data sequence consisting of GPU computing core utilization, video memory usage, task queue waiting time, and process priority information.

Images