Resource scheduling method, device and storage medium

Abstract

The present application provides a resource scheduling method, device and storage medium. Data-aware scheduling takes into account the access mode, data size and data type of the task to the data set, and cooperates with the scheduling strategy module to efficiently allocate GPU resources to improve the execution efficiency and overall performance of the task. Through intelligent perception and continuous analysis of data characteristics and memory access requirements, tasks can be dynamically assigned to the most suitable GPU device. This method takes into account the data locality, memory access mode and data dependency of the task, as well as the performance status and resource utilization of the GPU device, to achieve intelligent matching and optimized scheduling between tasks and GPU resources.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a resource scheduling method, device and storage medium. Background Art

[0002] Cloud-native applications are applications deployed and run in a cloud environment, with features such as flexibility, scalability, and elastic scaling. With the widespread application of tasks such as deep learning and scientific computing, GPUs have become an important computing resource, and their parallel computing capabilities can improve the execution efficiency and throughput of tasks.

[0003] However, the management and scheduling of GPU resources face challenges. Traditional scheduling methods often only consider the time constraints of tasks and lack a full understanding of task characteristics and data characteristics. For GPU resource scheduling in cloud-native application scenarios, in cloud-native environments, large-scale cloud computing platforms carry a variety of tasks, including machine learning, data analysis, image processing, etc., and these tasks have different requirements for GPU resources. However, traditional GPU resource scheduling methods often fail to fully consider the data characteristics and memory access requirements of the tasks trained, resulting in low resource utilization, low task execution efficiency, and resource conflicts. At the same time, with the widespread application of large models including AIGC, the demand for the use of GPU resources is increasing, accompanied by a large amount of resource consumption, especially the energy consumption of data centers, so the reasonable scheduling of GPUs is particularly critical. Summary of the invention

[0004] In view of the above problems in the prior art, the purpose of this application is to provide a resource scheduling method that can improve the utilization and performance of GPU resources, reduce energy consumption and operation and maintenance costs, and improve user experience.

[0005] In order to solve the above technical problems, the specific technical solutions of this article are as follows:

[0006] On the one hand, the present application provides a resource scheduling method, which is applied to a resource scheduling device, and the method includes:

[0007] Extracting a data feature vector of a task data set, wherein the task data set includes a task set to be assigned to a GPU device for processing and a data set corresponding to the task set, and the data feature includes a demand feature and an access feature of the task to the data in the data set;

[0008] The data feature vector is analyzed by using a scheduling strategy model, and a task scheduling strategy corresponding to the task data set is output; wherein the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm by using data feature vector samples and GPU scheduling vector samples, and the scheduling features include resource allocation features and data processing features of the GPU device when executing task processing;

[0009] The GPU device corresponding to each task in the task data set, as well as the execution order and priority of each task are determined based on the task scheduling strategy.

[0010] In some embodiments, the characteristic parameters of the data features include at least one of data scale, data dimension, data type complexity, data distribution uniformity, data sparsity, data correlation, data redundancy, and calculation accuracy requirements.

[0011] In some embodiments, the characteristic parameters in the scheduling characteristics include at least one of batch size, number of parallel computing threads, parallel computing window, shared memory usage, computing accuracy selection, instruction set selection, task allocation strategy, data storage format, and data preprocessing parameters.

[0012] In some embodiments, the scheduling policy model includes at least one of the following scheduling policy sub-models: a scheduling policy sub-model for batch size and data scale; a scheduling policy sub-model for the number of parallel computing threads and data scale; a scheduling policy sub-model for shared memory usage and data dimension; a scheduling policy sub-model for instruction set selection and data type complexity; a scheduling policy sub-model for task allocation strategy and data distribution uniformity; a scheduling policy sub-model for data storage format and data sparsity; a scheduling policy sub-model for parallel computing window and data correlation; a scheduling policy sub-model for data preprocessing parameters and data redundancy; and a scheduling policy sub-model for calculation accuracy selection and calculation accuracy requirements.

[0013] In some embodiments, the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples, task feature vector samples and GPU scheduling vector samples, wherein the feature parameters in the task feature vector samples include at least one of task calculation complexity, task execution time, task execution accuracy and task execution recall rate.

[0014] In some embodiments, the scheduling algorithm includes an objective function between a data feature vector and a GPU scheduling vector, and an optimal solution search algorithm for the objective function.

[0015] In some embodiments, the method further includes: monitoring the execution status of the task, and performing any of the following actions:

[0016] If the execution status is abnormal, fault isolation is triggered for the abnormal task;

[0017] Adjust the task scheduling strategy in real time based on the task resource utilization information and GPU device performance indicators;

[0018] After the task is completed, the task execution information and the scheduling information are stored to optimize the scheduling strategy model using the task execution information and the scheduling information.

[0019] On the other hand, the present application provides a resource scheduling device, the device includes a data perception module and a resource scheduling module, the resource scheduling module includes a scheduling policy maker, a resource allocator and a task queue manager; wherein,

[0020] The data perception module is used to extract data feature vectors of a task data set, wherein the task data set includes a task set to be assigned to a GPU device for processing and a data set corresponding to the task set, and the data feature includes a demand feature and an access feature of the task for data in the data set;

[0021] The scheduling strategy maker is used to analyze the data feature vector using a scheduling strategy model, and output a task scheduling strategy corresponding to the task data set; wherein the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples and GPU scheduling vector samples, and the scheduling features include resource allocation features and data processing features of the GPU device when executing task processing;

[0022] The resource allocator is used to determine the GPU device corresponding to each task in the task data set based on the task scheduling strategy;

[0023] The task queue manager is used to determine the execution order and priority of each task in the task data set based on the task scheduling strategy.

[0024] In some embodiments, the resource scheduling module further includes a task execution monitor, a resource utilization optimization unit, a task completion notifier, a scheduling log recorder, and a performance monitor;

[0025] The task execution monitor is used to monitor the execution status of the task. If the execution status is abnormal, fault isolation is triggered for the abnormal task.

[0026] The resource utilization optimization unit is used to adjust the task scheduling strategy in real time according to the resource utilization information of the task and the performance index of the GPU device;

[0027] The task completion notifier is used to issue a task status update notification when the task is completed;

[0028] The scheduling log recorder is used to record task execution information and scheduling information;

[0029] The performance monitor is used to detect the performance indicators of the GPU device in real time.

[0030] In some embodiments, the device further includes an optimization strategy module, and the optimization strategy module is used to optimize the scheduling strategy model using the task execution information and scheduling information recorded by the scheduling log recorder.

[0031] On the other hand, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0032] The present application provides a GPU resource scheduling device and method based on intelligent data perception. Data-aware scheduling considers the access mode, data size, and data type of the task to the data set, and cooperates with the scheduling strategy module to efficiently allocate GPU resources to improve the execution efficiency and overall performance of the task. By intelligently perceiving and continuously analyzing the data characteristics and memory access requirements, the task can be dynamically assigned to the most suitable GPU device. The method considers the data locality, memory access mode, and data dependency of the task, as well as the performance status and resource utilization of the GPU device, and realizes intelligent matching and optimized scheduling between tasks and GPU resources. In a cloud-native environment, GPU resource scheduling is combined with a cloud-native scheduling framework. The cloud-native scheduling framework can dynamically allocate resources and schedule tasks based on factors such as resource requirements, task constraints, and available resources. Through integration with the cloud-native scheduling framework, the intelligent data-aware GPU resource scheduling device can better adapt to cloud-native application scenarios and provide efficient GPU resource management and scheduling.

[0033] In order to make the above and other purposes, features and advantages of this article more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of this article or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of this article. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A flowchart of the operation steps of the intelligent data-aware GPU resource scheduling method is shown;

[0036] Figure 2A module schematic diagram of an intelligent data-aware GPU resource scheduling device is shown;

[0037] Figure 3 The information flow diagram of the entire intelligent data-aware GPU resource scheduling is shown;

[0038] Figure 4 The internal structure of a computer device is shown. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of this article to clearly and completely describe the technical solutions in the embodiments of this article. Obviously, the described embodiments are only part of the embodiments of this article, not all of the embodiments. Based on the embodiments of this article, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this article.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0041] refer to Figures 1 to 3 The embodiment of the present application provides a GPU resource scheduling device based on intelligent data perception, aiming to solve the problem of lack of intelligent perception of data features and memory access requirements in traditional GPU resource scheduling. The device includes a data perception module, a resource scheduling module and an optimization strategy module.

[0042] Among them, GPU, in the field of computer graphics and general computing, is the abbreviation of Graphics Processing Unit. GPU is a hardware device specially designed for processing graphics and image calculations. It is responsible for accelerating graphics rendering and image processing tasks in the computer. GPU and central processing unit (CPU) complement each other and together constitute the core components of the computer, but they have great differences in the way and characteristics of processing tasks.

[0043] The data perception module obtains the data perception information of the task and models it by collecting and analyzing the data characteristics, memory access patterns and dependencies of the task to obtain a data perception information model.

[0044] The resource scheduling module intelligently allocates GPU resources to different tasks based on the data perception information model of the task and the performance characteristics of the GPU device to improve resource utilization and task execution efficiency.

[0045] The optimization strategy module uses intelligent optimization algorithms based on data perception information and system performance monitoring to implement dynamic GPU resource scheduling strategies to meet the real-time requirements of tasks and the overall optimization of the system.

[0046] The resource scheduling device can fully consider data characteristics and provide intelligent GPU resource scheduling, thereby improving the performance, efficiency and reliability of the computing system. The details are as follows:

[0047] The data perception module is a key component of the intelligent data-aware GPU resource scheduling system. It perceives the characteristics of sample data and provides accurate information for resource scheduling based on the characteristics of training sample data and data access requirements. It mainly includes the following components:

[0048] 1. Data feature collection: 1. Collect the data set features when the task is executed, including the size and type of the data set [numeric type, text data (Bag-of-Words model, etc.), image data features, time series type, etc.], access frequency (such as {"url":"example.com","frequency":100}), access mode, etc. Involving the data access operation, access times, data processing operations, etc. of the monitoring task.

[0049] For large data sample sizes, such as processing massive graphics data, if you use the TensorFlow or PyTorch framework, you need to adjust the batch size to control the amount of data loaded into the video memory each time. When the amount of data is close to the video memory capacity, reduce the batch size again.

[0050] In addition, the data transmission strategy is also a key feature. Synchronous transmission or asynchronous transmission is very important. For large-scale data asynchronous transmission, when using CUDA for data transmission and calculation, calling CUDA asynchronous functions (such as cudaMemcpyAsync) can perform data transmission while calculating. Reduce the transmission waiting time. At the same time, the dimension of the data is also an important data feature. Whether it is high-dimensional data or low-dimensional data, the number of times the shared memory and global memory are accessed will affect the GPU scheduling efficiency.

[0051] The data type features mainly include calculation accuracy, and the choice of instruction set is also one aspect of data features. There are mainly half-precision floating point numbers (FP16) or single-precision floating point numbers (FP32). The distribution of data, the sparsity of data, the correlation of data, and the redundancy of data are all data features that the data acquisition module needs to extract.

[0052] 2. Data analysis engine: Analyze and process the collected data features to extract information about the task's data requirements and access patterns. This includes identifying data hotspots, predicting data access patterns, etc. Preliminary modeling can be performed based on some data features as follows:

[0053] 1) Define data characteristics

[0054] The data features at least include the demand features and access features of the data in the data set of the task. The demand features can be used to characterize the features of the data required for calculation, access and other operations when the task is executed, such as data scale, data dimension, data type complexity, data distribution uniformity, data sparsity, etc. The access features can be used to characterize the access to the data when the task is executed, such as access frequency, access mode, number of accesses, etc.

[0055] The data feature vector includes a vector composed of the feature values ​​of at least one data feature. In a specific example, the data feature vector can be set to X = [x1, x2, x3, x4, x5, x6, x7, x8], where:

[0056] x1: represents the data scale (for example, the number of data points or the overall size of the data, which can be normalized, such as mapping it to an interval);

[0057] x2: data dimension (e.g., vector dimension or matrix order, which can also be normalized);

[0058] x3: data type complexity (simple data types are set to 0, and complex data types take values ​​between according to their complexity);

[0059] x4: data distribution uniformity (uniform distribution is set to 1, non-uniform distribution takes values ​​in between according to the degree of unevenness, and 0 indicates extremely uneven distribution);

[0060] x5: data sparsity (non-sparse data is set to 0, sparse data takes values ​​between according to the sparsity, and 1 indicates extremely sparse);

[0061] x6: data correlation (weak correlation is set to 0, strong correlation takes values ​​in between according to the degree of correlation, and 1 indicates extremely strong correlation);

[0062] x7: data redundancy (no redundancy is set to 0, redundancy is set to a value between 1 and 1 depending on the degree of redundancy);

[0063] x8: Calculation precision requirement (low precision requirement is set to 0, high precision requirement is set to 1).

[0064] 2) Define scheduling characteristics

[0065] The scheduling characteristics may at least include resource allocation characteristics and data processing characteristics of the GPU device when executing task processing. Resource allocation characteristics can be used to characterize the resource information allocated by the GPU device for task processing, such as batch size, number of parallel computing threads, shared memory usage, parallel computing window, etc.; data processing characteristics can be used to characterize the task processing method and strategy of the GPU device, such as calculation accuracy, instruction set selection, task allocation strategy, data storage format, data preprocessing parameters, etc.

[0066] The GPU scheduling vector includes a vector composed of the eigenvalues ​​of at least one scheduling feature. For example, the GPU scheduling vector is P = [p1, p2, p3, p4, p5, p6, p7, w, z], where:

[0067] p1: batch size (positive integer);

[0068] p2: number of parallel computing threads (positive integer);

[0069] p3: shared memory usage (in bytes, positive integer);

[0070] p4: calculation precision (0 means half precision, 1 means single precision, 2 means double precision, etc.);

[0071] p5: instruction set selection (discrete value, different instruction sets can be represented by encoding);

[0072] p6: task allocation strategy (discrete value, different allocation encoding);

[0073] p7: data storage format (discrete values, e.g. different storage formats for encoding sparse data);

[0074] w: parallel computing window;

[0075] z: data preprocessing parameters (such as data compression ratio). For redundant data (z>0), preprocessing is performed before the data is transferred to the GPU.

[0076] 3) Define performance indicator function

[0077] The performance index function can be used as the objective function. Let the performance index function be Y represents performance indicators related to computing efficiency, such as floating-point operations per second (FLOPS), the inverse of training time (1 / T, T is training time), etc. The goal is to find the optimal P * , so that Y reaches its maximum value, that is

[0078] By defining different data characteristic parameters and scheduling characteristic parameters in the above way, formula modeling is performed, and the optimal P is derived based on repeated task execution and feedback mechanism. * , used to provide to the resource scheduling module for GPU scheduling.

[0079] 3. Task feature extraction: Task features are used to characterize the characteristics and properties of tasks during execution, including the computational complexity, data volume, execution time, priority, etc. of the task, and are used to guide intelligent data-aware GPU resource scheduling. In addition to data features, this module needs to obtain the characteristics of the task itself, such as the computational complexity, execution time, etc. of the task, accuracy (Precision), recall (Recall), F1 value (F1-score) In order to combine the task characteristics with the data requirements. Accordingly, the characteristic parameters of the task characteristics may include one of the task calculation complexity, task execution time, task execution accuracy, and task execution recall rate.

[0080] 4. Feature vector storage: The analyzed feature vector is stored in the database and modeled according to the above data feature formula for storage for use by the subsequent resource scheduling module.

[0081] Resource Scheduling Module

[0082] The resource scheduling module allocates resources and schedules tasks based on task characteristics, data requirements, GPU device status, and other information to achieve optimal performance and resource utilization. The main components of the resource scheduling module are:

[0083] 1. Task Queue Manager: manages the queue of tasks to be executed, sorts and schedules them according to their priority and characteristics.

[0084] 2. Resource allocator: Determines which task to assign to which GPU device for execution based on the task's data characteristics, computing requirements, and GPU device status.

[0085] 3. Scheduling strategy maker: formulate appropriate scheduling strategies based on the real-time status of the system. According to the data feature formula in the previous step, the corresponding strategy can be specified, and the following models can be determined:

[0086] 1) Scheduling strategy sub-model of batch size and data scale p1 = [k1 / x1]

[0087] Determine the batch size p1. Under a specific GPU model and data type, select the optimal k1 value (proportional coefficient) by repeatedly testing the computational efficiency corresponding to different k1 values, such as training time, convergence speed, etc., so as to accurately adjust the batch size according to the data scale.

[0088] 2) Parallel computing thread number and data size scheduling strategy sub-model p2 = k2x1N

[0089] As the data scale x1 increases, the number of parallel computing threads p2 should be increased accordingly. In the GPU resource scheduling system, it is necessary to clarify the total number of GPU computing cores N and determine the appropriate proportional coefficient k2. However, increasing the number of threads is not unlimited. Too many threads may cause threads to compete for resources, such as shared memory, registers, etc., thereby reducing computing efficiency. Therefore, during the scheduling process, it is necessary to perform performance tests on different types of data and computing tasks to find a suitable k2 value so that the increase in the number of threads can effectively improve computing efficiency without causing resource competition problems. At the same time, it is necessary to consider the hardware limitations of the GPU, such as the maximum number of threads that can be supported, to avoid setting the number of threads that exceeds the hardware capabilities.

[0090] 3) Scheduling strategy sub-model for shared memory usage and data dimension

[0091] For high-dimensional data (x2 is larger), more shared memory p3 should be allocated. When scheduling GPU resources, the dimension x2 of the data needs to be accurately provided. For example, when processing high-dimensional matrix operations, the scheduling module determines its dimension by analyzing the structure of the data, and then calculates the required amount of shared memory according to the formula. Establish a priority mechanism for shared memory allocation. For data and computing tasks with high shared memory requirements and a greater impact on computing efficiency, determine the appropriate proportional coefficient k3 and prioritize shared memory allocation. At the same time, monitor the usage of shared memory to prevent a task from occupying too much shared memory and causing other tasks to fail to run normally.

[0092] 4) Scheduling strategy sub-model of instruction set selection and data type complexity p5 = g(x3)

[0093] For tasks with different data types and complexity x3, select the appropriate instruction set p5.

[0094] In the cloud-native GPU resource scheduling system, the function g(x3) is predefined to analyze different complex data types (such as structures, nested data structures, etc.) and determine the best instruction set that matches them. At the same time, the cost of switching instruction sets must also be considered. Frequent switching of instruction sets will result in additional overhead, so the scheduling strategy should reduce unnecessary instruction set switching. For tasks with similar data types that are processed continuously, the same instruction set can be maintained.

[0095] 5) Scheduling strategy sub-model for task allocation strategy and data distribution uniformity

[0096] According to the relationship between the task allocation ratio r = k4x4 + (1-k4) and the task allocation strategy p6. For non-uniformly distributed data (x4 <1), the scheduling module determines the task allocation strategy p6 according to the calculated r value. For example, when processing data mining tasks in a multi-GPU cluster system, if the data is highly concentrated in certain areas (x4 is small), the r value is calculated by the formula. If r is close to 0, the strategy of concentrating tasks in areas with lighter loads is adopted, and more computing tasks are allocated to the GPUs corresponding to areas with relatively sparse data to achieve load balancing. At the same time, it is necessary to continuously monitor the execution of tasks and dynamically adjust the task allocation strategy according to the actual load conditions to adapt to possible changes in data distribution.

[0097] 6) Data storage format and data sparsity scheduling strategy sub-model p7 = h(x5)

[0098] For sparse data (x5>0), select the appropriate data storage format p7. In the GPU resource scheduling process, the sparsity of the data must be determined first. For example, when processing sparse matrix calculations (such as adjacency matrix processing in graph calculations), if the data is sparse, the scheduling module selects storage formats such as compressed coefficient row (CSR) or compressed sparse column (CSC) according to the predefined function h(x5).

[0099] 7) Parallel computing window and data correlation scheduling strategy sub-model w = [k5 / x6]

[0100] The parallel computing window size w is determined according to the formula w=[k5 / x6]. For strongly correlated data (x6>0), the parallel computing strategy is adjusted. In GPU resource scheduling, for example, when processing time series analysis tasks, the appropriate window size w is calculated according to the formula by evaluating the correlation x6 of the data.

[0101] 8) Data preprocessing parameters and data redundancy scheduling strategy sub-model z = k6x7

[0102] The data preprocessing parameters (e.g., data compression ratio) are determined according to the formula. For redundant data (z>0), preprocessing is performed before the data is transferred to the GPU. In the scheduling module, the redundancy of the data is first determined. For example, in the video processing task, the redundancy is evaluated by analyzing the similarity between video frames. The technology is preprocessed according to the calculated z value, such as data compression.

[0103] 9) Scheduling strategy sub-model for computational accuracy selection and computational accuracy requirements p4 = [x8]

[0104] According to the formula, select the calculation accuracy based on the requirements for calculation accuracy. In the GPU resource scheduling process, for tasks with lower precision requirements (x8=0), half precision (p4=0) can be selected for calculation to increase the calculation speed. For example, in the early training stages of some deep learning or in simulation calculations that do not require high result accuracy, the scheduling system can set the calculation accuracy to half precision. When scheduling, it is necessary to accurately judge the accuracy requirements of the task, and at the same time consider the impact of different calculation accuracy on the calculation speed and resource consumption. On the premise of meeting the accuracy requirements, try to choose a lower calculation accuracy to improve the calculation efficiency.

[0105] 10) Other strategies that can be established include task sorting priority scheduling, energy consumption optimization strategy, etc.

[0106] 4. Task execution monitor: monitors the execution status of tasks to ensure that tasks are executed as expected. If an exception occurs, the fault isolation mechanism will be triggered.

[0107] 5. Resource utilization optimization: According to the task execution situation, continuously adjust the resource allocation and scheduling strategy to optimize the resource utilization efficiency of the system.

[0108] 6. Task completion notifier: Once the task is completed, notify the relevant modules to update the task status, including data preservation and subsequent analysis.

[0109] 7. Scheduling log recorder: records the scheduling information of tasks for subsequent analysis and optimization.

[0110] 8. Performance Monitor: Real-time monitoring of GPU device performance indicators, such as computing power, temperature, memory usage, etc.

[0111] Optimization strategy module

[0112] In the intelligent data-aware GPU resource scheduling system, it is responsible for formulating task scheduling strategies to maximize system performance and resource utilization efficiency. This module consists of the following main components:

[0113] 1. Objective function definition: Define indicators to measure system performance, such as task completion time, resource utilization, water and electricity consumption, etc. These indicators will be used as optimization targets in the task scheduling process.

[0114] 2. Scheduling strategy formulation: formulate task scheduling strategies based on the objective function and real-time system status, involving decisions on task priority, resource allocation, task sequence, etc.

[0115] 3. Optimization algorithm: Select an appropriate optimization algorithm, such as greedy algorithm, genetic algorithm, etc., to search for the optimal solution in the scheduling strategy.

[0116] 4. Parameter Tuner: Adjust the parameters in the optimization algorithm to adapt to different tasks and system environments to achieve better performance.

[0117] 5. Strategy evaluation method: Evaluate and analyze the formulated scheduling strategy and predict its effect in the actual system through simulation or historical data analysis.

[0118] 6. Real-time status monitoring: monitor the real-time status of GPU devices, task execution, and data features provided by the data perception module to adjust strategies in real time.

[0119] 7. Adaptive mechanism: Adaptively adjust strategies based on real-time monitoring information to adapt to system changes and uncertainties.

[0120] 8. Policy executor: According to the formulated scheduling strategy, it executes the scheduling operation of tasks, allocates GPU resources, and arranges the order of task execution.

[0121] The above-mentioned resource scheduling equipment can be applied to cloud computing platforms to achieve efficient management and resource allocation of multi-tenant applications, and improve the performance and user experience of cloud services. Secondly, it is suitable for scientific research and simulation fields, optimizing GPU resource scheduling, accelerating complex computing tasks, and providing more accurate research results. In addition, it has application potential in the fields of deep learning and artificial intelligence, and can accelerate model training and reasoning through intelligent scheduling and allocation of GPU resources. In addition, it can be used for large-scale data processing to improve the efficiency of data processing and analysis, such as data mining, image processing, recommendation systems, etc. In addition, it is also suitable for virtualization technology and edge computing, realizing dynamic allocation and optimization of resources, and providing efficient edge computing capabilities.

[0122] Based on the solution provided by the above specific implementation method, the embodiment of the present application provides a resource scheduling device, the device includes a data perception module and a resource scheduling module, the resource scheduling module includes a scheduling policy maker, a resource allocator and a task queue manager; wherein,

[0123] The data perception module is used to extract data feature vectors of a task data set, wherein the task data set includes a task set to be assigned to a GPU device for processing and a data set corresponding to the task set, and the data feature includes a demand feature and an access feature of the task for data in the data set;

[0124] The scheduling strategy maker is used to analyze the data feature vector using a scheduling strategy model, and output a task scheduling strategy corresponding to the task data set; wherein the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples and GPU scheduling vector samples, and the scheduling features include resource allocation features and data processing features of the GPU device when executing task processing;

[0125] The resource allocator is used to determine the GPU device corresponding to each task in the task data set based on the task scheduling strategy;

[0126] The task queue manager is used to determine the execution order and priority of each task in the task data set based on the task scheduling strategy.

[0127] The task set may include at least one task. The data set includes data required in task processing or data that needs to be accessed.

[0128] For example, after the cloud computing platform receives a batch processing task, the resource scheduling device can analyze the batch processing tasks to be processed by the cloud computing platform, as well as the data required for task processing and the data to be accessed, and use the scheduling strategy model for analysis to output the task scheduling strategy corresponding to the task data set.

[0129] Data features may include demand features and access features of the data in the data set by the task. As shown in the above embodiment, demand features may include data scale, data dimension, data type complexity, data distribution uniformity, data sparsity, etc., and access features may include access frequency, access mode, number of accesses, etc. Parameter types of data features may be selected based on sample data analysis, such as the parameter types of data features may be selected by the above data perception module.

[0130] In some embodiments, the characteristic parameters in the data features may include at least one of data scale, data dimension, data type complexity, data distribution uniformity, data sparsity, data correlation, data redundancy, and computing accuracy requirements. It can be seen from the above embodiments that the selection of the above characteristic parameters fully considers the relationship between data characteristics and scheduling, and can further improve the performance, efficiency and reliability of the computing system. The meaning of each characteristic parameter can refer to the above embodiments and will not be repeated here.

[0131] The scheduling characteristics may at least include resource allocation characteristics and data processing characteristics of the GPU device when executing task processing. As shown in the above embodiment, resource allocation characteristics may include batch size, number of parallel computing threads, shared memory usage, parallel computing window, etc., and data processing characteristics may include calculation accuracy, instruction set selection, task allocation strategy, data storage format, data preprocessing parameters, etc. The parameter types of data characteristics may be selected based on sample data analysis, such as the parameter types of data characteristics may be selected by the above data perception module.

[0132] In some embodiments, the characteristic parameters of the scheduling characteristics may include one of batch size, number of parallel computing threads, parallel computing window, shared memory usage, computing precision selection, instruction set selection, task allocation strategy, data storage format, and data preprocessing parameters. It can be seen from the above embodiments that selecting the above characteristic parameters can fully characterize the processing performance of the GPU, thereby further improving the performance, efficiency and reliability of the computing system. The meaning of each characteristic parameter can refer to the above embodiments and will not be repeated here.

[0133] The scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples and GPU scheduling vector samples. For example, sample data can be extracted based on historical data, feature parameters in data features and scheduling features can be determined based on the sample data, and feature vectors can be constructed based on the feature parameters to obtain data feature vectors and GPU scheduling vectors, and then the data feature vectors and GPU scheduling vectors can be used to optimize the parameters in the scheduling algorithm to obtain a scheduling strategy model.

[0134] The data perception module can extract the sample data corresponding to each feature parameter based on the selected feature parameters and the sample data processing method to form a data feature vector sample and a GPU scheduling vector sample. Afterwards, the resource scheduling module can use the data feature vector sample and the GPU scheduling vector sample to optimize the parameters of the initially established scheduling algorithm to obtain a scheduling strategy model.

[0135] In some embodiments, the scheduling algorithm includes an objective function between a data feature vector and a GPU scheduling vector, and an optimal solution search algorithm for the objective function.

[0136] The objective function can be the above performance index function as Y represents performance indicators related to computing efficiency, such as floating-point operations per second (FLOPS), the inverse of training time (1 / T, T is training time), etc. The goal is to find the optimal P * , so that Y reaches its maximum value, that is

[0137] The optimal solution search algorithm may include a greedy algorithm, a genetic algorithm, etc., to search for the optimal solution to the objective function.

[0138] The data feature vector samples and the GPU scheduling vector samples can be input into the objective function to optimize the parameters of the optimal solution search algorithm to obtain the optimal value of the optimal solution search algorithm, thereby obtaining a scheduling strategy model.

[0139] By constructing an objective function that characterizes the relationship between the data feature vector and the GPU scheduling vector, and solving the objective function using the optimal solution search algorithm, the relationship between data features and scheduling can be fully considered, and the performance, efficiency and reliability of the computing system can be further improved.

[0140] In some embodiments, the scheduling policy model includes at least one of the following scheduling policy sub-models: a scheduling policy sub-model for batch size and data scale; a scheduling policy sub-model for the number of parallel computing threads and data scale; a scheduling policy sub-model for shared memory usage and data dimension; a scheduling policy sub-model for instruction set selection and data type complexity; a scheduling policy sub-model for task allocation strategy and data distribution uniformity; a scheduling policy sub-model for data storage format and data sparsity; a scheduling policy sub-model for parallel computing window and data correlation; a scheduling policy sub-model for data preprocessing parameters and data redundancy; a scheduling policy sub-model for calculation accuracy selection and calculation accuracy requirements. The specific implementation methods of the above sub-models can refer to the above embodiments and will not be repeated here.

[0141] By analyzing a large amount of sample data, extracting the correlation between different data features and scheduling features, constructing corresponding sub-models, and then constructing an overall scheduling strategy model, the scheduling strategy model can further accurately and fully consider the relationship between data features and scheduling, and further improve the performance, efficiency and reliability of the computing system.

[0142] In some embodiments, the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples, task feature vector samples, and GPU scheduling vector samples, wherein the feature parameters in the task feature vector samples include at least one of task computational complexity, task execution duration, task execution accuracy, and task execution recall. When constructing the scheduling strategy model, further considering the task characteristics can enable the scheduling strategy model to further accurately and fully consider the relationship between the data characteristics and the characteristics of the task itself and the scheduling, thereby further improving the performance, efficiency, and reliability of the computing system.

[0143] After determining the task scheduling strategy, the task queue manager and resource allocator can output the GPU device corresponding to each task in the task data set, as well as the execution order and priority of each task based on the task scheduling strategy, and then manage the queue of tasks to be executed, sort and schedule them according to the priority and characteristics of the tasks, and decide which task to assign to which GPU device to execute based on the data characteristics, computing requirements and GPU device status of the tasks.

[0144] Based on the above resource scheduling method, the utilization and performance of GPU resources can be improved, energy consumption and operation and maintenance costs can be reduced, and user experience can be improved. In the fields of cloud computing, scientific research, artificial intelligence, big data processing, etc., it can optimize resource scheduling, accelerate task execution, and provide efficient computing capabilities. It also helps to accelerate the process of scientific research and innovation and promote technological development. In addition, it has application potential in virtualization technology and edge computing. This application provides intelligent GPU resource management and scheduling, bringing more efficient, reliable and sustainable computing solutions to various fields.

[0145] In some embodiments, the resource scheduling module further includes a task execution monitor, a resource utilization optimization unit, a task completion notifier, a scheduling log recorder, and a performance monitor;

[0146] The task execution monitor is used to monitor the execution status of the task. If the execution status is abnormal, fault isolation is triggered for the abnormal task.

[0147] The resource utilization optimization unit is used to adjust the task scheduling strategy in real time according to the resource utilization information of the task and the performance index of the GPU device;

[0148] The task completion notifier is used to issue a task status update notification when the task is completed;

[0149] The scheduling log recorder is used to record task execution information and scheduling information;

[0150] The performance monitor is used to detect the performance indicators of the GPU device in real time.

[0151] Based on real-time data perception and prediction models, combined with scheduling strategies and optimization algorithms, the system's performance and efficiency are improved. Task priorities are adjusted, resources are reallocated, and faults are handled. Tasks are assigned to appropriate GPU devices, or appropriate K8S pod resources are created based on decision information to achieve efficient resource utilization and smooth task execution.

[0152] In some embodiments, the device further includes an optimization strategy module, and the optimization strategy module is used to optimize the scheduling strategy model using the task execution information and scheduling information recorded by the scheduling log recorder.

[0153] The optimization strategy module further analyzes data characteristics and constraints based on the collected task characteristics and data, designs objective functions and optimization algorithms, schedules and allocates resources, evaluates scheduling results, measures system performance, and makes adjustments and optimizations based on the evaluation results. Through this process, the optimization strategy module can intelligently decide on the allocation of GPU resources to maximize system performance and task execution efficiency. Continuously iterating the optimization strategy makes resource scheduling more intelligent, efficient, and adaptable to different tasks and data requirements.

[0154] The specific implementation methods of each embodiment of the resource scheduling device refer to the above content and will not be repeated here. Based on the above resource scheduling device, the utilization and performance of GPU resources can be improved, energy consumption and operation and maintenance costs can be reduced, and user experience can be improved. In the fields of cloud computing, scientific research, artificial intelligence, big data processing, etc., it can optimize resource scheduling, accelerate task execution, and provide efficient computing power. It also helps to accelerate the process of scientific research and innovation and promote technological development. In addition, it has application potential in virtualization technology and edge computing. This application provides intelligent GPU resource management and scheduling, bringing more efficient, reliable and sustainable computing solutions to various fields.

[0155] Based on the above device, the embodiment of this article provides a resource scheduling method. The order of steps listed in the embodiment is only one way of executing the steps among many steps, and does not represent the only execution order. When the actual system or device product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings. Specifically, the method is applied to the resource scheduling device, and the method includes:

[0156] S20: extracting a data feature vector of a task data set, wherein the task data set includes a task set to be assigned to a GPU device for processing and a data set corresponding to the task set, and the data feature includes a demand feature and an access feature of the task for data in the data set.

[0157] A task set may include at least one task. A data set includes data required or data that needs to be accessed during task processing.

[0158] For example, after the cloud computing platform receives a batch processing task, the resource scheduling device can analyze the batch processing tasks to be processed by the cloud computing platform, as well as the data required for task processing and the data to be accessed, and use the scheduling strategy model for analysis to output the task scheduling strategy corresponding to the task data set.

[0159] Data features may include demand features and access features of the data in the data set by the task. As shown in the above embodiment, demand features may include data scale, data dimension, data type complexity, data distribution uniformity, data sparsity, etc., and access features may include access frequency, access mode, number of accesses, etc. Parameter types of data features may be selected based on sample data analysis, such as the parameter types of data features may be selected by the above data perception module.

[0160] In some embodiments, the characteristic parameters in the data features may include at least one of data scale, data dimension, data type complexity, data distribution uniformity, data sparsity, data correlation, data redundancy, and computing accuracy requirements. It can be seen from the above embodiments that the selection of the above characteristic parameters fully considers the relationship between data characteristics and scheduling, and can further improve the performance, efficiency and reliability of the computing system. The meaning of each characteristic parameter can refer to the above embodiments and will not be repeated here.

[0161] S22: Analyze the data feature vector using a scheduling strategy model, and output the task scheduling strategy corresponding to the task data set; wherein the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples and GPU scheduling vector samples, and the scheduling characteristics include resource allocation characteristics and data processing characteristics of the GPU device when performing task processing.

[0162] The scheduling characteristics may at least include resource allocation characteristics and data processing characteristics of the GPU device when executing task processing. As shown in the above embodiment, resource allocation characteristics may include batch size, number of parallel computing threads, shared memory usage, parallel computing window, etc., and data processing characteristics may include calculation accuracy, instruction set selection, task allocation strategy, data storage format, data preprocessing parameters, etc. The parameter types of data characteristics may be selected based on sample data analysis, such as the parameter types of data characteristics may be selected by the above data perception module.

[0163] In some embodiments, the characteristic parameters of the scheduling characteristics may include one of batch size, number of parallel computing threads, parallel computing window, shared memory usage, computing precision selection, instruction set selection, task allocation strategy, data storage format, and data preprocessing parameters. It can be seen from the above embodiments that selecting the above characteristic parameters can fully characterize the processing performance of the GPU, thereby further improving the performance, efficiency and reliability of the computing system. The meaning of each characteristic parameter can refer to the above embodiments and will not be repeated here.

[0164] The scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples and GPU scheduling vector samples. For example, sample data can be extracted based on historical data, feature parameters in data features and scheduling features can be determined based on the sample data, and feature vectors can be constructed based on the feature parameters to obtain data feature vectors and GPU scheduling vectors, and then the data feature vectors and GPU scheduling vectors can be used to optimize the parameters in the scheduling algorithm to obtain a scheduling strategy model.

[0165] The data perception module can extract the sample data corresponding to each feature parameter based on the selected feature parameters and the sample data processing method to form a data feature vector sample and a GPU scheduling vector sample. Afterwards, the resource scheduling module can use the data feature vector sample and the GPU scheduling vector sample to optimize the parameters of the initially established scheduling algorithm to obtain a scheduling strategy model.

[0166] In some embodiments, the scheduling algorithm includes an objective function between a data feature vector and a GPU scheduling vector, and an optimal solution search algorithm for the objective function.

[0167] The objective function can be the above performance index function as Y represents performance indicators related to computing efficiency, such as floating-point operations per second (FLOPS), the inverse of training time (1 / T, T is training time), etc. The goal is to find the optimal P * , so that Y reaches its maximum value, that is

[0168] The optimal solution search algorithm may include a greedy algorithm, a genetic algorithm, etc., to search for the optimal solution to the objective function.

[0169] The data feature vector samples and the GPU scheduling vector samples can be input into the objective function to optimize the parameters of the optimal solution search algorithm to obtain the optimal value of the optimal solution search algorithm, thereby obtaining a scheduling strategy model.

[0170] By constructing an objective function that characterizes the relationship between the data feature vector and the GPU scheduling vector, and solving the objective function using the optimal solution search algorithm, the relationship between data features and scheduling can be fully considered, and the performance, efficiency and reliability of the computing system can be further improved.

[0171] In some embodiments, the scheduling policy model includes at least one of the following scheduling policy sub-models: a scheduling policy sub-model for batch size and data scale; a scheduling policy sub-model for the number of parallel computing threads and data scale; a scheduling policy sub-model for shared memory usage and data dimension; a scheduling policy sub-model for instruction set selection and data type complexity; a scheduling policy sub-model for task allocation strategy and data distribution uniformity; a scheduling policy sub-model for data storage format and data sparsity; a scheduling policy sub-model for parallel computing window and data correlation; a scheduling policy sub-model for data preprocessing parameters and data redundancy; a scheduling policy sub-model for calculation accuracy selection and calculation accuracy requirements. The specific implementation methods of the above sub-models can refer to the above embodiments and will not be repeated here.

[0172] By analyzing a large amount of sample data, extracting the correlation between different data features and scheduling features, constructing corresponding sub-models, and then constructing an overall scheduling strategy model, the scheduling strategy model can further accurately and fully consider the relationship between data features and scheduling, and further improve the performance, efficiency and reliability of the computing system.

[0173] In some embodiments, the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples, task feature vector samples, and GPU scheduling vector samples, wherein the feature parameters in the task feature vector samples include at least one of task computational complexity, task execution duration, task execution accuracy, and task execution recall. When constructing the scheduling strategy model, further considering the task characteristics can enable the scheduling strategy model to further accurately and fully consider the relationship between the data characteristics and the characteristics of the task itself and the scheduling, thereby further improving the performance, efficiency, and reliability of the computing system.

[0174] S24: Determine the GPU device corresponding to each task in the task data set, and the execution order and priority of each task based on the task scheduling strategy.

[0175] After determining the task scheduling strategy, the task queue manager and resource allocator can output the GPU device corresponding to each task in the task data set, as well as the execution order and priority of each task based on the task scheduling strategy, and then manage the queue of tasks to be executed, sort and schedule them according to the priority and characteristics of the tasks, and decide which task to assign to which GPU device to execute based on the data characteristics, computing requirements and GPU device status of the tasks.

[0176] Based on the above resource scheduling method, the utilization and performance of GPU resources can be improved, energy consumption and operation and maintenance costs can be reduced, and user experience can be improved. In the fields of cloud computing, scientific research, artificial intelligence, big data processing, etc., it can optimize resource scheduling, accelerate task execution, and provide efficient computing capabilities. It also helps to accelerate the process of scientific research and innovation and promote technological development. In addition, it has application potential in virtualization technology and edge computing. This application provides intelligent GPU resource management and scheduling, bringing more efficient, reliable and sustainable computing solutions to various fields.

[0177] In some other embodiments, the method may further include:

[0178] Monitor the execution status of the task and perform any of the following actions:

[0179] If the execution status is abnormal, fault isolation is triggered for the abnormal task;

[0180] Adjust the task scheduling strategy in real time based on the task resource utilization information and GPU device performance indicators;

[0181] After the task is completed, the task execution information and the scheduling information are stored to optimize the scheduling strategy model using the task execution information and the scheduling information.

[0182] Scheduling decisions can be based on real-time data perception and prediction models, combined with scheduling strategies and optimization algorithms to improve system performance and efficiency, and adjust task priorities, reallocate resources, handle faults, etc., assign tasks to appropriate GPU devices, or create appropriate K8S pod resources based on decision information to achieve efficient resource utilization and smooth task execution.

[0183] The optimization strategy module further analyzes data characteristics and constraints based on the collected task characteristics and data, designs objective functions and optimization algorithms, schedules and allocates resources, evaluates scheduling results, measures system performance, and makes adjustments and optimizations based on the evaluation results. Through this process, the optimization strategy module can intelligently decide on the allocation of GPU resources to maximize system performance and task execution efficiency. Continuously iterating the optimization strategy makes resource scheduling more intelligent, efficient, and adaptable to different tasks and data requirements.

[0184] The following is an embodiment of the method of the present application, and the specific steps of the present application are as follows:

[0185] S1-1: Collect features of training data for the task - static analysis.

[0186] Through static analysis of task code and data dependencies, we analyze the data access pattern, data size, computational density and other characteristics of the task. We obtain static features through code parsing, abstract syntax tree analysis, dependency graph construction and other technologies.

[0187] S1-2: Collect features of training data for the task - runtime analysis.

[0188] During the task execution process, by monitoring the running status and behavior of the task, the task's memory access mode, data locality, access frequency, data transmission volume, etc. can be monitored through performance monitoring tools, hardware performance counters, memory access tracking, etc. to obtain richer runtime characteristics.

[0189] S1-3: Training dataset collection and sampling.

[0190] For tasks with large-scale data sets, data sampling and sampling are performed through random sampling, stratified sampling, time interval sampling and other methods to obtain representative data samples and perform feature analysis on them.

[0191] S1-4: User feedback and learning.

[0192] Based on user feedback and learning algorithms, the characteristics of the task are dynamically perceived and adjusted. Through user-labeled data sets, user behavior records, etc., the characteristic preferences and requirements of the task can be understood, and personalized task feature perception can be performed.

[0193] S2: Data analysis.

[0194] The collected data needs to be analyzed and processed to extract useful data perception information, including statistics and analysis of data locality, memory access mode, data size, and data dependency, generating data feature descriptions of tasks, and persisting the perceived information to the database.

[0195] S3: Data feature modeling.

[0196] According to the data perception information obtained by S2 analysis, the corresponding data model is established, including the data locality model of the task, the memory access demand model, the data transmission model, etc. These models can be used for subsequent resource scheduling decisions and optimization algorithms.

[0197] S4: Resource Scheduling Module - Resource Management Metric Information Acquisition

[0198] The resource scheduling module needs to manage and monitor the available GPU resources. This includes obtaining information about GPU devices (such as model, performance indicators, usage status, etc.), tracking the utilization of GPU resources, and recording the health status of GPU devices. Resource management functions can be implemented by interacting with the underlying system or scheduler.

[0199] S5: Scheduling strategy

[0200] Implement a scheduling strategy that adapts to task characteristics and system requirements. A matching algorithm based on task characteristics selects the appropriate GPU device according to the task's data access mode and memory access requirements; including an optimization algorithm based on system performance, which performs dynamic scheduling and load balancing based on the GPU device's load and resource utilization.

[0201] S6: Scheduling Decision

[0202] The resource scheduling module makes reasonable scheduling decisions based on the system status and task characteristics, including adjustment of task priorities, reallocation of resources, and fault handling. Scheduling decisions can be based on real-time data perception and prediction models, combined with scheduling strategies and optimization algorithms to improve system performance and efficiency. According to the scheduling strategy, tasks are assigned to appropriate GPU devices or appropriate K8Spod resources are created based on decision information by establishing task queues and maintaining task priorities. Efficient resource utilization and smooth task execution are achieved.

[0203] S7: Optimization strategy module

[0204] The optimization strategy module analyzes data characteristics and constraints based on the collected task characteristics and data, designs objective functions and optimization algorithms, schedules and allocates resources, evaluates scheduling results, measures system performance, and makes adjustments and optimizations based on the evaluation results. Through this process, the optimization strategy module can intelligently decide on the allocation of GPU resources to maximize system performance and task execution efficiency. Continuously iterating the optimization strategy makes resource scheduling more intelligent, efficient, and adaptable to different tasks and data requirements.

[0205] This application introduces a data perception module that can collect and analyze the data requirements and access patterns of tasks, and provide data feature information for resource scheduling. Secondly, the GPU status monitoring function is implemented to monitor the GPU utilization, temperature, memory usage and other indicators in real time to provide real-time data support for scheduling decisions. Third, a fault isolation mechanism is proposed to detect and isolate GPU faults and ensure the stability and reliability of the entire system. An optimization strategy module is designed to define the objective function and algorithm, and intelligently schedule and allocate GPU resources according to task characteristics and data perception results to improve system performance and task execution efficiency. Finally, dynamic scheduling and adaptability functions are implemented to dynamically adjust resource allocation strategies according to factors such as system load, task requirements and data characteristics to adapt to different workloads and environmental changes.

[0206] Compared with the prior art, the embodiments of the present application have the advantages and effects of improving task execution efficiency, optimizing resource utilization, improving overall system performance, and adapting to diverse tasks and data. This will provide a more efficient, scalable and flexible GPU resource management and scheduling solution for cloud-native applications:

[0207] 1. Improve task execution efficiency. This application can intelligently allocate GPU resources through data-aware scheduling and optimization strategy modules, schedule tasks according to their data requirements, and maximize the use of GPU computing power. This can significantly improve task execution efficiency and speed up task completion time.

[0208] 2. Optimize resource utilization. Traditional GPU resource scheduling methods often fail to fully utilize GPU resources, resulting in resource waste and low efficiency. This application uses a data perception module to perceive task characteristics and data characteristics, reasonably allocate GPU resources, improve resource utilization, and thus achieve better performance.

[0209] 3. Improve the overall performance of the system. The intelligent data-aware GPU resource scheduling device and method can be combined with the cloud-native scheduling framework to achieve dynamic resource allocation and task scheduling. By optimizing the objective function defined by the strategy module, the overall system performance can be maximized while meeting the task constraints. This can improve the overall performance and user experience of cloud-native applications.

[0210] 4. Adapting to diverse tasks and data, this application fully considers the characteristics of the task's access mode to the data set, data size, and data type. This enables the system to adapt to different types of tasks and data, perform targeted resource scheduling, and provide better adaptability and flexibility.

[0211] This embodiment provides a computer device, whose internal structure diagram can be as follows: Figure 4 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection.

[0212] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0213] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0214] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0215] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0216] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0217] It should also be understood that in the embodiments of this article, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0218] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.

[0219] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0220] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0221] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0222] Specific embodiments are used in this article to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for those skilled in the art, according to the ideas of this article, there will be changes in the specific implementation methods and application scopes. In summary, the content of this application should not be understood as a limitation on this article.

Claims

1. A resource scheduling method, characterized in that: Applied to a resource scheduling device, the method comprises: Extracting a data feature vector of a task data set, wherein the task data set includes a task set to be assigned to a GPU device for processing and a data set corresponding to the task set, and the data feature includes a demand feature and an access feature of the task to the data in the data set; The data feature vector is analyzed by using a scheduling strategy model, and a task scheduling strategy corresponding to the task data set is output; wherein the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm by using data feature vector samples and GPU scheduling vector samples, and the scheduling features include resource allocation features and data processing features of the GPU device when executing task processing; The GPU device corresponding to each task in the task data set, as well as the execution order and priority of each task are determined based on the task scheduling strategy.

2. The method according to claim 1, characterized in that The characteristic parameters of the data features include at least one of data scale, data dimension, data type complexity, data distribution uniformity, data sparsity, data correlation, data redundancy, and calculation accuracy requirements.

3. The method according to claim 1, characterized in that The characteristic parameters in the scheduling characteristics include at least one of batch size, number of parallel computing threads, parallel computing window, shared memory usage, calculation accuracy selection, instruction set selection, task allocation strategy, data storage format, and data preprocessing parameters.

4. The method according to claim 1, characterized in that: The scheduling strategy model includes at least one of the following scheduling strategy sub-models: Scheduling policy sub-models for batch size and data scale; Scheduling policy sub-models for number of parallel computing threads and data scale; Scheduling policy sub-models for shared memory usage and data dimension; Scheduling policy sub-models for instruction set selection and data type complexity; Scheduling policy sub-models for task allocation strategy and data distribution uniformity; Scheduling policy sub-models for data storage format and data sparsity; Scheduling policy sub-models for parallel computing windows and data relevance; Scheduling strategy sub-model of data preprocessing parameters and data redundancy; Scheduling strategy sub-model of computing accuracy selection and computing accuracy requirements.

5. The method according to claim 1, characterized in that The scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples, task feature vector samples and GPU scheduling vector samples, wherein the feature parameters in the task feature vector samples include at least one of task calculation complexity, task execution time, task execution accuracy and task execution recall rate.

6. The method according to any one of claims 1 to 5, characterized in that: The scheduling algorithm includes an objective function between a data feature vector and a GPU scheduling vector, and an optimal solution search algorithm for the objective function.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Monitor the execution status of the task and perform any of the following actions: If the execution status is abnormal, fault isolation is triggered for the abnormal task; Adjust the task scheduling strategy in real time based on the task resource utilization information and GPU device performance indicators; After the task is completed, the task execution information and the scheduling information are stored to optimize the scheduling strategy model using the task execution information and the scheduling information.

8. A resource scheduling device, characterized in that: The device includes a data perception module and a resource scheduling module, wherein the resource scheduling module includes a scheduling strategy maker, a resource allocator and a task queue manager; wherein, The data perception module is used to extract data feature vectors of a task data set, wherein the task data set includes a task set to be assigned to a GPU device for processing and a data set corresponding to the task set, and the data feature includes a demand feature and an access feature of the task for data in the data set; The scheduling strategy maker is used to analyze the data feature vector using a scheduling strategy model, and output a task scheduling strategy corresponding to the task data set; wherein the scheduling strategy model refers to a model constructed after optimizing the scheduling algorithm using data feature vector samples and GPU scheduling vector samples, and the scheduling features include resource allocation features and data processing features of the GPU device when executing task processing; The resource allocator is used to determine the GPU device corresponding to each task in the task data set based on the task scheduling strategy; The task queue manager is used to determine the execution order and priority of each task in the task data set based on the task scheduling strategy.

9. The device according to claim 8, characterized in that The resource scheduling module also includes a task execution monitor, a resource utilization optimization unit, a task completion notifier, a scheduling log recorder, and a performance monitor; The task execution monitor is used to monitor the execution status of the task. If the execution status is abnormal, fault isolation is triggered for the abnormal task. The resource utilization optimization unit is used to adjust the task scheduling strategy in real time according to the resource utilization information of the task and the performance index of the GPU device; The task completion notifier is used to issue a task status update notification when the task is completed; The scheduling log recorder is used to record task execution information and scheduling information; The performance monitor is used to detect the performance indicators of the GPU device in real time.

10. The device according to claim 9, characterized in that The device also includes an optimization strategy module, which is used to optimize the scheduling strategy model by using the task execution information and scheduling information recorded by the scheduling log recorder.

Images