Heterogeneous platform computing task scheduling method and system based on machine learning algorithm and application

Abstract

The invention discloses a heterogeneous platform calculation task scheduling method based on a machine learning algorithm. The scheduling method comprises the following steps of 1, analyzing and collecting static characteristics and dynamic characteristics of a data set program and hardware characteristics of an operation platform; 2, selecting a proper machine learning scheduling model and optimizing hyper-parameters, and evaluating the performance of the scheduling model through a plurality of indexes; and 3, selecting optimal hardware equipment during operation according to a model prediction result to realize efficient task scheduling. The invention also discloses a scheduling system for realizing the scheduling method and a corresponding application, and has a wide application scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of heterogeneous computing and performance optimization, and relates to a method, system and application for scheduling computing tasks on a heterogeneous platform based on a machine learning algorithm. Background Art

[0002] Heterogeneous computing refers to a computing method in which multiple computing units with different instruction set architectures and different architectures are combined in the same system to collaborate in computing. The development of heterogeneous computing is driven by real-world workloads. In recent years, a new generation of artificial intelligence technology has also emerged worldwide, bringing new demands and high standards for computer computing power. General-purpose processors (CPUs) have been widely used for many years. Researchers have improved their performance bottlenecks by increasing the clock frequency and number of computing units of the CPU, but in the face of huge amounts of data, the computing power of the CPU is relatively low. Graphics processors (GPUs) are usually designed for high-performance graphics processing and general computing tasks. Their computing power is usually much higher than that of CPUs because they have more parallel processing units, making them ideal for big data and simple calculations. AI-specific chips (such as NPU, TPU, etc.) are products optimized specifically for artificial intelligence applications. They usually have higher energy efficiency and higher computing power when facing artificial intelligence tasks. In addition to the common chips mentioned above, there are also some dedicated accelerator cards and processors that can be optimized for specific fields to achieve higher performance. If the traditional single architecture model can be changed and the advantages of different architectures can be brought into play, it will bring higher computing power, higher throughput, lower latency, lower energy consumption and lower costs. Specifically, heterogeneous platforms should be able to leverage the flexibility of the CPU in general computing, combined with the high computing power of the GPU, and the ability of special domain processors to handle domain-specific tasks, to rationally allocate computing resources and improve overall performance.

[0003] As early as the beginning of the 20th century, developers realized that the completion of general parallel computing tasks required a unified standard, and OpenCL (Open Computing Language) was proposed as a framework for writing programs for heterogeneous platforms. As of July 2024, OpenCL is still widely used, and dozens of large hardware vendors including NVIDIA, Intel, AMD, etc. have designed a set of OpenCL implementations in accordance with the OpenCL standard. The application scenarios of OpenCL also involve high-performance computing, artificial intelligence, image processing and other fields. Moreover, with the update of the OpenCL specification, the relevant implementations of various companies will be quickly updated and released. The advantages of OpenCL lie in its cross-platform support, openness and broad community support. In terms of cross-platform support, OpenCL can run on a variety of devices and platforms, supports multiple types of processors including NVIDIA, Intel, ARM, and can support parallel code running on the CPU. At the same time, it also has a unique parallel task execution mode, which can better support heterogeneous computing. The popularity of OpenCL is closely related to its openness: any company can be invited to join Khronos, contribute to and influence the formulation of its specifications; any company can implement the Khronos specifications, and the participating implementers can obtain trademark licenses for implementations that meet the requirements; developers can freely use the implementation of the Khronos specifications. In addition, OpenCL also supports multiple programming languages, including C, C++, Java, etc., and has extensive community support and resources. In summary, OpenCL is a cross-platform, open source, highly flexible and scalable heterogeneous computing framework, and there is also a lot of room for performance optimization.

[0004] In the heterogeneous system set by OpenCL, the device model consists of a host machine (usually a CPU) and several hardware devices in the system (which can be GPUs or CPUs). Tasks running on the device are called OpenCL kernel tasks, which are uniformly managed by the runtime scheduler running on the host. Since there may be a variety of hardware devices on the platform, the scheduling of multiple kernels of multiple applications on the platform is a very important and manageable performance issue. In previous studies, although solutions have been proposed for scheduling on heterogeneous systems, these solutions have the following problems:

[0005] 1. When considering the characteristics of program code, the prior art does not fully analyze the dynamic characteristics of the application program, nor does it consider the characteristics of the hardware platform, making the model not portable.

[0006] 2. The scheduling strategies used by some existing technologies are relatively simple, and only include traditional heuristic algorithms such as FIFO and priority FIFO. The algorithms do not consider the compile-time characteristics of the program or the processing characteristics of the processor, and cannot bring about significant performance improvements.

[0007] 3. The existing technology does not combine the model with the open source and widely used OpenCL runtime, and does not provide users with a convenient scheduler interface. Summary of the invention

[0008] In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide a method, system and application for scheduling computing tasks on heterogeneous platforms based on machine learning algorithms. The present invention provides an OpenCL runtime system that can run on heterogeneous platforms, combine the dynamic and static characteristics of programs and schedule computing tasks based on machine learning algorithms.

[0009] The present invention provides a method for scheduling computing tasks on heterogeneous platforms based on a machine learning algorithm, the scheduling method comprising the following steps:

[0010] Step 1: Analyze and collect the static and dynamic features of the data set program and the hardware features of the running platform;

[0011] Step 2: Select a suitable machine learning scheduling model and optimize hyperparameters, and evaluate the performance of the scheduling model through multiple indicators;

[0012] Step 3: Select the best hardware device at runtime based on the model prediction results to achieve efficient task scheduling.

[0013] In step 1, the static features of the data set program include: the number of instructions of different code types, the number of basic blocks, etc.; the dynamic features of the data set program include: input data size, loop upper bound, memory access, etc.; the hardware features of the operating platform include CPU features and GPU features, etc.; and / or,

[0014] The CPU characteristics include the number of CPU cores, BogoMIPS, the size of the first-level data cache, the size of the first-level instruction cache, the size of the second-level cache and the size of the third-level cache, etc.; the GPU characteristics include the video memory clock frequency, the video memory bus width, the peak video memory bandwidth, the total amount of video memory, the amount of shared memory available for each thread block and the number of threads contained in each thread bundle, etc.

[0015] The BogoMIPS is a measure of Linux processor speed and can show whether the system options are in optimal performance when the system is booted.

[0016] Step one further comprises the following steps:

[0017] Step 1.1, compile the kernel code in the benchmark suite through the LLVM tool chain; the benchmark suite is pre-collected data that is widely used in the field of heterogeneous computing;

[0018] Step 1.2: Perform static feature analysis on the LLVM intermediate representation compiled in step 1.1 to obtain static features including the number of intermediate representation (IR) instruction classifications; estimate dynamic features including loop upper bounds and memory accesses by constructing LLVMPass;

[0019] The LLVM Pass refers to the compilation strategy or optimization process in LLVM, which acts on LLVM IR and is used to optimize the program at compile time, link time, runtime, and "idle" time to improve the performance and efficiency of the program.

[0020] and / or,

[0021] Use bash commands to extract CPU features in hardware, and use CUDA interface functions to extract GPU features in hardware;

[0022] Step 1.3: Perform feature engineering on the static features, dynamic features, CPU features, and GPU features obtained in step 1.2.

[0023] After the static feature analysis of the compiled LLVM intermediate representation, the type and number of statements and expressions, control flow structure and function call chain information are obtained by parsing the program abstract syntax tree;

[0024] The upper bound of the loop is estimated through LoopBoundAnalysis in LLVM Pass, and AccessMemoryAnalysis identifies the memory access pattern and estimates the memory access amount;

[0025] and / or,

[0026] The feature engineering process includes: data preprocessing and feature selection; the data preprocessing includes introducing dummy variables into hardware features and standardizing and normalizing the features; the feature selection refers to using correlation analysis method, presetting screening thresholds, screening features according to the thresholds, and retaining the best feature subset.

[0027] Step 2 further includes the following steps:

[0028] Step 2.1, select a suitable machine learning model based on the target problem; the machine learning model includes linear regression model, decision tree regressor, support vector machine regressor, K nearest neighbor regressor, random forest regressor, AdaBoost regressor, gradient boosted random forest regressor, Bagging regressor, ExtraTree regressor;

[0029] Step 2.2: Use grid search and / or random search to select the hyperparameters that achieve the best performance.

[0030] Step 2.3: Evaluate the performance of the machine learning model to obtain the final machine learning model, i.e., the scheduling model used in step 3; the evaluation indicators include comparative mean square error, mean absolute error, R squared, and adjusted R squared.

[0031] Step three further includes the following steps:

[0032] Step 3.1: During the pre-compilation phase of the OpenCL runtime, add a bash script to run the analysis program for the kernel code and hardware platform;

[0033] Step 3.2, obtain the static features, dynamic feature estimation and hardware platform features of the kernel code, input the trained scheduling model, and obtain the scheduling prediction results;

[0034] Step 3.3: Before creating the command object queue, select a hardware device according to the inference result of the scheduling model and execute the scheduling strategy.

[0035] The scheduling strategy is based on a trained scheduling model, using prediction results generated by multiple machine learning algorithms to select the best device from multiple hardware devices to perform the computing task; and / or,

[0036] Providing a scheduling interface by extending the OpenCL API allows users to select a custom scheduler and scheduling strategy to meet the performance requirements of specific applications; and / or,

[0037] Perform offline training on computing tasks on heterogeneous platforms, collect benchmark data, and generate feature subsets for training; and / or,

[0038] The feature subset is used to perform hyperparameter optimization and evaluation on multiple candidate models to select the optimal scheduling model.

[0039] The present invention also provides a scheduling system for implementing the above scheduling method, the scheduling system comprising: a data collection module, a feature engineering module, a model training and selection module, a runtime scheduling module, and a scheduling interface module;

[0040] The data collection module is responsible for collecting static features, dynamic features and hardware features of the program to provide basic data for model training and scheduling decisions;

[0041] The feature engineering module processes and optimizes the collected feature data to improve the training effect and generalization ability of the model;

[0042] The model training and selection module uses a machine learning algorithm to perform model training, optimize hyperparameters, select the best model and evaluate its performance;

[0043] The runtime scheduling module selects the most suitable hardware device for each computing task according to the trained model and real-time feature data when the task is running;

[0044] The scheduling interface module provides users with a flexible interface and extended functions, allows the selection of different scheduling strategies and models, and ensures compatibility with the existing OpenCL API.

[0045] The present invention also provides the above-mentioned scheduling method, or the application of the above-mentioned scheduling system in artificial intelligence model training and reasoning, big data efficient processing and analysis, graphics and image rendering processing, etc.

[0046] For the scenarios of artificial intelligence model training and reasoning, during the model training and reasoning process, the method of the present invention can analyze the computing requirements of the task and the characteristics of the hardware platform, and dynamically select the best CPU, GPU or other dedicated accelerators to optimize the training speed and reasoning performance and improve resource utilization.

[0047] For the scenario of efficient processing and analysis of big data, the method of the present invention can intelligently analyze the computing requirements of large-scale data processing and match the most suitable hardware resources, such as high-performance computing clusters, to achieve efficient processing and analysis of data and reduce processing delays and power consumption.

[0048] For the scenario of graphics and image rendering processing, in graphics rendering and image processing tasks, the method of the present invention can intelligently schedule tasks to the most suitable graphics processor or dedicated image processing unit according to the complexity of the tasks and the parallel processing capabilities of the hardware, so as to accelerate the speed of graphics rendering and image processing and improve the user experience.

[0049] The present invention designs a heterogeneous platform scheduling method based on machine learning, including processes such as extracting program features, training machine learning models, and designing OpenCL runtime scheduling API, which brings beneficial effects to program performance on heterogeneous platforms.

[0050] From the design of the method of the present invention, the beneficial effects mainly include but are not limited to the following aspects:

[0051] 1. Integrity, reliability and representativeness of features: The present invention has done a series of work in the data set collection stage to ensure that the data is complete and reliable. Compared with previous technologies, the influence of platform hardware features and dynamic features is considered, and an approximate estimate of dynamic features is obtained through the latest technology. In view of the data missing caused by the hardware itself, feature engineering methods are used to deal with it. Finally, the present invention screens variables through correlation analysis. The correlation coefficient between features is calculated to determine the degree of correlation between two features. A negative correlation coefficient indicates that the two features have a complementary relationship, a positive correlation coefficient indicates that the two features have a substitution relationship, and a correlation coefficient of 0 indicates that there is no correlation between the two. If the correlation between two features is strong, just keep one of the features. Therefore, the absolute value of the correlation coefficient is taken here, and any feature of the two features with an absolute value of the correlation coefficient greater than 90% is eliminated to obtain a more suitable feature subset.

[0052] 2. Model accuracy: In the model selection stage, the present invention considers as many regression models as possible for trial, and also performs hyperparameter estimation, model evaluation, etc. according to the commonly used machine learning process. Compared with the heuristic algorithms commonly used in previous technologies, and the OpenCL runtime itself does not consider the scheduling scheme, the present invention provides effective prediction for program scheduling on heterogeneous platforms.

[0053] 3. Interface consistency and scalability: This invention allows users to select custom schedulers and scheduling policies in the OpenCL runtime by adding extensions to meet the performance requirements of specific applications. The new extension is designed as a flexible interface to support the addition of more schedulers and scheduling policies while maintaining compatibility with the existing OpenCL API.

[0054] From the test data, the AdaBoost regressor, gradient boosting random forest regressor and ExtraTree regressor used in the present invention achieved 73%, 78% and 86% accuracy respectively, that is, more than 70% of the kernels were scheduled to the appropriate hardware devices relative to the perfect situation. Compared with the running results of OpenCL runtime itself without setting the scheduler, adding these schedulers brought 17%, 14% and 35% performance improvement including shortening the running time. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 It is a system architecture diagram of the offline training phase of the present invention.

[0057] Figure 2 It is a system architecture diagram of the runtime reasoning phase of the present invention.

[0058] Figure 3 It is a schematic diagram of the program analysis and feature collection process of the present invention.

[0059] Figure 4 It is a schematic diagram of the model training and model selection process of the present invention.

[0060] Figure 5 It is a schematic diagram of the process of adding an OpenCL runtime scheduling interface in the present invention.

[0061] Figure 6 It is a reference diagram for implementing embodiment 1 of the present invention.

[0062] Figure 7 It is a reference diagram for implementing embodiment 2 of the present invention.

[0063] Figure 8 It is a reference diagram for implementing embodiment 3 of the present invention.

[0064] Fig. 9 It is a reference diagram for implementing embodiment 4 of the present invention.

[0065] Fig.10 It is a reference diagram for implementing embodiment 5 of the present invention. DETAILED DESCRIPTION

[0066] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0067] The present invention provides a method for scheduling computing tasks on a heterogeneous platform based on a machine learning algorithm. The first technical problem solved by the present invention is the acquisition, screening and application of program features. Most of the prior art only considers the static features of the program in this part, such as the number of instructions of different types of code, the number of basic blocks, etc. These static features cannot fully represent the features of the application program, and only using static features for scheduling will obviously lead to prediction errors. The dynamic features of the program refer to the features that can be obtained when the program is running, such as the input data size, the upper bound of the loop, the amount of memory access, etc. The dynamic features can only be accurately obtained at runtime. If the dynamic feature data can only be obtained after running in the scheduling strategy, the scheduler overhead will be greatly increased, and the optimal device mapping will not always be generated. Therefore, the present invention focuses on how to obtain an approximate estimate of the dynamic features through an estimation method. In addition, the performance of the program running is also related to the features of the hardware device. The prior art does not pay attention to the hardware features of the training platform when training the model, and the obtained model will most likely not be transferable. Therefore, the present invention will also collect as many hardware features as possible and perform feature engineering on all features to prepare for subsequent model training. In this part, the present invention solves the problem that the prior art does not fully analyze the dynamic characteristics of the application program and does not consider the characteristics of the hardware platform when considering the characteristics of the program code, which makes the model not portable.

[0068] In addition, the present invention focuses on the reliability of the scheduler algorithm. The scheduling algorithms used by different prior arts can be divided into two categories, namely heuristic algorithms and machine learning algorithms. In general, the effect of heuristic algorithms is relatively poor, while fewer prior arts use machine learning algorithms, but the overall process still has room for optimization. For example, AUGONNET [1] et al. designed STARPU, a runtime system whose main goal is to provide a convenient way for numerical kernel designers to generate parallel tasks on heterogeneous hardware and provide an interface for developing and adjusting scheduling algorithms. It developed several strategies that can be seamlessly selected at runtime, but the scheduling strategy is relatively simple, including only FIFO, priority FIFO, etc. The scheduling algorithm does not consider the compile-time characteristics of the program, nor the processing characteristics of the processor, and cannot bring significant performance improvements. WEN [2]et al. proposed a runtime framework that schedules multi-user OpenCL tasks to the devices that best suit them in a CPU / GPU system. A machine learning-based prediction model is used at runtime to detect whether to merge OpenCL kernels or schedule them individually to the most appropriate device without prior analysis. This study has a certain degree of performance improvement over the state-of-the-art work at the time, but it did not fully analyze the dynamic characteristics of the application and did not consider the characteristics of the hardware platform, making the model not portable. At the same time, there is room for optimization in the model selection and model training process. Like all machine learning tasks, different models show different performance on different data sets and tasks. Choosing the right model is crucial to the final prediction accuracy. For example, in some cases, a decision tree model may be more effective than a neural network, while in other cases it may be the opposite. At the same time, most machine learning algorithms have some hyperparameters that need to be manually set, such as learning rate, tree depth, regularization parameter, etc. The selection of these hyperparameters has an important impact on the performance and convergence speed of the model. Incorrect hyperparameter selection may lead to overfitting, underfitting, or unstable training of the model. In summary, correctly selecting the model, optimizing the hyperparameters, and evaluating the model are key steps to ensure the success of machine learning tasks. These steps need to combine domain knowledge, experimental design, and data analysis skills to ensure that the final selected model can perform well in practical applications. This part solves the problem that the scheduling strategies used in some existing technologies are relatively simple. The feature engineering mentioned in the first technical problem also has a great impact on the accuracy of the model. The present invention also adds additional hardware features, resulting in hardware-related missing data values ​​for each piece of data. Appropriate data preprocessing methods are needed to help the model better understand and utilize classification information, thereby improving the performance and generalization ability of the model.

[0069] The third core problem solved by the present invention is how to combine the scheduling model with an open source, complete and influential OpenCL runtime to provide users with a convenient scheduling selection interface. POCL (Portable Computing Language) is an open source project that aims to provide a cross-platform OpenCL implementation, so that OpenCL can be used for parallel computing on different hardware. It is also an OpenCL implementation with a wide range of users. Therefore, POCL will serve as the basis of the open source tool framework of the present invention. The present invention will add reasonable extensions to POCL, allowing users to select customized schedulers and scheduling strategies to meet the performance requirements of specific applications. At the same time, the new extensions should provide flexible interfaces so that more schedulers and scheduling strategies can be easily added in the future, maintaining compatibility with the existing OpenCL API. This part solves the problem that the prior art does not combine the model with the open source and widely used OpenCL runtime, and does not provide users with a convenient scheduler interface.

[0070] The present invention proposes a heterogeneous platform scheduling method based on machine learning to complete the scheduling tasks on the heterogeneous platform, bringing load operation performance improvement to the entire system. When facing a simple computationally intensive computing task, scheduling the task to the GPU will be beneficial to the entire system. Conversely, when the computing task is adapted to the CPU, scheduling the task to the CPU will be more beneficial. However, the current OpenCL implementation does not provide a scheduling interface, and the kernel task will be scheduled to which device will be specified by the user or randomly assigned to an idle device. The scheduling method proposed in the present invention will give device recommendations through an efficient and reliable scheduling algorithm, and provide the prediction results to the user with a suitable interface to guide the adaptation of the computing task to the hardware device and improve the performance of the overall system. In terms of feature extraction, the present invention will analyze the static features of the program, estimate the dynamic features of the program, and collect the hardware platform features, and perform feature engineering based on these features to prepare for subsequent model training. The goal of the machine learning model is to predict the execution time of a computing task kernel. The present invention will apply a complete machine learning training process to complete model selection, hyperparameter selection, model evaluation, etc. Through the label predicted by the model, the scheduler selects the best available device for the kernel, and this prediction result will be provided to POCL before running. The present invention finally extends the API of POCL to support optional schedulers and scheduling strategies.

[0071] 5.1 System Architecture

[0072] The architecture diagram of the present invention is as follows Figure 1 , Figure 2As shown. In the offline training stage, the present invention first collects the benchmark test suites widely used in the field of heterogeneous computing, and runs the OpenCL kernel code of the test case on the GPU and CPU respectively, and counts its running time as the target feature of model training. Next, the present invention converts the OpenCL kernel code into LLVM IR with the help of the LLVM tool to complete the static feature analysis. For the dynamic features of the program that can only be determined after running, the present invention uses the Libtooling tool to perform abstract syntax tree matching, and completes the approximate estimation of the dynamic features of the program by building LLVM Pass. In addition, the present invention also collects the hardware device features of the training platform as model input. After that, a suitable feature subset is obtained through feature engineering, a suitable regression model and a suitable hyperparameter are selected for model training, and the results of model evaluation are given. In the runtime reasoning stage, when the user gives an application on a heterogeneous platform, the present invention will complete the kernel task feature analysis work in the pre-compilation part of the OpenCL runtime POCL, obtain the static features of the kernel code, the dynamic feature estimation and the hardware features of the platform in the same way, and use the trained model to obtain the reasoning result. When the user does not specify a running device and starts the scheduler, the present invention helps the user schedule the application kernel task to the hardware device with the best performance through the modified OpenCL command queue interface.

[0073] 5.2 System Process and Implementation Steps

[0074] The specific steps of the task scheduling method implemented by the scheduling system in the present invention are as follows:

[0075] Step 1: Collect the dynamic and static characteristics of the dataset program and the hardware characteristics of the running platform

[0076] The process of this step is as follows Figure 3 The heterogeneous computing test benchmark suites used in the present invention include Parboil, Polybench, AMD samples, and Rodinia, etc. These test benchmark suites are widely used in the research of heterogeneous computing performance analysis. [1]-[3] There is a certain degree of representativeness in it.

[0077] For the hardware features of the operating platform that need to be extracted, on the model training platform, the present invention uses the bash command lscpu to extract the CPU architecture and information, including the number of CPU cores, BogoMIPS, the size of the first-level data cache, the size of the first-level instruction cache, the size of the second-level cache, and the size of the third-level cache; and uses the CUDA API (i.e., the cudaGetDeviceProperties function) to obtain the GPU architecture and information, and the obtained hardware features include the video memory clock frequency, the video memory bus width, the peak video memory bandwidth, the total amount of video memory, the amount of shared memory available for each thread block, and the number of threads contained in each thread bundle.

[0078] For the characteristics of program code, LLVM is first used to compile the OpenCL kernel code into LLVM intermediate representation (IR), based on which the IR is analyzed to obtain eight types of static features, including the number of aggregation instructions, the number of arithmetic instructions, the number of function call instructions, the number of conversion instructions, the number of floating-point operation instructions, the number of memory access instructions, the number of terminator instructions, and the number of vector instructions. On this basis, the Libtooling tool is used to parse the program abstract syntax tree, and the type and number of statements and expressions, the control flow structure, and the function call chain can be further obtained. Finally, the present invention obtains an approximate estimate of the two program dynamic characteristics of the loop upper bound and the memory access by constructing LLVM Pass. LLVMPassLoopBoundAnalysis obtains the number of executions of each loop according to LLVM's ScalarEvolution analysis. If the number of loops is not a constant, the loop upper bound is estimated by the abstract interpretation method; LLVMPassAccessMemoryAnalysis can identify the memory access mode in the code and estimate the memory access amount. For example, given an array, the memory access amount can be roughly estimated by calculating the index range and frequency of access.

[0079] In the data preprocessing stage of feature engineering, for missing values, the present invention attempts to select the mean or median of all samples of the feature value to fill the missing values, but the effect is not ideal compared with the introduction of dummy variables. Introducing dummy variables to hardware features can make the model understand that these data missing are reasonable. At the same time, the present invention also introduces standardization and normalization of features, scaling the data in proportion so that it falls into a specific area for easy comprehensive analysis.

[0080] In the feature engineering stage, the present invention also performs feature selection by correlation analysis to retain the best feature subset. Using the correlation analysis method, the correlation of each feature is first calculated, and then according to the threshold, the feature with a variance greater than the threshold is selected. In this example, there is a feature variance of 0, that is, all samples have the same value of this feature (the limitation of the hardware platform causes some features to have the same value), then this feature has no effect on model training and can be directly discarded. Finally, the feature subset of the present invention retains 17 categories, including the number of arithmetic instructions, the number of function call instructions, the number of conversion instructions, the number of floating-point operation instructions, the number of memory access instructions, the number of terminator instructions, the number of vector instructions, the loop upper bound, the amount of memory access, the number of CPU cores, BogoMIPS, the size of the primary data cache, the size of the primary instruction cache, the size of the secondary cache, the clock frequency of the video memory, the peak value of the video memory bandwidth, and the total amount of video memory.

[0081] Step 2: Model selection, hyperparameter selection, and model evaluation of the scheduler model

[0082] The process of this step is as follows Figure 4 shown.

[0083] Since the target problem is a regression problem, suitable candidate linear models include linear regression model, decision tree regressor, support vector machine regressor, K nearest neighbor regressor, random forest regressor, AdaBoost regressor, gradient boosted random forest regression, Bagging regressor, ExtraTree regressor, etc.

[0084] For each model, hyperparameter selection is a key step in model training, which directly affects the performance and generalization ability of the model. Hyperparameters are parameters that need to be set before model training. They cannot be learned through training data and need to be manually adjusted or an automated hyperparameter optimization algorithm is used to search for the optimal value. The present invention uses grid search and / or random search in this process to select the hyperparameters that optimize the performance of each type of model, and compares all models under the premise of setting them to the optimal hyperparameters.

[0085] In a specific implementation, a grid search method and a random search method may be used simultaneously, and then one of them with better effect is selected for use.

[0086] Finally, in the model evaluation process, the present invention performs model evaluation by comparing the four indicators of mean square error, mean absolute error, R square, and adjusted R square. The mean absolute error (MAE) is used to measure the average absolute error between the predicted value and the true value. It is a non-negative value. The smaller the MAE, the better the model. The mean square error (MSE) is also used to measure the gap between the predicted value and the true value. When the dimensions are different, MAE and MSE are sometimes difficult to measure the effectiveness of the model. At this time, the coefficient of determination R square is needed. R square reflects the proportion of the total variation of the dependent variable that can be explained by the independent variable through the regression relationship. However, R square will exaggerate the degree of explanation of the independent variable on the variation of the dependent variable, and the more independent variables in the model, the larger the R square. Adjusting R square eliminates the influence of the number of independent variables. In summary, when comparing models through these four indicators, a higher weight is given to adjusting R square.

[0087] Based on this, AdaBoost regressor, gradient boosted random forest regressor and ExtraTree regressor were selected as candidate models for the scheduling scheme.

[0088] Step 3: User turns on the scheduling option and applies the device recommendation of the scheduler

[0089] The process of this step is as follows Figure 5 As shown. The kernel code is uniformly compiled, loaded and scheduled by the OpenCL runtime on the host. The present invention adds a bash script in the pre-compilation stage of the runtime, and runs analysis programs for the kernel code and the hardware platform. These analysis programs are consistent with the analysis in the offline training stage, and can obtain the static features, dynamic feature estimation and hardware platform features of the kernel code, and hand these data over to the scheduling model to obtain the scheduling prediction results. In the runtime stage, the present invention proposes a new attribute called SCHEDULER, which can be used as a parameter for creating a command queue. At present, the present invention supports three scheduler strategies, corresponding to the AdaBoost regressor, the gradient boosting random forest regressor and the ExtraTree regressor, respectively. When the scheduler is triggered, it binds the command queue to the available devices recommended by the model.

[0090] The following embodiments describe in detail the specific implementation situations in different application scenarios for the technical means of the three stages of the present invention. Example 1 introduces the implementation situation of using only the data set collected by the present invention to perform performance evaluation related work; Example 2 introduces the implementation situation of using the dynamic feature estimation method to evaluate the program running time; Example 3 introduces the implementation situation of the present invention on a heterogeneous platform when the scheduling policy is set to OFF; Example 4 introduces the implementation situation when the scheduling policy is set to use different machine learning models and a single program is run on a heterogeneous platform; Example 5 introduces the implementation situation when the scheduling policy is set to use different machine learning models and multiple programs are run on a heterogeneous platform.

[0091] Example 1: Using the dataset of the present invention to perform performance evaluation related work

[0092] like Figure 6 As shown, the data set includes three types of features: program static features, program dynamic features, and hardware features, with a total of 22 categories. After feature engineering in the model training stage of the present invention, 17 categories remain, containing 308 data. The specific implementation steps are as follows:

[0093] 1. The user downloads the dataset.

[0094] 2. Preprocess the data set according to the actual application scenario, for example, convert the file format, change the data type, delete unnecessary features, etc.

[0095] 3. The last column of the dataset is the target feature, which indicates the running time of the kernel program on a specific platform. We can continue the performance evaluation work based on this.

[0096] Example 2: Using dynamic feature estimation method to evaluate program running time

[0097] like Figure 7 As shown, users can use the dynamic feature estimation method to estimate the relative running time of the program, and based on this, provide a reference for real-time scheduling of tasks, priority arbitration of tasks, conflict arbitration of resources, and communication between tasks. The steps of the specific embodiment are as follows:

[0098] 1. The user compiles the kernel code into LLVMIR using the clang-emit-llvm command.

[0099] 2. Use the Libtooling tool to perform code checking and semantic analysis, collect information related to the abstract syntax tree through the AST matcher, and obtain more static features of the kernel program.

[0100] 3. Estimate program dynamic information through LLVMPass. The present invention constructs a new LLVMPass. LoopBoundAnalysis obtains the execution times of each loop according to LLVM's ScalarEvolution analysis. If the number of loops is not a constant, the loop upper bound is estimated through the abstract interpretation method; AccessMemoryAnalysis can identify the memory access pattern in the code and estimate the memory access amount. For example, given an array, the memory access amount can be roughly estimated by calculating the index range and frequency of access.

[0101] 4. Based on the obtained dynamic characteristics, the relative running time of kernel programs can be compared, providing a reference for other tasks such as real-time scheduling of tasks, task priority arbitration, resource conflict arbitration, and inter-task communication.

[0102] Example 3: Implementation of the present invention on a heterogeneous platform when the scheduling policy is set to OFF

[0103] like Figure 8 As shown, when the user uses the scheduling strategy of OFF, the system selects the first available device among the idle devices. The specific implementation steps are as follows:

[0104] 1. At the OpenCL platform level, the OpenCL implementation first queries the platform information. When building an OpenCL application, you must rely on a certain platform, that is, know the vendor's device and the corresponding SDK. This function supports returning the most recently supported OpenCL version, confirming that the platform supports the full version or embedded version of OpenCL, and can also return the extensions supported by the platform.

[0105] 2. At the OpenCL platform layer, the second step is to query the device information, including the device name, device global memory size, whether the device is available, and the device's maximum computing unit.

[0106] 3. At the OpenCL platform layer, the last step is to create a context and a command queue. The context object is used to manage devices on the same platform, and the command queue is responsible for sending commands from the host to the device. Each command queue object and context corresponds to a device (specified when created). A command queue can contain multiple commands, and commands are generally executed sequentially. At this time, since the scheduling policy is not set, the command queue will be directly bound to the first idle device on the platform.

[0107] 4. In the OpenCL runtime part, the command queue now knows which device the kernel is bound to, and it can create the cache, create the application object, compile the kernel, set parameters and run the kernel in sequence.

[0108] 5.The program ends.

[0109] Example 4: Implementation of Scheduling Strategy Setting to Use Different Machine Learning Models and a Single Program Running on a Heterogeneous Platform

[0110] like Fig. 9 As shown, compared with the third embodiment, after setting the startup scheduling model, the pre-compilation stage is mainly changed, which does not affect the runtime of the application. The specific embodiment steps are as follows:

[0111] 1. At the OpenCL platform level, the OpenCL implementation first queries the platform information. When building an OpenCL application, you must rely on a certain platform, that is, know the vendor's device and the corresponding SDK. This function supports returning the most recently supported OpenCL version, confirming that the platform supports the full version or embedded version of OpenCL, and can also return the extensions supported by the platform.

[0112] 2. At the OpenCL platform layer, the second step is to query the device information, including the device name, device global memory size, whether the device is available, and the device's maximum computing unit.

[0113] 3. At the OpenCL platform layer, the third step is to collect features to prepare for model inference. Features include kernel code static features, dynamic feature estimation, and platform hardware features, which are obtained in the same way as collecting datasets.

[0114] 4. At the OpenCL platform level, model inference is then completed. For each idle device, three different machine learning models are applied and the relative program runtime results of model inference are passed to the command queue.

[0115] 5. Before creating the command object queue, select the hardware device based on the model inference results.

[0116] 6. In the OpenCL runtime part, the command queue now knows which device the kernel is bound to, and it can create the cache, create the application object, compile the kernel, set parameters and run the kernel in order.

[0117] 7.The program ends.

[0118] Example 5: Implementation of Scheduling Strategy Setting to Use Different Machine Learning Models and Multiple Programs Running on Heterogeneous Platforms

[0119] like Fig.10 As shown, when there are multiple OpenCL tasks running, the scheduler will find the most suitable device among the idle devices, and will not affect the operation of the occupied devices. The specific implementation steps are as follows:

[0120] 1. At the OpenCL platform level, the OpenCL implementation first queries the platform information. When building an OpenCL application, you must rely on a certain platform, that is, know the vendor's device and the corresponding SDK. This function supports returning the most recently supported OpenCL version, confirming that the platform supports the full version or embedded version of OpenCL, and can also return the extensions supported by the platform.

[0121] 2. At the OpenCL platform layer, the second step is to query the device information, including the device name, device global memory size, whether the device is available, and the device's maximum computing unit.

[0122] 3. At the OpenCL platform layer, the third step is to collect features to prepare for model inference. Features include kernel code static features, dynamic feature estimation, and platform hardware features, which are obtained in the same way as collecting datasets.

[0123] 4. At the OpenCL platform level, model inference is then completed. For each idle device, three different machine learning models are applied and the relative program runtime results of model inference are passed to the command queue.

[0124] 5. Before creating the command object queue, select the hardware device based on the model inference results. In the case of no data correlation between tasks, set the properties when creating the command queue object so that the order of execution is out of order, which allows the kernel task of one device to be started when the kernel task of another device is not completed. The occupied device will be marked and cannot be used to execute new kernel tasks.

[0125] 6. In the OpenCL runtime part, the command queue now knows which device the kernel is bound to, and it can create the cache, create the application object, compile the kernel, set parameters and run the kernel in order.

[0126] 7.The program ends.

[0127] References

[0128] [1]AUGONNET C, THIBAULT S, NAMYST R, et al.StarPU: a unified platform fortaskschedulingon heterogeneous multicore architectures[C] / / Euro-Par2009Parallel Processing:15thInternational Euro-Par Conference,Delft,TheNetherlands,August 25-28,2009.Proceedings15.Springer Berlin Heidelberg,2009:863-874.

[0129] [2]WEN Y,WANG Z,O'BOYLE MF P.Smart multi-task scheduling for OpenCLprograms on CPU / GPUheterogeneous platforms[C] / / 201421st Internationalconference on highperformance computing(HiPC).IEEE,2014:1-10.

[0130] [3]AHMED U,LIN JCW,SRIVASTAVA G,et al.A load balance multi-scheduling model forOpenCL kernel tasks in an integrated cluster[J].SoftComputing,2021,25(1):407-420.

[0131] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0136] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0137] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A method for scheduling computing tasks on heterogeneous platforms based on machine learning algorithms, characterized in that: The scheduling method comprises the following steps: Step 1: Analyze and collect the static and dynamic features of the data set program and the hardware features of the running platform; Step 2: Select a suitable machine learning scheduling model and optimize hyperparameters, and evaluate the performance of the scheduling model through multiple indicators; Step 3: Select the best hardware device to achieve efficient task scheduling at runtime based on the prediction results of the scheduling model.

2. The scheduling method according to claim 1, characterized in that: In step 1, the static features of the data set program include: the number of instructions of different types of code and the number of basic blocks; the dynamic features of the data set program include: input data size, loop upper bound, memory access; the hardware features of the operating platform include CPU features and GPU features; and / or, The CPU characteristics include the number of CPU cores, BogoMIPS, the size of the first-level data cache, the size of the first-level instruction cache, the size of the second-level cache, and the size of the third-level cache; the GPU characteristics include the video memory clock frequency, the video memory bus width, the peak video memory bandwidth, the total amount of video memory, the amount of shared memory available for each thread block, and the number of threads contained in each thread bundle.

3. The scheduling method according to claim 1, characterized in that: Step one further comprises the following steps: Step 1.1: Compile the kernel code in the benchmark suite using the LLVM toolchain. Step 1.2, perform static feature analysis on the LLVM intermediate representation obtained by compiling in step 1.1 to obtain static features including the number of intermediate representation IR instruction classifications; Estimating dynamic features including loop upper bounds and memory accesses by building LLVM Pass; and / or, Use bash commands to extract CPU features in hardware, and use CUDA interface functions to extract GPU features in hardware; Step 1.3: Perform feature engineering on the static features, dynamic features, CPU features, and GPU features obtained in step 1.

2.

4. The scheduling method according to claim 3, characterized in that: After the static feature analysis of the compiled LLVM intermediate representation, the type and number of statements and expressions, control flow structure and function call chain information are obtained by parsing the program abstract syntax tree; The upper bound of the loop is estimated through LoopBoundAnalysis in LLVM Pass, and AccessMemoryAnalysis identifies the memory access pattern and estimates the memory access amount; and / or, The feature engineering process includes: data preprocessing and feature selection; the data preprocessing includes introducing dummy variables into hardware features and standardizing and normalizing the features; The feature selection refers to using a correlation analysis method, presetting a screening threshold, screening features according to the threshold, and retaining the best feature subset.

5. The scheduling method according to claim 1, characterized in that: Step 2 further includes the following steps: Step 2.1, select a suitable machine learning model based on the target problem; the machine learning model includes linear regression model, decision tree regressor, support vector machine regressor, K nearest neighbor regressor, random forest regressor, AdaBoost regressor, gradient boosted random forest regressor, Bagging regressor, ExtraTree regressor; Step 2.2: Use grid search and / or random search to select the hyperparameters that achieve the best performance. Step 2.3: Evaluate the performance of the machine learning model to obtain the final machine learning model; the evaluation indicators include comparative mean square error, mean absolute error, R square, and adjusted R square.

6. The scheduling method according to claim 1, characterized in that: Step three further includes the following steps: Step 3.1: During the pre-compilation phase of the OpenCL runtime, add a bash script to run the analysis program for the kernel code and hardware platform; Step 3.2, obtain the static features, dynamic feature estimation and hardware platform features of the kernel code, input the trained scheduling model, and obtain the scheduling prediction results; Step 3.3: Before creating the command object queue, select a hardware device according to the inference result of the scheduling model and execute the scheduling strategy.

7. The scheduling method according to claim 6, characterized in that: The scheduling strategy is based on a trained scheduling model, using prediction results generated by multiple machine learning algorithms to select the best device from multiple hardware devices to perform the computing task; and / or, Provides a scheduling interface by extending the OpenCL API, allowing users to select custom schedulers and scheduling strategies to meet the performance requirements of specific applications; and / or, Perform offline training on computing tasks on heterogeneous platforms, collect benchmark data, and generate feature subsets for training; and / or, The feature subset is used to perform hyperparameter optimization and evaluation on multiple candidate models to select the optimal scheduling model.

8. A heterogeneous platform computing task scheduling system based on machine learning algorithm, characterized in that: The scheduling system is used to implement the scheduling method according to any one of claims 1 to 7, comprising: a data collection module, a feature engineering module, a model training and selection module, a runtime scheduling module, and a scheduling interface module; The data collection module is responsible for collecting static features, dynamic features and hardware features of the program to provide basic data for model training and scheduling decisions; The feature engineering module processes and optimizes the collected feature data to improve the training effect and generalization ability of the model; The model training and selection module uses a machine learning algorithm to perform model training, optimize hyperparameters, select the best model and evaluate its performance; The runtime scheduling module selects the most suitable hardware device for each computing task according to the trained model and real-time feature data when the task is running; The scheduling interface module provides users with a flexible interface and extended functions, allows the selection of different scheduling strategies and models, and ensures compatibility with the existing OpenCL API.

9. Application of the scheduling method as described in any one of claims 1 to 7, or the scheduling system as described in claim 8 in artificial intelligence model training and reasoning, big data efficient processing and analysis, and graphics and image rendering processing.

Images