A heterogeneous compilation optimization method, device and medium

Abstract

The present application discloses a method, device and medium for heterogeneous compilation optimization, which relates to the field of computer software technology and is used to solve the problem of how the compiler can accurately and reasonably set the thread hierarchy. The method includes: when the compiler compiles the program, after detecting the macro that optimizes the thread structure, it sets the iteration number threshold, the dynamically changing iteration time approximation threshold and the target thread access time. Query the GPU hardware parameters to determine the initial thread block and grid parameter values ​​and compile the program to obtain the runtime thread access time. Compare the time with the target time, the iteration number threshold and the dynamic threshold. If the conditions are met, iterative compilation is performed, and the thread block and grid parameter values ​​are adjusted and recompiled. This cycle is repeated until the optimized thread block and grid parameter values ​​are obtained, the target thread access time is approached, and the thread hierarchy is optimized. This iterative adjustment of the thread block and grid size method greatly improves program performance.

Description

Technical Field

[0001] The present application relates to the field of computer software technology, and in particular to a heterogeneous compilation optimization method, device and medium. Background Art

[0002] With the rapid development of computer graphics processing and parallel computing technology, CUDA (Compute Unified Device Architecture), as a parallel computing platform and programming model launched by NVIDIA, has been widely used in many fields such as scientific computing, image processing, and machine learning. The CUDA compiler greatly improves computing performance by allowing developers to write programs directly on NVIDIA GPUs, allowing complex computing tasks that originally required a long time to be completed in a short time.

[0003] However, to fully realize the performance potential of CUDA programs, it is crucial to properly set the thread hierarchy (thread block size and grid size) of the CUDA kernel. These two parameters directly determine the organization and number of parallel execution threads on the GPU, and have a significant impact on the execution efficiency and resource utilization of the program.

[0004] However, programmers usually need to manually set these parameters according to specific problems and GPU hardware characteristics. This requires programmers to have deep CUDA programming knowledge and a deep understanding of GPU architecture, which increases the complexity and difficulty of programming. At the same time, due to the differences in hardware characteristics between different GPU models, manually set parameters are often difficult to achieve optimal performance on all target platforms. Therefore, in CUDA programming practice, manual setting of thread hierarchy has become one of the bottlenecks restricting program performance improvement and portability. Summary of the invention

[0005] The embodiments of the present application provide a method, device and medium for heterogeneous compilation optimization, which are used to solve the problem of how a CUDA compiler can automatically, accurately and reasonably set a thread hierarchy.

[0006] The present application embodiment adopts the following technical solutions:

[0007] On the one hand, an embodiment of the present application provides a method for heterogeneous compilation optimization, the method comprising: when a CUDA compiler compiles a CUDA program, if the CUDA compiler detects a macro definition for optimizing the thread hierarchy, determining a set iteration number threshold, an iteration time approximation dynamic threshold, and a target thread access time; the iteration time approximation dynamic threshold changes during the iteration process, so that the thread access time of the CUDA program at runtime gradually approaches the target thread access time; by querying the hardware parameter values ​​of the current GPU, determining the initial thread block parameter values ​​and grid parameter values; the hardware parameters include the maximum thread block The number of threads and the number of stream multiprocessors in the GPU; compile the CUDA program according to the thread block parameter value and the grid parameter value to obtain the thread access time at runtime; determine whether to perform iterative compilation according to the thread access time, the target thread access time, the iteration number threshold and the iteration time approaching the dynamic threshold; if iterative compilation is performed, adjust the thread block parameter value and the grid parameter value according to the thread access time, and recompile the CUDA program according to the adjusted thread block parameter value and the adjusted network parameter value; execute the above recompilation process cyclically until the iterative compilation is stopped, and generate the optimized thread block parameter value and the optimized grid parameter value.

[0008] In one example, whether to perform iterative compilation is determined based on the thread access time, the target thread access time, the iteration number threshold and the iteration time approximation dynamic threshold, specifically including: when the iteration number is less than the iteration number threshold, the thread access time is compared with the target thread access time; if the thread access time is greater than the target thread access time, the thread access time is compared with the iteration time approximation dynamic threshold; if the thread access time is higher than the iteration time approximation dynamic threshold, it is determined to perform iterative compilation.

[0009] In one example, the method also includes: if the thread access time is less than or equal to the target thread access time, inputting the iteration time approximation dynamic threshold and the target thread access time into a preset time approximation expression, and updating the iteration time approximation dynamic threshold according to the output result; re-comparing the updated iteration time approximation dynamic threshold with the thread access time; looping the above re-comparison process until the thread access time is higher than the updated iteration time approximation dynamic threshold, and determining to execute iterative compilation.

[0010] In an example, the preset time-consuming approximation expression is as follows:

[0011] =

[0012] in, It is a time-consuming iterative approach to the dynamic threshold. Time-consuming access for the target thread.

[0013] In one example, the method further includes: when the number of iterations is equal to an iteration number threshold, determining to stop executing the iterative compilation; or if the thread access time is less than or equal to the target thread access time, determining to stop executing the iterative compilation.

[0014] In one example, thread block parameter values ​​and grid parameter values ​​are adjusted according to thread access timing, specifically including: inputting thread access timing, thread block parameter values, grid parameter values, target thread access timing and hardware parameter values ​​into a pre-built adjustment neural network model to obtain multiple groups of thread hierarchies with similar access performance; running tests on multiple groups of thread hierarchies to obtain minimum thread access timing; and determining adjustment thread block parameter values ​​and adjustment grid parameter values ​​based on the thread hierarchy corresponding to the minimum thread access timing.

[0015] In one example, the method also includes: obtaining sample hardware parameter values, sample thread block parameter values, and sample grid parameter values ​​of the sample GPU; compiling the CUDA sample program according to the sample thread block parameter values ​​and the sample grid parameter values ​​to obtain sample thread access time; determining multiple groups of sample thread hierarchies that theoretically correspond to reducing the sample thread access time; using the sample thread access time, sample thread block parameter values, sample grid parameter values, and sample hardware parameter values ​​as sample inputs, and using the multiple groups of sample thread hierarchies as sample labels, to train the initial neural network model to obtain an adjusted neural network model.

[0016] In one example, after generating optimized thread block parameter values ​​and optimized grid parameter values, the method further includes: obtaining thread hierarchy data and thread access time for each iteration during the iteration process; generating a display chart of the thread hierarchy data and thread access time for each iteration, and displaying the display chart on the client.

[0017] On the other hand, an embodiment of the present application provides a heterogeneous compilation optimization device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the above-mentioned heterogeneous compilation optimization methods.

[0018] On the other hand, an embodiment of the present application provides a heterogeneous compilation optimization non-volatile computer storage medium storing computer executable instructions, which can execute any of the above-mentioned heterogeneous compilation optimization methods.

[0019] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0020] The optimization is performed in an iterative manner, and the iteration time approximation dynamic threshold is set as an important basis for iteration, which provides iterative conditions for the iterative process, ensuring that each iteration will adjust the thread block and grid size according to the thread access time of the previous run, and recompile and run the program until the number of iterations specified by the user is reached or the target thread access time is met. This iterative mechanism helps to gradually approach the optimal thread hierarchy configuration, greatly improving the performance of the optimized program. At the same time, macro definitions are used to control whether to enable the compilation optimization function of the thread hierarchy, providing flexible optimization switch options, so that users can decide whether to enable the optimization function according to their needs. The thread block parameter value and grid parameter value are adjusted according to the thread access time, which ensures the basis and reliability of each adjustment and greatly reduces the computational complexity of the iterative process. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of the present application, some embodiments of the present application will be described in detail below in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A schematic diagram of a flow chart of a heterogeneous compilation optimization method provided in an embodiment of the present application;

[0023] Figure 2 A rendering of a specific embodiment of heterogeneous compilation optimization provided in an embodiment of the present application;

[0024] Figure 3 A GPU thread hierarchy diagram provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the structure of a heterogeneous compilation optimization device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0027] Some embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0028] Figure 1 A schematic diagram of a heterogeneous compilation optimization method provided in an embodiment of the present application. The method can be applied to different business fields. Certain input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0029] The analysis method involved in the embodiment of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For the convenience of understanding and description, the following embodiments are described in detail by taking a compiler as an example.

[0030] It should be noted that:

[0031] CUDA is a general-purpose parallel computing architecture launched by NVIDIA that enables GPUs to solve complex computing problems. CUDA mainly provides the following features to support CUDA programming.

[0032] Kernel function: In CUDA programming, programmers can define a function called a "kernel" that will be executed in parallel on the GPU. Kernel functions are executed simultaneously by many threads, each thread is responsible for processing a small part of the input data.

[0033] Thread hierarchy: Threads in the GPU are logically organized into thread blocks and grids. Threads in a thread block can share shared memory and work together to solve subtasks. A grid contains multiple thread blocks, and the entire CUDA program can be viewed as multiple grids executing in parallel on the GPU.

[0034] Memory hierarchy: CUDA devices have different levels of memory, including global memory, shared memory, and registers. Programmers can use these different levels of memory to optimize access patterns to improve performance.

[0035] CUDA Runtime API: CUDA provides a set of APIs that enable developers to perform operations such as data transfer and parallel computing between the host (CPU) and the device (GPU).

[0036] Based on this, Figure 1 The process in may include the following steps:

[0037] S101: When the CUDA compiler compiles the CUDA program, if the CUDA compiler detects a macro definition for optimizing the thread hierarchy, it determines the set iteration number threshold, the iteration time approximation dynamic threshold, and the target thread access time; the iteration time approximation dynamic threshold will change during the iteration process, so that the thread access time of the CUDA program during runtime gradually approaches the target thread access time.

[0038] In some embodiments of the present application, when the CUDA compiler compiles a CUDA program, it is necessary to detect the macro definition for optimizing the thread hierarchy. The macro definition for optimizing the thread hierarchy is the macro _THREAD_STRUCTURE_OPT_, which is used to control whether the compilation optimization function of the thread hierarchy is enabled. A flexible optimization switch option is provided to facilitate users to decide whether to enable the optimization function according to their needs.

[0039] Furthermore, after detecting the macro definition for optimizing the thread hierarchy, it is necessary to determine the iteration number threshold, the iteration time approximation dynamic threshold and the target thread access time. The iteration number threshold is the total number of iterations in the optimization iteration process; the iteration time approximation dynamic threshold is an important basis in the iteration process, which will change dynamically with the iteration process, so that the thread access time of the CUDA program at runtime gradually approaches the target thread access time; the target thread access time is the optimal thread access time after the CUDA program is optimized.

[0040] Furthermore, the threshold of the number of iterations is customized by those skilled in the art according to their needs. Although the iteration time approaches the dynamic threshold and changes dynamically with the iteration process, the initial value is customized by those skilled in the art according to their needs. The target thread access time, that is, the ideal value achieved by the iteration, is determined by those skilled in the art based on experience.

[0041] S102: determining initial thread block parameter values ​​and grid parameter values ​​by querying hardware parameter values ​​of the current GPU; the hardware parameters include the maximum number of threads in the thread block and the number of stream multiprocessors in the GPU.

[0042] In some embodiments of the present application, in order to query the hardware parameter values ​​of the current GPU, it is necessary to run the CUDA program first. The compiler queries the hardware parameters of the current GPU through the CUDA runtime API. The hardware parameters include the maximum number of threads in the thread block and the number of streaming multiprocessors in the GPU.

[0043] It should be noted that the CUDA runtime API is a set of high-level functions and data structures provided in the CUDA toolkit to simplify the development of CUDA applications. That is, it provides functions for transferring data between the host (CPU) and the device, including the transfer from host memory to device memory and from device memory to host memory. It also allows developers to launch kernel functions on the device for parallel computing.

[0044] Further, in some embodiments of the present application, for the initial thread block parameter value and grid parameter value, those skilled in the art may determine them based on empirical formulas or preset optimization strategies. For example, according to the number of stream multiprocessors obtained by querying the hardware parameter value, a strategy of decreasing the grid parameter value and increasing the thread block parameter value is adopted. If it is found that the thread access time is shortening, the grid parameter value is continued to be reduced and the thread block parameter value is increased until the thread access time is found to be increasing. Then, the grid parameter value is stopped from being reduced and the thread block parameter value is increased. The data with the shortest thread access time is selected from all the data, and the corresponding thread block parameter value and grid parameter value become the initial thread block parameter value and grid parameter value.

[0045] S103: Compile the CUDA program according to the thread block parameter value and the grid parameter value to obtain the thread access time consumption during runtime.

[0046] In some embodiments of the present application, the CUDA program is compiled using thread block parameter values ​​and grid parameter values, and the executable program is run to generate an operation analysis file. Then, the executable file is run on the GPU, and the performance data at runtime, including the kernel function running time and the data transmission time, are collected and saved in the operation analysis file, that is, the sum of the kernel function running time and the data transmission time is the thread access time at runtime. An operation analysis file containing key performance indicators is generated during the operation process, and the file is then used by the compiler as the basis for subsequent iterative optimization. This mechanism enables the optimization process to be fed back and adjusted based on actual operation data, greatly enhancing the pertinence and effectiveness of the optimization.

[0047] S104: Determine whether to perform iterative compilation according to the thread access time, the target thread access time, the iteration number threshold, and the iteration time approaching the dynamic threshold.

[0048] In some embodiments of the present application, when the number of iterations is less than an iteration number threshold, the thread access time is compared with the target thread access time; if the thread access time is greater than the target thread access time, the thread access time is compared with the iteration time approximating a dynamic threshold; if the thread access time is higher than the iteration time approximating a dynamic threshold, it is determined to perform iterative compilation.

[0049] If the thread access time is less than or equal to the dynamic threshold of the iteration time approximation, the dynamic threshold of the iteration time approximation and the target thread access time are input into the preset time approximation expression, and the dynamic threshold of the iteration time approximation is updated according to the output result, and the updated dynamic threshold of the iteration time approximation is re-compared with the thread access time, and the above re-comparison process is executed repeatedly until the thread access time is higher than the updated dynamic threshold of the iteration time approximation, and it is determined to execute iterative compilation.

[0050] It should be noted that the target thread access time is the custom ideal time. If the thread access time is greater than the target thread access time, it means that the CUDA program can continue to be optimized. The iteration time approximation dynamic threshold is the user-defined value that approximates the target thread access time. Therefore, if the thread access time is greater than the iteration time approximation dynamic threshold, it means that the CUDA program can continue to be optimized and it is determined to perform iterative compilation for optimization.

[0051] When the thread access time is less than or equal to the iteration time approximation dynamic threshold, the iteration time approximation dynamic threshold and the target thread access time are input into the preset time approximation expression. The preset time approximation expression is:

[0052] =

[0053] in, It is a time-consuming iterative approach to the dynamic threshold. Time-consuming access for the target thread.

[0054] This expression adopts the idea of ​​gradual decrease, that is, each decreasing value is half of the increase in the target thread access time. In this way, the iterative time approaching the dynamic threshold will be infinitely close to the target thread access time, that is, the iterative time approaching the dynamic threshold can gradually approach the target thread access time, thereby guiding the iterative thread access time to the target thread access time.

[0055] When the number of iterations is equal to the iteration number threshold, it is determined to stop executing the iterative compilation; or if the thread access time is less than or equal to the target thread access time, it is determined to stop executing the iterative compilation.

[0056] It should be noted that the two conditions for stopping iterative compilation are to first meet the condition of the number of iterations, and then meet the condition that the thread access time is less than or equal to the target thread access time. If the set number of iterations is reached, but the thread access time is greater than the target thread access time, it is determined to stop the iterative compilation and select the smallest access time from the iterative thread access time as the output. Only when the number of iterations is not set will the condition of stopping iterative compilation when the thread access time is less than or equal to the target thread access time be taken as the priority.

[0057] Taking the entire iteration process as an example, for example, the target thread access time is 0.4ms, the iteration time approaches the dynamic threshold of 0.8ms, the thread access time is 0.9ms, and the thread access time is higher than the iteration time approximation dynamic threshold. Iterative compilation is performed, and the thread access time after iteration is 0.7ms. The thread access time is less than or equal to the iteration time approximation dynamic threshold. The iteration time approximation dynamic threshold is updated to 0.6ms according to the preset time approximation expression, and then the above re-comparison process is executed repeatedly until the number of iterations is equal to the iteration number threshold or the thread access time is less than or equal to the target thread access time, and the iterative compilation is stopped.

[0058] S105: If iterative compilation is performed, the thread block parameter value and the grid parameter value are adjusted according to the thread access time, and the CUDA program is recompiled according to the adjusted thread block parameter value and the adjusted network parameter value.

[0059] In some embodiments of the present application, the adjustment strategy is based on a heuristic algorithm, a machine learning model, or a simple performance comparison. This embodiment adopts a machine learning model method.

[0060] For example, the thread block parameter values ​​and grid parameter values ​​are adjusted by pre-building an adjusted neural network model. First, the sample hardware parameter values, sample thread block parameter values, and sample grid parameter values ​​of the sample GPU are obtained; the CUDA sample program is compiled according to the sample thread block parameter values ​​and the sample grid parameter values ​​to obtain the sample thread access time, and multiple groups of sample thread hierarchies corresponding to theoretically reducing the sample thread access time are determined, that is, multiple groups of sample thread hierarchies input by the user, and the sample thread access time, sample thread block parameter values, sample grid parameter values, and sample hardware parameter values ​​are used as sample inputs, and the multiple groups of sample thread hierarchies are used as sample labels to train the initial neural network model to obtain the adjusted neural network model.

[0061] The thread access time, thread block parameter value, grid parameter value, target thread access time and hardware parameter value are input into a pre-built adjustment neural network model to obtain multiple groups of thread hierarchies with similar access performance; then the multiple groups of thread hierarchies are run and tested to obtain the minimum thread access time; according to the thread hierarchy corresponding to the minimum thread access time, the adjustment thread block parameter value and the adjustment grid parameter value are determined.

[0062] S106: the above recompilation process is executed cyclically until the iterative compilation is stopped, and the optimized thread block parameter values ​​and the optimized grid parameter values ​​are generated.

[0063] In some embodiments of the present application, after the iterative compilation is completed, the thread hierarchy data and the thread access time of each iteration in the iterative process are obtained; based on the obtained thread hierarchy data and the thread access time of each iteration, a display chart of the thread hierarchy data and the thread access time of each iteration is generated, and the display chart is displayed on the client.

[0064] It should be noted that although the embodiments of the present application are based on Figure 1 Steps S101 to S106 are described in sequence, but this does not mean that steps S101 to S106 must be performed in a strict order. Figure 1 The order shown in the figure is to introduce and explain step S101 to step S106 in sequence, in order to facilitate those skilled in the art to understand the technical solution of the embodiment of the present application. In other words, in the embodiment of the present application, the order between step S101 to step S106 can be appropriately adjusted according to actual needs.

[0065] pass Figure 1 The method uses iterative optimization of the thread hierarchy to reduce the optimization burden of users, allowing users to focus more on the implementation of algorithm logic without paying too much attention to the underlying parallel optimization details. It also queries the GPU hardware characteristics and adjusts the thread hierarchy accordingly, so that the program can better adapt to different GPU environments, enhance the portability and adaptability of the program, and can adjust the size of thread blocks and grids according to specific GPU hardware characteristics, thereby maximizing the parallel computing capabilities of the GPU and significantly improving the execution efficiency of CUDA programs. This optimization reduces the waste of computing resources caused by improper thread configuration and improves the performance of the program.

[0066] Figure 2 This is a rendering of a specific embodiment of heterogeneous compilation optimization provided in an embodiment of the present application.

[0067] exist Figure 2 In the first step, the CUDA program named add.cu is input, the _THREAD_STRUCTURE_OPT_ macro definition is detected, and the optimization of the thread hierarchy is started.

[0068] The second step is to add compilation parameters. The compilation command is: nvcc add.cu –opt_thread_structure –opt_num=4, and the number of iterations is 4.

[0069] The third step is to query the GPU hardware parameters. The compiler queries the hardware parameters of the current GPU through the CUDA runtime API. Here, it is a GT730 graphics card, the maximum number of threads per thread block is 1024, and the number of stream multiprocessors in the GPU is 2.

[0070] In the fourth step, the compiler compiles add.cu according to the thread hierarchy selected by the GPU hardware parameters and generates an executable file. Then, the executable file runs on the GPU and collects the performance data during the runtime and saves it to the run analysis file.

[0071] In the fifth step, the compiler reads the run analysis file, analyzes the performance data (thread access time), and adjusts the size of the thread block and grid based on the analysis results.

[0072] Based on this, determine whether the number of iterations is equal to 4, or whether the thread access time is less than or equal to 1.5ms. If not, loop through steps 4 and 5. If yes, output the thread block size and grid size adjusted by the code.

[0073] After the above steps, we get Figure 2 The iterative process is:

[0074] The thread block size is (256,1,1), the grid block size is (4096,1,1), and the kernel takes about 1.65ms.

[0075] The thread block size is adjusted to (128,1,1), the grid block size is (8192,1,1), and the kernel takes about 1.6ms.

[0076] The thread block size is adjusted to (64,1,1), the grid block size is adjusted to (16384,1,1), and the kernel takes about 2.3ms.

[0077] The thread block size is adjusted to (512,1,1), the grid block size is adjusted to (2048,1,1), and the kernel takes 1.7ms.

[0078] At the end of this iteration, the final thread block size is (128,1,1) and the grid block size is (8192,1,1).

[0079] Figure 3 A GPU thread hierarchy diagram provided in an embodiment of the present application.

[0080] exist Figure 3 In the figure, the left side is the logical level, including threads, thread blocks composed of threads, and grids composed of thread blocks. The right side is the physical level, including threads, thread warps composed of threads, and streaming multiprocessors.

[0081] It should be noted that the thread hierarchy is the core component of the CUDA programming model, which allows developers to efficiently organize and utilize parallel computing resources on the GPU. It mainly includes the following levels:

[0082] Thread: A thread is the smallest execution unit in CUDA. Each thread executes the same kernel function. There is no direct execution order relationship between threads. They execute in parallel. Each thread has a unique thread ID, which is used to distinguish different threads inside the kernel function.

[0083] Thread block: A thread block is a collection of multiple threads, which is the basic unit of CUDA thread scheduling. All threads in the same thread block will be executed on the same streaming multiprocessor (SM), so they can share some resources, such as shared memory. Threads in a thread block can be synchronized through synchronization primitives to ensure data consistency and collaboration between threads. The size of the thread block can be specified when the kernel function is started, but it is limited by the GPU hardware resources.

[0084] Grid: A grid is a larger set of multiple thread blocks, which is used to organize the execution of the entire kernel function. One kernel function corresponds to one grid, and the thread blocks in the grid can be executed in parallel on multiple SMs, thereby achieving higher parallelism. The dimension of the grid (that is, the number of thread blocks contained) can also be specified when the kernel function is started, but it is also limited by the GPU hardware resources. Thread blocks within a grid cannot communicate or synchronize directly, and communication and synchronization between them need to be achieved through global memory or other mechanisms.

[0085] Thread warp: A thread warp is an internal unit of CUDA thread scheduling, usually containing 32 threads. When the kernel function is started, the GPU will schedule the execution of threads according to the granularity of the thread warp. That is, each scheduling will execute all threads in a thread warp at the same time. The scheduling of thread warps helps to hide memory access latency and improve GPU utilization. It should be noted that the thread warp is a logical concept that is transparent to developers, but it is very important when understanding and optimizing CUDA programs.

[0086] Figure 4 A schematic diagram of the structure of a heterogeneous compilation optimization device provided in an embodiment of the present application includes:

[0087] at least one processor; and,

[0088] a memory communicatively connected to at least one processor; wherein,

[0089] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute any one of the above heterogeneous compilation optimization methods.

[0090] Some embodiments of the present application provide a heterogeneous compilation optimization non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions can execute any one of the above-mentioned heterogeneous compilation optimization methods.

[0091] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0092] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0093] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0097] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0098] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM), and non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0099] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0100] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0101] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the technical principle of the present application should fall within the protection scope of the present application.

Claims

1. A heterogeneous compilation optimization method, characterized in that: The method comprises: When a CUDA compiler compiles a CUDA program, if the CUDA compiler detects a macro definition for optimizing a thread hierarchy, a set iteration number threshold, an iteration time approximation dynamic threshold, and a target thread access time are determined; the iteration time approximation dynamic threshold changes during the iteration process, so that the thread access time of the CUDA program during runtime gradually approaches the target thread access time; By querying the hardware parameter values ​​of the current GPU, the initial thread block parameter values ​​and grid parameter values ​​are determined; the hardware parameters include the maximum number of threads in the thread block and the number of stream multiprocessors in the GPU; Compile the CUDA program according to the thread block parameter value and the grid parameter value to obtain the thread access time consumption at runtime; Determining whether to perform iterative compilation according to the thread access time, the target thread access time, the iteration number threshold, and the iteration time approaching a dynamic threshold; If iterative compilation is performed, the thread block parameter value and the grid parameter value are adjusted according to the thread access time, and the CUDA program is recompiled according to the adjusted thread block parameter value and the adjusted network parameter value; The above recompilation process is executed cyclically until the iterative compilation is stopped, and the optimized thread block parameter values ​​and the optimized grid parameter values ​​are generated; The determining whether to perform iterative compilation according to the thread access time, the target thread access time, the iteration number threshold, and the iteration time approaching the dynamic threshold specifically includes: When the number of iterations is less than the iteration number threshold, comparing the thread access time with the target thread access time; If the thread access time is greater than the target thread access time, comparing the thread access time with the iteration time approximation dynamic threshold; If the thread access time is higher than the iteration time and approaches a dynamic threshold, determining to perform iterative compilation; The method further comprises: If the thread access time is less than or equal to the iteration time approximation dynamic threshold, input the iteration time approximation dynamic threshold and the target thread access time into a preset time approximation expression, and update the iteration time approximation dynamic threshold according to the output result; Re-comparing the updated iteration time approximating the dynamic threshold with the thread access time; The above re-comparison process is executed cyclically until the thread access time is higher than the updated iteration time and approaches the dynamic threshold, and it is determined to execute iterative compilation.

2. The method according to claim 1, characterized in that The preset time-consuming approximation expression is as follows: = in, It is a time-consuming iterative approach to the dynamic threshold. Time-consuming access for the target thread.

3. The method according to claim 1, characterized in that The method further comprises: When the number of iterations is equal to the iteration number threshold, determining to stop executing iterative compilation; Or if the thread access time is less than or equal to the target thread access time, it is determined to stop executing iterative compilation.

4. The method according to claim 1, characterized in that: The adjusting the thread block parameter value and the grid parameter value according to the thread access time specifically includes: Inputting the thread access time, the thread block parameter value, the grid parameter value, the target thread access time and the hardware parameter value into a pre-built adjustment neural network model to obtain multiple groups of thread hierarchies with similar access performance; Run tests on multiple sets of thread hierarchies to obtain the minimum thread access time; According to the thread hierarchy structure corresponding to the minimum thread access time consumption, the thread block parameter value and the grid parameter value are adjusted.

5. The method according to claim 4, characterized in that The method further comprises: Get sample hardware parameter values, sample thread block parameter values, and sample grid parameter values ​​of a sample GPU; Compile the CUDA sample program according to the sample thread block parameter value and the sample grid parameter value to obtain the sample thread access time consumption; Determine a plurality of groups of sample thread hierarchical structures corresponding to theoretically reducing the access time of the sample threads; The sample thread access time, the sample thread block parameter value, the sample grid parameter value and the sample hardware parameter value are used as sample inputs, and the multiple groups of sample thread hierarchical structures are used as sample labels to train the initial neural network model to obtain an adjusted neural network model.

6. The method according to claim 1, characterized in that After generating the optimized thread block parameter value and the optimized grid parameter value, the method further includes: Get the thread hierarchy data and thread access time of each iteration in the iteration process; Generate a display chart of thread hierarchy data and thread access time consumption for each iteration, and display the display chart on the client.

7. A heterogeneous compilation optimization device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the heterogeneous compilation optimization method described in any one of claims 1 to 6.

8. A heterogeneous compilation optimized non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions can execute a heterogeneous compilation optimization method as described in any one of claims 1 to 6.

Images