A software acceleration method for numerical computation based on CPU-GPU collaboration

Abstract

This invention discloses a method for accelerating numerical computation software based on CPU-GPU collaboration. The method divides the program into basic blocks and uses a code analysis module to predict the runtime information of these basic blocks, including whether they are computationally intensive tasks, whether they are easily parallelized, and whether the memory copy time of the computation task is less than the CPU execution time. Based on the analysis results, basic blocks that meet the above conditions are marked as GPU modules, and the rest are CPU modules. GPU modules are compiled into GPU code for execution on the fly. If CPU modules have hotspot code, they are compiled into machine code on the fly for execution; otherwise, they are interpreted. This method employs a CPU-GPU collaborative just-in-time compilation approach, fully utilizing CPU and GPU computing resources, achieving higher execution efficiency than CPU or GPU execution alone. It also leverages the advantages of just-in-time compilation of interpreted languages, significantly improving the performance of numerical computation software.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of heterogeneous computing, and particularly relates to a numerical calculation software acceleration method based on CPU-GPU cooperation. BACKGROUND

[0002] CPU-GPU cooperation is applied to parallel computing tasks, and there are two kinds of cooperation. One is that CPU is responsible for managing GPU, providing calculation data for GPU and receiving calculation results, and GPU undertakes all calculation tasks. The other is that CPU manages GPU while being responsible for part of the calculation tasks.

[0003] Numerical calculation software solves linear and nonlinear problems in a numerical way, uses a high-level programming language, has simple coding, and is run in an interpreted execution mode based on CPU. Taking Octave as an example, Octave has a syntax specification basically consistent with Matlab, is weakly typed, but the efficiency of the interpreted execution is low.

[0004] In order to solve the problem of low efficiency of the interpreted execution of numerical calculation software, Matlab introduces just-in-time (JIT) compilation technology. In Matlab, the JIT compiler compiles and optimizes the code into local code during the execution of the program, including converting the source code or bytecode into machine code, and then directly executing it, which significantly improves the performance. Compared with static compilation, JIT compilation can selectively compile hot code, saving unnecessary compilation time and space, and thus achieving higher execution efficiency.

[0005] Octave introduced JIT compilation technology in 2012, but as of now, it is still in its early stages and development has stagnated. At the same time, the JIT compiler in Octave has low completion degree and high coupling degree with the original Octave interpreter, which leads to the difficulty of maintaining the JIT when the Octave version is updated. After Octave 6.1.0, Octave has abandoned the compatibility of this JIT compiler.

[0006] At present, the execution method of numerical calculation software does not fully utilize the advantages of hardware. If it is based on CPU JIT compilation, GPU computing resources are wasted. If it is parallel computing using CPU-GPU cooperation, the performance improvement brought by the JIT compiler of the interpreted language is not fully utilized. SUMMARY

[0007] In view of the above defects of the prior art, the technical problem to be solved by the present application is to provide a numerical calculation software acceleration method based on CPU-GPU cooperation, which fully utilizes CPU and GPU computing resources, and instantly compiles the calculation-intensive and easily-parallel code into GPU code for execution, and if the remaining code is hot code, it is instantly compiled into machine code for execution based on CPU, otherwise, it is executed based on CPU, thereby improving the execution efficiency.

[0008] To achieve the above object, the present application provides a numerical calculation software acceleration method based on CPU-GPU cooperation, comprising:

[0009] dividing the source program into basic blocks;

[0010] estimating the basic block running information through a code analysis module;

[0011] dividing the basic block into a GPU module and a CPU module according to the running information;

[0012] instantly compiling the GPU module to obtain GPU code;

[0013] instantly compiling or interpreting the CPU module.

[0014] Further, before the source program is divided into basic blocks, the source program is further subjected to syntax tree construction and type inference.

[0015] Further, the source program is divided into basic blocks, and each basic block satisfies:

[0016] (1) the control flow can only enter from the entry statement of the basic block, and the entry statement is the first statement of the program, the target statement of the jump or the next statement of the conditional jump;

[0017] (2) the control flow can only exit from the end statement of the basic block, and the end statement is the jump statement, the stop statement or the previous statement of the next entry statement.

[0018] Further, the basic block running information is estimated through the code analysis module, and the basic block is divided into a GPU module and a CPU module according to the running information: including but not limited to whether the basic block is a calculation-intensive task, whether the task size is greater than a threshold, and whether the memory copy time of the calculation task is less than the CPU execution time; if the analysis result is correct, the basic block is marked as a GPU module, otherwise, it is marked as a CPU module.

[0019] Further, the instant compiling for the GPU module to obtain GPU code is specifically: converting the GPU module data type into OpenCL data type, converting the GPU module code into OpenCL code, and further performing GPU compiling to obtain GPU code.

[0020] Further, the instant compiling or interpretive execution for the CPU module is specifically: analyzing the CPU module, if it is hot code, instant compiling the hot code into machine code, and if it is not hot code, directly performing interpretive execution.

[0021] A numerical calculation software acceleration system based on CPU-GPU cooperation, comprising:

[0022] A source program division module divides the source program into basic blocks;

[0023] A running information estimation module estimates the basic block running information through the code analysis module;

[0024] A basic block division module divides the basic block into a GPU module and a CPU module according to the running information;

[0025] A GPU running module instant compiles the GPU module to obtain GPU code;

[0026] A CPU running module instant compiles or interpretively executes the CPU module.

[0027] The beneficial effects of the present application are:

[0028] The present application utilizes CPU-GPU cooperation instant compiling, distributes the calculation task to the GPU or CPU according to a more optimal scheme, synchronously interacts data between the two devices, and the synchronization time is far less than the execution time of all codes directly on the CPU, thereby improving the execution efficiency.

[0029] The concept, specific structure and generated technical effects of the present application will be further described below in combination with the drawings, so as to fully understand the purpose, features and effects of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 It is an execution architecture schematic diagram of numerical calculation software Octave;

[0031] Figure 2 It is an execution flowchart of an Octave actual example;

[0032] Figure 3 It is an execution architecture schematic diagram of CPU-GPU cooperation;

[0033] Figure 4A CPU-GPU cooperative example execution flowchart. DETAILED DESCRIPTION

[0034] As shown in the figure, the present application provides a numerical calculation software acceleration method based on CPU-GPU cooperation, comprising: Figure 1

[0035] Dividing the source program into basic blocks;

[0036] Estimating basic block running information through a code analysis module;

[0037] Dividing the basic block into a GPU module and a CPU module according to the running information;

[0038] Instantly compiling the GPU module to obtain GPU code;

[0039] Instantly compiling or interpreting the CPU module.

[0040] In this embodiment, before dividing the source program into basic blocks, the source program is further subjected to syntax tree construction and type inference.

[0041] In this embodiment, the source program is divided into basic blocks, and each basic block satisfies:

[0042] (1) Control flow can only enter from the entry statement of the basic block, and the entry statement is the first statement of the program, the target statement of the jump, or the next statement of the conditional jump;

[0043] (2) Control flow can only exit from the end statement of the basic block, and the end statement is the jump statement, the stop statement, or the previous statement of the next entry statement.

[0044] In this embodiment, the code analysis module estimates the basic block running information, and divides the basic block into a GPU module and a CPU module according to the running information: including but not limited to whether the basic block is a compute-intensive task, whether the task size is greater than a threshold, and whether the memory copy time of the computing task is less than the CPU execution time; if the analysis result is correct, mark this basic block as a GPU module, otherwise mark it as a CPU module.

[0045] In this embodiment, the GPU module is instantly compiled to obtain GPU code, specifically: converting the data type of the GPU module to OpenCL data type, converting the code of the GPU module to OpenCL code, and further performing GPU compilation to obtain GPU code.

[0046] In this embodiment, the CPU module is instantly compiled or interpreted, specifically: analyzing the CPU module, if it is a hot code, then instantly compiling the hot code into machine code; if it is not a hot code, then directly interpreting and executing.​

[0047] The application also provides a CPU-GPU cooperative numerical calculation software acceleration system, comprising:

[0048] A source program division module divides the source program into basic blocks;

[0049] A running information estimation module estimates the basic block running information through the code analysis module;

[0050] A basic block division module divides the basic blocks into GPU modules and CPU modules according to the running information;

[0051] A GPU running module instantaneously compiles the GPU modules to obtain GPU codes;

[0052] A CPU running module instantaneously compiles or interprets the CPU modules.

[0053] In summary, the application provides a CPU-GPU cooperative numerical calculation software acceleration method, which fully utilizes the CPU and GPU computing resources, divides the program basic blocks, analyzes the information of each basic block, instantaneously compiles the calculation-intensive and easily-parallel basic blocks into GPU codes for execution, instantaneously compiles the remaining basic blocks into machine codes for execution based on the CPU if they are hot codes, or interprets the remaining basic blocks based on the CPU.

[0054] The following is a brief description of the abbreviations and key terms used in the embodiments of the application:

[0055] JIT (Just-In-Time Compilation): instant compilation, a technology for improving the execution efficiency of interpreted languages. It involves compiling during program execution rather than before execution, which may include source code conversion, and more commonly, bytecode conversion to machine code, and then direct execution.

[0056] AST (Abstract Syntax Tree): abstract syntax tree, an abstract representation of source code, which represents the syntax structure of source code in a tree form, and each node is a structure of source code.

[0057] IR (Intermediate Representation): intermediate representation, a data structure or code used by a compiler or virtual machine to represent source code. IR can represent source code without losing information, is independent of specific source code or target language, and is beneficial for further compilation and optimization.

[0058] To make the purpose, technical solutions and advantages of the application clearer, the technical solutions in the application will be described in more detail below with reference to the drawings in the application.

[0059] Unlike general-purpose languages, numerical computing software tends to have simple syntax and is applied to compute-intensive tasks, so it is necessary to compile as much of the code as possible for the compute-intensive modules and hotspots, while other less time-consuming code can be interpreted. Figure 1 An architecture diagram for executing numerical computing software Octave.

[0060] First, the source code is analyzed by lexical analysis and syntax analysis to obtain a syntax tree.

[0061] If the code segment of the syntax tree is a loop structure and the number of loops is greater than the threshold, the code segment is compiled immediately, and if the compilation time is greater than the threshold, the execution is returned to the interpreter.

[0062] If there is no hot code in the code segment of the syntax tree, the code segment is directly interpreted and executed.

[0063] The original JIT compiler of Octave is simple to implement, and the hot code is compiled into machine code for acceleration, without fully utilizing GPU computing resources.

[0064] Figure 2 An execution flowchart for an actual Octave example, in which only the immediate compilation and execution process of the hot code are described:

[0065] 1) The source code is analyzed by lexical analysis and syntax analysis to obtain a syntax tree

[0066] 2) Get the syntax tree node

[0067] 3) If the current syntax tree node is the loop code block entry, execute 4), otherwise execute 11)

[0068] 4) If there is compiled machine code, execute 10), otherwise execute 5)

[0069] 5) The number of executions of the current code block is incremented by 1

[0070] 6) If the number of executions of the current code block is greater than the threshold, execute 7), otherwise execute 11)

[0071] 7) Convert the code block to Octave IR

[0072] 8) Convert the Octave IR to LLVM IR

[0073] 9) Get the machine code by immediate compilation

[0074] 10) Execute the machine code, and execute 12)

[0075] 11) The interpreter interprets and executes the code block, and executes 12)

[0076] 12) If the program is executed, then perform 13), otherwise perform 2)

[0077] 13) End

[0078] In this example, Octave compiles the hot code into machine code and executes it immediately, but the GPU computing resources are not fully utilized. To solve the above problems, the present application provides a numerical calculation software acceleration method based on CPU-GPU cooperation, Figure 3 The CPU-GPU cooperative execution architecture is shown in the figure. The CPU just-in-time compiler is basically the same as the Octave just-in-time compiler, and will not be described again. The GPU just-in-time compiler converts the code into OpenCL code and further performs GPU compilation to obtain GPU code.

[0079] First, the source code is analyzed by lexical syntax analysis to obtain basic blocks.

[0080] The basic block is analyzed by the code analysis module to estimate the running information, including whether the basic block is a compute-intensive task, whether it is easy to parallelize, and whether the data copy time between heterogeneous devices is less than the running time on the CPU (GPU operation start time threshold). If the above conditions are met, mark it as a GPU module, otherwise mark it as a CPU module.

[0081] The GPU module data type is converted to OpenCL data type, the GPU module code is converted to OpenCL code, and further GPU compilation is performed to obtain GPU code.

[0082] The CPU module is compiled into CPU code for execution if there is hot code, and the rest of the code is interpreted and executed; otherwise, it is interpreted and executed.

[0083] After the execution of the CPU module and the GPU module, the variables need to be written back to the symbol table for synchronization.

[0084] Figure 4 The CPU-GPU cooperative example execution flowchart is shown in the figure:

[0085] 1) After the source code is analyzed by lexical analysis and syntax analysis to obtain a syntax tree, the basic blocks are divided;

[0086] 2) Get the basic block;

[0087] 3) The code analysis module estimates the basic block running information, and divides the basic block into a CPU module or a GPU module

[0088] 4) If it is a CPU module, perform 5); if it is a GPU module, perform 8)

[0089] 5) If there is hot code, perform 6), otherwise perform 7)

[0090] 6) Hotspot code is compiled into machine code in real time, and the hotspot code machine code is directly executed

[0091] 7) The interpreter interprets and executes the code that is not compiled in real time (CPU platform), synchronizes the symbol table, and executes 10)

[0092] 8) The GPU module is compiled into GPU code in real time

[0093] 9) The GPU executes the GPU code, synchronizes the symbol table, and executes 10)

[0094] 10) If the program is executed, 11) is executed, otherwise 2) is executed

[0095] 11) End

[0096] In this example, the program can fully utilize the CPU and GPU computing resources, and under the cooperation of CPU-GPU, the interpretative syntax is compiled into machine code or GPU code by using the real-time compilation technology, so as to accelerate the execution of hotspot code and computationally intensive tasks. At the same time, since the GPU operation starting time threshold is set, the basic block data copying time between the heterogeneous devices is less than the time of running directly on the CPU, so it can be ensured that the CPU-GPU cooperative execution efficiency is higher than the CPU alone execution. Since the CPU undertakes the hotspot code and the remaining code computing task, the CPU-GPU cooperative execution efficiency is also higher than the GPU alone execution.

[0097] The preferred embodiments of the present application are described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and changes without creative work based on the concept of the present application. Therefore, any technical solutions obtained by logical analysis, reasoning or limited experiments based on the prior art within the concept of the present application should be within the protection scope determined by the claims.

Claims

1. A CPU-GPU cooperative based numerical computation software acceleration method, characterized in that, Comprise: S1), syntax tree construction, type inference on the source program; S2), the source program is divided into basic blocks; Each basic block meets: (1) control flow can only enter from the entry statement of the basic block, the entry statement is the first statement of the program, the target statement of the jump or the next statement of the conditional jump; (2) control flow can only exit from the end statement of the basic block, the end statement is the jump statement, the stop statement or the previous statement of the next entry statement; S3), estimate the basic block running information through the code analysis module; S4), according to the running information, the basic block is divided into GPU module and CPU module; Specifically: Whether the basic block is a compute-intensive task, whether the task size is greater than the threshold, whether the memory copy time in the computing task is less than the CPU execution time; If the analysis result is correct, mark this basic block as GPU module, otherwise mark as CPU module; S5), for GPU module, get GPU code by just-in-time compilation; Specifically: Convert the data type of the GPU module to OpenCL data type, convert the GPU module code to OpenCL code, and further perform GPU compilation to get GPU code; S6), for CPU module, perform just-in-time compilation or interpretive execution; Specifically: Analyze the CPU module, if it is a hot code, then compile the hot code into machine code; If it is not a hot code, directly interpretive execution.

2. A CPU-GPU cooperative based numerical computing software acceleration system, characterized in that, The system uses the method of claim 1 to realize acceleration, and the system comprises: A source program division module, which divides the source program into basic blocks; A running information estimation module, which estimates the basic block running information through the code analysis module; A basic block division module, which divides the basic block into GPU module and CPU module according to the running information; A GPU running module, which gets GPU code by just-in-time compilation for GPU module; A CPU running module, which performs just-in-time compilation or interpretive execution for CPU module.

Images