Code optimization method and device, computer device and readable storage medium
By instrumenting and optimizing the model, GPU code is optimized, which solves the problem of low efficiency in existing GPU code optimization technologies. It achieves efficient and accurate GPU code optimization, improves running performance, and saves manpower.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 沐曦科技(成都)有限公司
- Filing Date
- 2024-12-09
- Publication Date
- 2026-06-09
Smart Images
Figure CN122173132A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a code optimization method, apparatus, computer device, and readable storage medium. Background Technology
[0002] In the field of computer technology, the application of graphics processing units (GPUs) is becoming increasingly widespread, playing a vital role in many fields such as graphics rendering, scientific computing, and artificial intelligence.
[0003] Currently, to improve the performance of GPU code, developers often need to analyze the GPU code and continuously try various compilation options, repeatedly experimenting and adjusting to find GPU code with better performance. Clearly, current code optimization methods suffer from inefficiency. Summary of the Invention
[0004] Therefore, it is necessary to provide a code optimization method, apparatus, computer device, and readable storage medium that can improve efficiency in response to the above-mentioned technical problems.
[0005] Firstly, this application provides a code optimization method, including:
[0006] Instrument the initial GPU code based on the instrumentation information to obtain the intermediate GPU code;
[0007] Run intermediate GPU code to obtain information on the execution of basic blocks in the initial GPU code across various GPU threads;
[0008] The initial GPU code is optimized based on the runtime information to obtain the target GPU code.
[0009] In one embodiment, the initial GPU code is optimized based on runtime information to obtain the target GPU code, which includes at least one of the following:
[0010] Based on the runtime information, determine the weights corresponding to the basic blocks in the initial GPU code to obtain the target GPU code;
[0011] If the runtime information meets the first preset condition, the indirect call function in the basic block of the initial GPU code is updated to the direct call function to obtain the target GPU code;
[0012] If the runtime information meets the second preset condition, the initial GPU code is inlined to obtain the target GPU code;
[0013] Based on the runtime information, determine the stripping loop unrolling factor corresponding to the basic block in the initial GPU code to obtain the target GPU code.
[0014] In one embodiment, the runtime information includes the runtime frequency of basic blocks in the initial GPU code; based on the runtime information, the weights corresponding to the basic blocks in the initial GPU code are determined to obtain the target GPU code, including:
[0015] The weights of the basic blocks are determined based on their execution frequency in the initial GPU code, thus obtaining the target GPU code.
[0016] In one embodiment, the runtime information includes the call frequency of indirectly called functions in the basic blocks of the initial GPU code; when the runtime information meets a first preset condition, the indirectly called functions in the basic blocks of the initial GPU code are updated to directly called functions to obtain the target GPU code, including:
[0017] If the frequency of indirect function calls in the basic blocks of the initial GPU code is greater than the first preset threshold, then the runtime information is determined to meet the first preset condition.
[0018] If the runtime information meets the first preset condition, the indirect call function in the basic block of the initial GPU code with a call frequency greater than the first preset threshold will be updated to the direct call function.
[0019] In one embodiment, the runtime information includes the runtime frequency of basic blocks in the initial GPU code; if the runtime information meets a second preset condition, the initial GPU code is inlined to obtain the target GPU code, including:
[0020] If the execution frequency of the target basic block in the initial GPU code is greater than the second preset threshold, then the execution information is determined to meet the second preset condition.
[0021] If the runtime information meets the second preset condition, the target basic block and the function basic block are inlined to obtain the target GPU code; the function basic block is the basic block corresponding to the function called in the target basic block.
[0022] In one embodiment, the runtime information includes the runtime frequency of basic blocks in the initial GPU code; based on the runtime information, the stripping loop unrolling factor corresponding to the basic blocks in the initial GPU code is determined to obtain the target GPU code, including:
[0023] If the running frequency of the target basic block in the initial GPU code is greater than the preset frequency threshold, the stripping loop unrolling factor corresponding to the target basic block is determined according to the preset correspondence and the running frequency of the target basic block, and the target GPU code is obtained; the preset correspondence includes the correspondence between the running frequency and the stripping loop unrolling factor.
[0024] In one embodiment, the initial GPU code is instrumented according to the instrumentation information to obtain intermediate GPU code, including:
[0025] Analyze the initial GPU code to identify the basic blocks within it;
[0026] Instrument each basic block according to the instrumentation information to obtain intermediate GPU code.
[0027] In one embodiment, intermediate GPU code is run to obtain runtime information of the basic blocks in the initial GPU code across various GPU threads, including:
[0028] The runtime environment for generating intermediate GPU code;
[0029] Run intermediate GPU code based on the runtime environment to obtain runtime information.
[0030] Secondly, this application also provides a code optimization apparatus, comprising:
[0031] The instrumentation module is used to instrument the initial GPU code based on the instrumentation information to obtain intermediate GPU code.
[0032] The acquisition module is used to run intermediate GPU code to obtain the execution information of the basic blocks in the initial GPU code in each GPU thread;
[0033] The optimization module is used to optimize the initial GPU code based on runtime information to obtain the target GPU code.
[0034] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.
[0035] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.
[0036] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.
[0037] The aforementioned code optimization method, apparatus, computer device, and readable storage medium can instrument the initial GPU code based on instrumentation information to obtain intermediate GPU code, and then run the intermediate GPU code to obtain the execution information of the basic blocks in the initial GPU code across various GPU threads. Therefore, the execution information reflects the actual cross-thread execution of the basic blocks in the initial GPU code, making it suitable for the GPU's SIMT architecture. Furthermore, by optimizing the initial GPU code based on the execution information, the initial GPU code can be optimized efficiently and accurately according to the actual execution situation, improving its performance and resulting in target GPU code with better execution performance—that is, the target GPU code's performance will be greater than that of the initial GPU code. This process eliminates the need for manual optimization by developers, thus not only improving the efficiency of GPU code optimization but also saving manpower. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 This is a diagram illustrating the application environment of the code optimization method in one embodiment;
[0040] Figure 2 This is a flowchart illustrating a code optimization method in one embodiment;
[0041] Figure 3 This is a flowchart illustrating the process of obtaining intermediate GPU code in one embodiment;
[0042] Figure 4 This is a flowchart illustrating the process of obtaining runtime information in one embodiment;
[0043] Figure 5 This is a schematic diagram illustrating a code optimization method in one embodiment.
[0044] Figure 6 This is a schematic diagram illustrating another code optimization method in one embodiment;
[0045] Figure 7 This is a schematic diagram illustrating another code optimization method in one embodiment;
[0046] Figure 8 This is a schematic diagram illustrating the effect of a code optimization in one embodiment;
[0047] Figure 9 This is a schematic diagram illustrating the effect of another code optimization in one embodiment.
[0048] Figure 10 This is a schematic diagram illustrating the effect of another code optimization in one embodiment.
[0049] Figure 11 This is a structural block diagram of a code optimization device in one embodiment. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0051] In the field of computer technology, the application of graphics processing units (GPUs) is becoming increasingly widespread, playing a vital role in many fields such as graphics rendering, scientific computing, and artificial intelligence.
[0052] Currently, in order to improve the running performance of GPU code, developers often need to analyze the GPU code and constantly try various different compilation options, searching for GPU code with better running performance through repeated trials and adjustments.
[0053] In some applications, methods for automatically optimizing Central Processing Unit (CPU) code have emerged, such as Profile-Guided Optimization (PGO) for Low Level Virtual Machines (LLVM). However, CPUs and GPUs have different architectures; GPUs use a Single Instruction, Multiple Threads (SIMT) architecture. Therefore, methods for optimizing GPU code are not suitable for GPU code optimization scenarios.
[0054] Based on this, this embodiment provides an efficient method for optimizing GPU code, which will be described below.
[0055] Figure 1 This is an application environment diagram of the code optimization method in one embodiment. In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows. Figure 1As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores relevant data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a code optimization method.
[0056] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0057] Figure 2 This is a flowchart illustrating a code optimization method in one embodiment. In an exemplary embodiment, such as... Figure 2 As shown, a code optimization method is provided, which can be applied to... Figure 1 Taking a computer device as an example, it should be noted that in some embodiments, the computer device may include a compiler that executes the code optimization method. The code optimization method includes the following steps S201 to S203.
[0058] S201, instrument the initial GPU code according to the instrumentation information to obtain the intermediate GPU code.
[0059] In this embodiment, the initial GPU code is the code applied to the GPU, which can be determined based on the GPU's high-level programming language. For example, the initial GPU code could be the Compute Unified Device Architecture (CUDA) code.
[0060] Computer devices can determine instrumentation information. This instrumentation information can be used to record the execution information of basic blocks (BBs) in the initial GPU code across various GPU threads.
[0061] Optionally, the instrumentation information may include the instrumentation content within the initial GPU code. In one embodiment, the instrumentation information may further include the location of the instrumentation content within the initial GPU code.
[0062] Optionally, the instrumentation content may include instrumentation code. For example, instrumentation code includes, but is not limited to, a counter for counting, a callsite for recording function calls, a variable or function for recording global memory traces, etc., and this embodiment is not limited to these.
[0063] In one embodiment, instrumentation information may include global variables capable of recording runtime information, such as an array of global variables. Further optionally, the instrumentation information may also include statistical rules for these global variables. For example, the statistical rules may include counting the corresponding global variable once if the GPU thread is active. This facilitates cross-thread statistics, making it suitable for application scenarios involving GPU code.
[0064] In one embodiment, a user can specify instrumentation information and the instrumentation location of the instrumentation information in the initial GPU code, and initiate an insertion operation. The computer device can then respond to the user's insertion operation by instrumenting the initial GPU code according to the instrumentation information, thereby obtaining intermediate GPU code.
[0065] In some embodiments, the computer device may also analyze the initial GPU code to determine the corresponding instrumentation information, and automatically instrument the initial GPU code according to the instrumentation information to obtain intermediate GPU code. This embodiment is not limited.
[0066] S202 runs intermediate GPU code to obtain information on the execution of basic blocks in the initial GPU code across various GPU threads.
[0067] In this embodiment, the computer device is capable of running intermediate GPU code, and this embodiment does not limit the number of times the intermediate GPU code can be run. Optionally, the computer device can run the intermediate GPU code multiple times. For example, the computer device can run the intermediate GPU code 1000 times.
[0068] Since the intermediate GPU code is obtained by instrumenting the initial GPU code based on the instrumentation information, running the intermediate GPU code can obtain the execution information of the basic blocks in the initial GPU code in each GPU thread.
[0069] The basic block is obtained by logically dividing the initial GPU code. Optionally, one basic block corresponds to one execution logic. The execution logic can include, but is not limited to, conditional statements, loop structures, etc.
[0070] The runtime information can be the number of times or frequency of execution of a basic block, the number of times or frequency of function calls within the basic block, or the number of times or frequency of reads and writes to the memory space corresponding to the basic block. This embodiment does not limit these.
[0071] Taking the initial GPU code as having basic blocks including basic block A, basic block B, and basic block C, and GPU threads including thread A and thread B as an example, after running the initial GPU code, the computer device can obtain the running information of basic block A in threads A and B, the running information of basic block B in threads A and B, and the running information of basic block C in threads A and B.
[0072] Optionally, after the computer device runs the intermediate GPU code on the GPU, the GPU saves the runtime information in a preset format (e.g., profile_data format) and obtains the corresponding file (e.g., a profile file). Further, the GPU can send the obtained file to the Central Processing Unit (CPU) via the Peripheral Component Interconnect Express (PCIe). The CPU then sends the file to the hard disk via the Input / Output Interface (I / O interface). The computer device retrieves the file from the hard disk and parses it to obtain the runtime information.
[0073] S203 optimizes the initial GPU code based on the runtime information to obtain the target GPU code.
[0074] In this embodiment, optionally, the computer device can input the runtime information and the initial GPU code into the trained optimization model, and the optimization model can output the target GPU code, which is the code obtained after optimizing the initial GPU code based on the runtime information.
[0075] The optimization model can be determined based on a first GPU code sample, the corresponding runtime information of the first GPU code sample, and a second GPU code sample corresponding to the first GPU code sample. The second GPU code sample is obtained by optimizing the first GPU code sample based on the runtime information corresponding to the first GPU code sample.
[0076] The optimization model can be a supervised learning model, a semi-supervised learning model, an unsupervised learning model, etc. For example, the optimization model can include, but is not limited to, at least one of the following: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Fully Convolutional Neural Networks (FCN), Generative Adversarial Networks (GAN), Back-propagation (BP) machine learning models, Radial Basis Function (RBF) models, Deep Belief Networks (DBN) models, Elman models, or combinations thereof.
[0077] Of course, computer devices can also optimize the initial GPU code based on the running information in other ways to obtain the target GPU code, and this embodiment is not limited to this.
[0078] Furthermore, the computer device can obtain the corresponding executable file based on the target GPU code.
[0079] In the code optimization method described above, the initial GPU code is instrumented based on instrumentation information to obtain intermediate GPU code. Running the intermediate GPU code then reveals the execution information of the basic blocks in the initial GPU code across various GPU threads. This execution information reflects the actual cross-thread execution of the basic blocks in the initial GPU code, making it suitable for the GPU's SIMT architecture. Furthermore, optimizing the initial GPU code based on this execution information allows for efficient and accurate optimization based on actual execution conditions, improving the initial GPU code's performance and resulting in target GPU code with superior execution performance—that is, the target GPU code's performance will be greater than that of the initial GPU code. This process eliminates the need for manual optimization by developers, thus improving GPU code optimization efficiency and saving manpower.
[0080] In one exemplary embodiment, optionally, S203 above includes at least one of the following:
[0081] (1) Based on the running information, determine the weights corresponding to the basic blocks in the initial GPU code to obtain the target GPU code.
[0082] In this embodiment, the weight corresponding to a basic block can represent the execution priority of the basic block. Optionally, the weight corresponding to a basic block can be positively correlated with the execution priority of the block. That is, the higher the weight corresponding to a basic block, the higher the execution priority of that basic block in the initial GPU code, i.e., the higher it will be executed.
[0083] Furthermore, the computer device can determine the weights corresponding to the basic blocks in the initial GPU code based on the runtime information, thereby obtaining the target GPU code. For example, if the runtime information of the basic blocks is within a first preset range, the computer device can continue to use the weights corresponding to the basic blocks in the initial GPU code as the weights corresponding to the basic blocks in the target GPU code. If the runtime information of the basic blocks is not within the first preset range, the weights corresponding to the basic blocks in the initial GPU code are updated to the first preset value to obtain the target GPU code. Alternatively, the computer device can store the correspondence between runtime information and weights. In this way, after determining the actual runtime information, the computer can update the weights corresponding to the basic blocks in the initial GPU code based on the correspondence between runtime information and weights, as well as the actual runtime information, to obtain the target GPU code.
[0084] (2) If the running information meets the first preset condition, the indirect call function in the basic block of the initial GPU code is updated to the direct call function to obtain the target GPU code.
[0085] In this embodiment, the computer device determines whether the running information meets the first preset condition. Optionally, the computer device determines that the running information meets the first preset condition if the running information of the basic block is within a second preset range.
[0086] Furthermore, if the runtime information meets the first preset condition, the computer device updates the indirect call functions in the basic blocks of the initial GPU code to direct call functions, thereby obtaining the target GPU code.
[0087] Optionally, the computer device may update all indirect call functions in the basic block of the initial GPU code to direct call functions, or it may update some indirect call functions in the basic block of the initial GPU code to direct call functions. This embodiment does not impose any restrictions.
[0088] (3) If the running information meets the second preset condition, the initial GPU code is inlined to obtain the target GPU code.
[0089] In this embodiment, the computer device determines whether the running information meets the second preset condition. Optionally, the computer device determines that the running information meets the second preset condition if the running information of the basic block is within a third preset range.
[0090] Furthermore, if the runtime information meets the second preset condition, the computer device will inline the initial GPU code to obtain the target GPU code. For example, if the basic block is within the third preset range, the computer device can directly insert the code of the called function in the basic block into the call site, instead of jumping through the conventional function call mechanism.
[0091] (4) Based on the running information, determine the stripping loop unrolling factor corresponding to the basic block in the initial GPU code to obtain the target GPU code.
[0092] In this embodiment, the PeelLoop factor is the number of times loop peeling is performed. Loop peeling is used to remove misaligned references at the beginning of the loop and data at the end of the loop that is not large enough to fit into a vector register, ensuring that the remaining data meets vector alignment requirements. In other words, loop peeling can separate special cases at the beginning or end of the loop from the loop body for separate processing, allowing the loop body to execute in a more regular and efficient manner. Optionally, the PeelLoop factor can instruct the loop to be peeled into at least two parts. For example, assuming a loop structure needs to iterate 10 times, a PeelLoop factor of 2 will result in the loop being peeled into two iterations followed by the last eight iterations.
[0093] Furthermore, the computer device can determine the unrolling factor corresponding to the basic blocks in the initial GPU code based on the runtime information, thereby obtaining the target GPU code. For example, if the runtime information of the basic blocks is within a fourth preset range, the computer device can continue to use the unrolling factor corresponding to the basic blocks in the initial GPU code as the unrolling factor corresponding to the basic blocks in the target GPU code. If the runtime information of the basic blocks is not within the fourth preset range, the unrolling factor corresponding to the basic blocks in the initial GPU code is updated to a second preset value to obtain the target GPU code. Alternatively, the computer device can store the correspondence between runtime information and unrolling factors. Thus, after determining the actual runtime information, the computer device can update the unrolling factors corresponding to the basic blocks in the initial GPU code based on the correspondence between runtime information and unrolling factors, and the actual runtime information, to obtain the target GPU code.
[0094] In the above embodiments, since the target GPU code can be obtained by at least one of the following methods: determining the weights corresponding to the basic blocks in the initial GPU code, updating the indirect call functions in the basic blocks of the initial GPU code to direct call functions, inlining the initial GPU code to obtain the target GPU code, and determining the stripping loop unrolling factor corresponding to the basic blocks in the initial GPU code, the initial GPU code can be optimized efficiently and flexibly based on the running information to obtain the target GPU code with better running performance.
[0095] In an exemplary embodiment, optionally, the runtime information includes the runtime frequency of basic blocks in the initial GPU code. The aforementioned "determining the weights corresponding to the basic blocks in the initial GPU code based on the runtime information to obtain the target GPU code" can be implemented in the following way:
[0096] The weights of the basic blocks are determined based on their execution frequency in the initial GPU code, thus obtaining the target GPU code.
[0097] In this embodiment, the runtime information includes the runtime frequency of basic blocks in the initial GPU code. Continuing with the example of basic block A in S202 above, the runtime information can include the runtime frequency of basic block A, which indicates the sum of the runtime frequencies of basic block A in thread A and thread B. Assuming the initial GPU code runs a total of 1000 times, and basic block A runs 300 times in thread A and 500 times in thread B, then the runtime frequency of basic block A can be (300+500) / 1000*100%=80%. The same applies to other basic blocks, and will not be elaborated here.
[0098] Optionally, the computer device can determine the weight of a basic block based on the actual running frequency and the correspondence between running frequency and weight. Further, optionally, the running frequency and weight are positively correlated. That is, the higher the running frequency of a basic block, the higher its weight, indicating that the basic block is executed with higher priority.
[0099] Taking a weight between 0 and 1 as an example, continuing the example above, suppose that in the correspondence between running frequency and weight: a running frequency of 10% corresponds to a weight of 0.1, a running frequency of 20% corresponds to a weight of 0.2, ... a running frequency of 80% corresponds to a weight of 0.8, and so on. If the running frequency of basic block A is 80%, then the computer device can determine that the weight corresponding to basic block A is 0.8.
[0100] In the above embodiments, since the weights of basic blocks can be determined based on their running frequency in the initial GPU code, and the target GPU code can be obtained, the execution order of basic blocks can be determined by their running frequency. This is beneficial for prioritizing the execution of basic blocks with high running frequency, thereby improving running efficiency and thus improving the running performance of the target GPU code.
[0101] In an exemplary embodiment, optionally, the runtime information includes the call frequency of indirectly called functions in the basic blocks of the initial GPU code; the aforementioned "updating the indirectly called functions in the basic blocks of the initial GPU code to directly called functions to obtain the target GPU code when the runtime information meets the first preset condition" may further include the following steps:
[0102] If the call frequency of indirect function calls in the basic blocks of the initial GPU code is greater than a first preset threshold, then the runtime information is determined to meet the first preset condition; if the runtime information meets the first preset condition, the indirect function calls in the basic blocks of the initial GPU code with a call frequency greater than the first preset threshold are updated to direct function calls.
[0103] In this embodiment, the runtime information includes the call frequency of indirectly called functions in the basic blocks of the initial GPU code. Continuing with basic block A as an example, assuming that basic block A includes indirectly called function A and indirectly called function B, the runtime information may include the call frequency of indirectly called function A and the call frequency of indirectly called function B.
[0104] Furthermore, the computer device can determine whether the runtime information meets the first preset condition based on whether the call frequency is greater than the first preset threshold. Thus, if the call frequency of indirectly called functions in the basic blocks of the initial GPU code is greater than the first preset threshold, it can be determined that the runtime information meets the first preset condition. Further, the computer device will update the indirectly called functions in the basic blocks of the initial GPU code with a call frequency greater than the first preset threshold to directly called functions.
[0105] Continuing with the example above, assuming the first preset threshold is 60%, if the call frequency of indirect function A is 30% and the call frequency of indirect function B is 80%, then the running information meets the first preset condition, and the computer device updates the indirect function B to the direct function.
[0106] In the above embodiments, since the indirect function call frequency in the basic block of the initial GPU code is greater than the first preset threshold, it can be determined that the running information meets the first preset condition. When the running information meets the first preset condition, the indirect function call frequency in the basic block of the initial GPU code that is greater than the first preset threshold is updated to the direct function call. Therefore, on the one hand, after updating the indirect function call to the direct function call, the direct function call does not need to perform a search jump like the indirect function call, which improves the running speed of the target GPU code. On the other hand, in some scenarios, if the called function in the basic block is called frequently, while the called function is only called once or a few times in other basic blocks, then only one basic block needs to be optimized, and the code corresponding to other basic blocks does not need to be optimized, thus improving optimization efficiency.
[0107] In an exemplary embodiment, optionally, the runtime information includes the runtime frequency of basic blocks in the initial GPU code; the aforementioned "inlining the initial GPU code to obtain the target GPU code when the runtime information meets the second preset condition" may include the following steps:
[0108] If the execution frequency of the target basic block in the initial GPU code is greater than the second preset threshold, then the execution information is determined to meet the second preset condition. If the execution information meets the second preset condition, the target basic block and the function basic block are inlined to obtain the target GPU code. The function basic block is the basic block corresponding to the function called in the target basic block.
[0109] In this embodiment, the runtime information includes the runtime frequency of basic blocks in the initial GPU code. The runtime frequency of basic blocks can be found in the above embodiments and will not be repeated here.
[0110] Furthermore, the computer device can determine whether the execution information meets the second preset condition based on whether the call frequency is greater than the second preset threshold. Thus, if the execution frequency of the target basic block in the initial GPU code is greater than the second preset threshold, it can be determined that the execution information meets the second preset condition.
[0111] Continuing with the example above, assuming the second preset frequency is 70%, since the operating frequency of basic block A is 80%, the operating information meets the second preset condition, and the target basic block is also basic block A.
[0112] Furthermore, if the runtime information meets the second preset condition, the computer device will inline the target basic block and the function basic block to obtain the target GPU code. Here, the function basic block is the basic block corresponding to the function called in the target basic block. Continuing with the example of basic block A as the target basic block, assuming that basic block A calls function 1 in basic block B and function 2 in basic block C, then the function basic blocks include basic block B and basic block C. The computer device inlines basic blocks A, B, and C in the initial GPU code to obtain the target optimized code.
[0113] In the above embodiments, since the running frequency of the target basic block in the initial GPU code is greater than the second preset threshold, it is determined that the running information meets the second preset condition. When the running information meets the second preset condition, the target basic block and the function basic block are inlined to obtain the target GPU code. The function basic block is the basic block corresponding to the function called in the target basic block. Therefore, on the one hand, inlining can reduce the running time caused by querying and jumping to the corresponding function. On the other hand, inlining only the target basic block and the function basic block can ensure that the high-frequency function calls are inlined, improve running efficiency, and avoid the excessive size of other basic blocks that do not need to be inlined, so that the size of the target GPU code is not too large.
[0114] In an exemplary embodiment, optionally, the runtime information includes the runtime frequency of basic blocks in the initial GPU code; the aforementioned "determining the stripping loop unrolling factor corresponding to the basic blocks in the initial GPU code based on the runtime information to obtain the target GPU code" can be implemented in the following way:
[0115] If the running frequency of the target basic block in the initial GPU code is greater than the preset frequency threshold, the stripping loop unrolling factor corresponding to the target basic block is determined according to the preset correspondence and the running frequency of the target basic block, and the target GPU code is obtained; the preset correspondence includes the correspondence between the running frequency and the stripping loop unrolling factor.
[0116] The runtime information includes the runtime frequency of basic blocks in the initial GPU code. The runtime frequency of these basic blocks can be found in the above embodiments and will not be repeated here.
[0117] In this embodiment, the computer device can determine a preset correspondence between the operating frequency and the stripping cycle unrolling factor. For example, in the preset correspondence, an operating frequency of 60% corresponds to stripping cycle unrolling factor A, an operating frequency of 80% corresponds to stripping cycle unrolling factor B, and so on.
[0118] Furthermore, if the operating frequency of the target basic block in the initial GPU code is greater than a preset frequency threshold, the computer device can determine the stripping loop unrolling factor corresponding to the target basic block based on the operating frequency of the target basic block and the aforementioned preset correspondence. For example, assuming the preset frequency threshold is 75% and the operating frequency of basic block A is 80%, then the target basic block includes basic block A. Based on the operating frequency of basic block A and the aforementioned preset correspondence, the computer device can determine that the stripping loop unrolling factor corresponding to basic block A is stripping loop unrolling factor B, thereby obtaining the target GPU code.
[0119] In the above embodiments, since the running frequency of the target basic block in the initial GPU code is greater than the preset frequency threshold, the stripping loop unrolling factor corresponding to the target basic block can be determined according to the preset correspondence and the running frequency of the target basic block, thereby obtaining the target GPU code. The preset correspondence includes the correspondence between the running frequency and the stripping loop unrolling factor. Different stripping loop unrolling factors correspond to different running performance. Therefore, by using the actual cross-thread running situation of the basic block in the initial GPU code, a better stripping loop unrolling factor can be efficiently determined, thereby obtaining the target GPU code with better running performance.
[0120] Figure 3 This is a flowchart illustrating the process of obtaining intermediate GPU code in one embodiment. In an exemplary embodiment, such as... Figure 3 As shown, S201 includes S301 to S302.
[0121] S301, analyze the initial GPU code to identify the basic blocks in the initial GPU code.
[0122] In this embodiment, the computer device is able to acquire GPU code. Optionally, the computer device can receive initial GPU code input by the user, or it can acquire initial GPU code from a preset storage space.
[0123] Furthermore, the computer device can analyze the initial GPU code to identify the basic blocks within it. Optionally, the computer device can convert the initial GPU code into an intermediate representation (IR) and identify the basic blocks within the initial GPU code based on the intermediate representation.
[0124] S302 instrumentes each basic block according to the instrumentation information to obtain intermediate GPU code.
[0125] In this embodiment, after determining the basic blocks in the initial GPU code, instrumentation can be performed on each basic block according to the instrumentation information to obtain intermediate GPU code. This embodiment does not limit the position of the instrumentation information in each basic block; the instrumentation information corresponding to each basic block can be the same, different, or not completely the same.
[0126] In the above embodiments, since the initial GPU code can be analyzed, the basic blocks in the initial GPU code can be determined, and each basic block can be instrumented according to the instrumentation information to obtain the intermediate GPU code, the running information of the basic blocks in the initial GPU code in each GPU thread can be obtained after running the intermediate GPU code.
[0127] Figure 4 This is a flowchart illustrating the process of obtaining runtime information in one embodiment. In an exemplary embodiment, such as... Figure 4 As shown, S202 includes S401 to S402.
[0128] S401 is the runtime environment for generating intermediate GPU code.
[0129] For example, a computer device can lower instrumentation information and create a profiling runtime. Hook functions are inserted into the profiling runtime to generate an instrumented kernel and initialize prof structure data, thereby generating a runtime environment for GPU code. Lowering the instrumentation information is the process of converting high-level language instrumentation information into a lower-level representation that is closer to the hardware.
[0130] S402 runs intermediate GPU code based on the runtime environment to obtain runtime information.
[0131] In this embodiment, after generating the runtime environment for intermediate GPU code, the intermediate GPU code can be run based on the runtime environment, thereby obtaining runtime information.
[0132] In the above embodiments, since an intermediate GPU code runtime environment can be generated and the intermediate GPU code can be run based on the runtime environment to obtain runtime information, users only need to set up the runtime environment, and then they can perform one-click fully automatic optimization based on the runtime environment without having to pay attention to optimization details, and can efficiently obtain target GPU code with high running performance.
[0133] To more clearly illustrate the code optimization methods used in this application, this section combines... Figure 5 , Figure 6 and Figure 7 Please provide an explanation. Figure 5This is a schematic diagram of a code optimization method in one embodiment. In one embodiment, a computer device can execute the code optimization method according to the following process.
[0134] S501 analyzes the initial GPU code to identify the basic blocks within it.
[0135] S502 instrumentes each basic block according to the instrumentation information to obtain intermediate GPU code.
[0136] S503 is the runtime environment for generating intermediate GPU code.
[0137] S504 runs intermediate GPU code based on the runtime environment to obtain the runtime information of the basic blocks in the initial GPU code in each GPU thread.
[0138] S505 optimizes the initial GPU code based on the runtime information to obtain the target GPU code.
[0139] S501~S505 can be referred to the above embodiments, and will not be repeated here.
[0140] Figure 6 This is a schematic diagram illustrating another code optimization method in one embodiment. Figure 7 This is a schematic diagram illustrating another code optimization method in one embodiment, which can be applied to a compiler. For example... Figure 6 As shown and Figure 7 As shown, the process begins with the initial GPU code, which is then instrumented. During this phase, the initial GPU code is instrumented based on the instrumentation information to obtain intermediate GPU code. For example, the PGO instrumentation generator (PGOinstrumentGen) or the instrumentation analysis tool (InstrProfiling) can be used to instrument the initial GPU code based on the instrumentation information to obtain intermediate GPU code.
[0141] The process then enters the runtime phase (also known as the profiling phase). After the intermediate GPU code runs based on the runtime environment, data can be collected from the runtime data and written to a file. This file is saved according to a preset format (e.g., profile_data format) to obtain the corresponding file (e.g., the profile file). Furthermore, by parsing this file, the runtime information of the basic blocks in the initial GPU code across various GPU threads can be obtained.
[0142] Finally, the optimization phase begins. In this phase, the initial GPU code is optimized based on the runtime information to obtain the target GPU code.
[0143] In the optimization process, such as Figure 7 The "weights" shown in the figure can be used to determine the weights of basic blocks in the initial GPU code based on runtime information, thus obtaining the target GPU code; for example... Figure 7 The method shown in the "direct function call" section can update the indirect function calls in the basic blocks of the initial GPU code to direct function calls, provided that the runtime information meets the first preset condition, thus obtaining the target GPU code; for example... Figure 7 The "inline" method shown in the text can also be used to inline the initial GPU code to obtain the target GPU code, provided that the runtime information meets the second preset condition; for example... Figure 7 The method shown in "Removing Loop Unrolling Factor" also determines the removal loop unrolling factor corresponding to the basic blocks in the initial GPU code based on the runtime information, thereby obtaining the target GPU code.
[0144] As can be seen, this application provides a PGO method based on high-level programming languages for GPUs (such as CUDA). By instrumenting high-level code, it obtains the execution information of basic blocks in the initial GPU code in each GPU thread, and then optimizes the initial GPU code based on the execution information by inlining, setting weights, changing indirect calls to direct calls, and determining the loop unrolling factor, etc., to obtain target GPU code with better running performance.
[0145] Figure 8 , Figure 9 and Figure 10 These are schematic diagrams illustrating the effect of code optimization in one embodiment. Figure 8 The horizontal axis represents the different GPU functions in the GPU code. Figure 8 The vertical axis represents the percentage improvement in the runtime performance of the corresponding GPU function after using the code optimization method provided in this embodiment. For example, the swish-cuda function achieved a 5% performance improvement after using the code optimization method provided in this embodiment.
[0146] Figure 9 The horizontal axis represents different data types, such as "INT16" for 16-bit integers, "FP32" for single-precision floating-point numbers, and "FP64" for double-precision floating-point numbers. The vertical axis represents runtime (in milliseconds). For the same data type, the left bar represents the runtime speed of the initial GPU code, and the right bar represents the runtime speed of the target GPU code. It is evident that the code optimization method described in this application can improve runtime performance for different data types.
[0147] Figure 10The horizontal axis represents different data scales; for example, "num_items=1" represents a data scale of 1 unit, and so on. For the same data scale, the left bar represents the running speed of the initial GPU code, and the right bar represents the running speed of the target GPU code. It is evident that the code optimization method described in this application can improve performance for different data scales.
[0148] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0149] Based on the same inventive concept, this application also provides a code optimization apparatus for implementing the code optimization method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more code optimization apparatus embodiments provided below can be found in the limitations of the code optimization method described above, and will not be repeated here.
[0150] Figure 11 This is a structural block diagram of a code optimization device in one embodiment. In an exemplary embodiment, such as... Figure 11 As shown, a code optimization device 1100 is provided, including: an instrumentation module 1101, an acquisition module 1102, and an optimization module 1103, wherein:
[0151] Instrumentation module 1101 is used to instrument the initial GPU code according to the instrumentation information to obtain intermediate GPU code.
[0152] The acquisition module 1102 is used to run intermediate GPU code to obtain the execution information of the basic blocks in the initial GPU code in each GPU thread.
[0153] The optimization module 1103 is used to optimize the initial GPU code based on the runtime information to obtain the target GPU code.
[0154] In the aforementioned code optimization device, because it can instrument the initial GPU code according to the instrumentation information to obtain intermediate GPU code, and then run the intermediate GPU code to obtain the execution information of the basic blocks in the initial GPU code across various GPU threads, the execution information can reflect the actual cross-thread execution of the basic blocks in the initial GPU code, which is suitable for the GPU's SIMT architecture. Furthermore, by optimizing the initial GPU code based on the execution information, the initial GPU code can be optimized efficiently and accurately according to the actual execution situation, improving its performance and resulting in target GPU code with better execution performance; that is, the performance of the target GPU code will be greater than that of the initial GPU code. In this process, no manual optimization by developers is required, thus not only improving the optimization efficiency of GPU code but also saving manpower.
[0155] Optionally, optimization module 1103 is used for at least one of the following:
[0156] Based on the runtime information, determine the weights corresponding to the basic blocks in the initial GPU code to obtain the target GPU code;
[0157] If the runtime information meets the first preset condition, the indirect call function in the basic block of the initial GPU code is updated to the direct call function to obtain the target GPU code;
[0158] If the runtime information meets the second preset condition, the initial GPU code is inlined to obtain the target GPU code;
[0159] Based on the runtime information, determine the stripping loop unrolling factor corresponding to the basic block in the initial GPU code to obtain the target GPU code.
[0160] Optionally, the runtime information includes the runtime frequency of basic blocks in the initial GPU code; the optimization module 1103 is used to determine the weights corresponding to the basic blocks based on the runtime frequency of the basic blocks in the initial GPU code, and obtain the target GPU code.
[0161] Optionally, the runtime information includes the call frequency of indirectly called functions in the basic blocks of the initial GPU code; the code optimization device 1100 is further configured to determine that the runtime information meets the first preset condition if the call frequency of indirectly called functions in the basic blocks of the initial GPU code is greater than the first preset threshold, and update the indirectly called functions in the basic blocks of the initial GPU code with a call frequency greater than the first preset threshold to directly called functions if the runtime information meets the first preset condition.
[0162] Optionally, the running information includes the running frequency of the basic blocks in the initial GPU code; the optimization module 1103 is further configured to determine that the running information meets the second preset condition if the running frequency of the target basic block in the initial GPU code is greater than the second preset threshold; and in the case that the running information meets the second preset condition, to inline the target basic block and the function basic block to obtain the target GPU code; the function basic block is the basic block corresponding to the function called in the target basic block.
[0163] Optionally, the running information includes the running frequency of the basic blocks in the initial GPU code; the optimization module 1103 is used to determine the stripping loop unrolling factor corresponding to the target basic block according to the preset correspondence and the running frequency of the target basic block if the running frequency of the target basic block in the initial GPU code is greater than a preset frequency threshold, so as to obtain the target GPU code; the preset correspondence includes the correspondence between the running frequency and the stripping loop unrolling factor.
[0164] Optionally, the instrumentation module 1101 is used to analyze the initial GPU code, determine the basic blocks in the initial GPU code, and instrument each basic block according to the instrumentation information to obtain the intermediate GPU code.
[0165] Optionally, module 1102 is used to generate the runtime environment for intermediate GPU code; the intermediate GPU code is run based on the runtime environment to obtain runtime information.
[0166] Each module in the aforementioned code optimization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0167] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0168] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.
[0169] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0172] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A code optimization method, characterized in that, The method includes: Instrument the initial GPU code based on the instrumentation information to obtain the intermediate GPU code; Run the intermediate GPU code to obtain the execution information of the basic blocks in the initial GPU code on each GPU thread; The initial GPU code is optimized based on the runtime information to obtain the target GPU code.
2. The method according to claim 1, characterized in that, The step of optimizing the initial GPU code based on the runtime information to obtain the target GPU code includes at least one of the following: Based on the runtime information, the weights corresponding to the basic blocks in the initial GPU code are determined to obtain the target GPU code; If the running information meets the first preset condition, the indirect call function in the basic block of the initial GPU code is updated to the direct call function to obtain the target GPU code; If the running information meets the second preset condition, the initial GPU code is inlined to obtain the target GPU code; Based on the runtime information, the stripping loop unrolling factor corresponding to the basic block in the initial GPU code is determined to obtain the target GPU code.
3. The method according to claim 2, characterized in that, The runtime information includes the runtime frequency of basic blocks in the initial GPU code; the step of determining the weights corresponding to the basic blocks in the initial GPU code based on the runtime information to obtain the target GPU code includes: Based on the running frequency of the basic blocks in the initial GPU code, the weights corresponding to the basic blocks are determined, and the target GPU code is obtained.
4. The method according to claim 2, characterized in that, The runtime information includes the call frequency of indirectly called functions in the basic blocks of the initial GPU code; the step of updating the indirectly called functions in the basic blocks of the initial GPU code to directly called functions to obtain the target GPU code when the runtime information satisfies a first preset condition includes: If the frequency of indirect function calls in the basic blocks of the initial GPU code is greater than a first preset threshold, then the running information is determined to meet the first preset condition. If the running information meets the first preset condition, the indirect call function in the basic block of the initial GPU code with a call frequency greater than the first preset threshold is updated to the direct call function.
5. The method according to claim 2, characterized in that, The runtime information includes the runtime frequency of basic blocks in the initial GPU code; the step of inlining the initial GPU code to obtain the target GPU code when the runtime information satisfies a second preset condition includes: If the running frequency of the target basic block in the initial GPU code is greater than the second preset threshold, then the running information is determined to meet the second preset condition. When the running information meets the second preset condition, the target basic block and the function basic block are inlined to obtain the target GPU code; the function basic block is the basic block corresponding to the function called in the target basic block.
6. The method according to claim 2, characterized in that, The runtime information includes the runtime frequency of basic blocks in the initial GPU code; the step of determining the stripping loop unrolling factor corresponding to the basic blocks in the initial GPU code based on the runtime information to obtain the target GPU code includes: If the running frequency of the target basic block in the initial GPU code is greater than a preset frequency threshold, then the stripping loop unrolling factor corresponding to the target basic block is determined according to the preset correspondence and the running frequency of the target basic block, and the target GPU code is obtained; the preset correspondence includes the correspondence between the running frequency and the stripping loop unrolling factor.
7. The method according to any one of claims 1-6, characterized in that, The step of instrumenting the initial GPU code based on the instrumentation information to obtain intermediate GPU code includes: Analyze the initial GPU code to identify the basic blocks within it; Instrument each of the basic blocks according to the instrumentation information to obtain the intermediate GPU code.
8. The method according to any one of claims 1-6, characterized in that, The process of running the intermediate GPU code to obtain the execution information of the basic blocks in the initial GPU code on each GPU thread includes: The runtime environment for generating the intermediate GPU code; The intermediate GPU code is run based on the aforementioned runtime environment to obtain the runtime information.
9. A code optimization device, characterized in that, The device includes: The instrumentation module is used to instrument the initial GPU code based on the instrumentation information to obtain intermediate GPU code. An acquisition module is used to run the intermediate GPU code to obtain the execution information of the basic blocks in the initial GPU code in each GPU thread; An optimization module is used to optimize the initial GPU code based on the runtime information to obtain the target GPU code.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.
12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.