Compiler automatic tuning method based on discrete particle swarm optimization

Abstract

The invention relates to an automatic compiler tuning method based on a discrete particle swarm optimization algorithm, which improves the particle swarm optimization algorithm so that the particle swarm optimization algorithm can adapt to a discrete solution space of a compiler optimization problem, and proposes a speed updating strategy combining a crowding distance and fitness. The moving speed of the particles is dynamically balanced by evaluating the crowding distance and the adaptive value of a solution sample of an area where the current particles are located, so that the best optimization sequence is selected, and the performance of a compiler during automatic tuning is improved; the problem of premature convergence existing in many discrete particle swarm algorithms at present and the problem of insufficient global search capability when the problem dimension is relatively high are solved, and the automatic tuning effect of the compiler is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of computer program source program compilation optimization, and in particular, to a compiler automatic tuning method based on discrete particle swarm algorithm. Background Art

[0002] Compilers (such as GCC and LLVM) convert source programs written in programming languages ​​(such as C and C++) into executable programs and support optimization through compilation optimization options to improve the runtime performance of the program. However, predefined optimization levels (such as -O1, -O2, -O3) usually cannot achieve the best performance for each specific program, so optimization flags and sequencing need to be carefully tuned for each program, especially for applications that are sensitive to runtime performance.

[0003] But it is obviously very time-consuming and unrealistic to manually analyze and tune each new program, because this is a combinatorial optimization problem, and the sorted combinations of hundreds of optimization options constitute an exponential search space. Therefore, the research on compiler automatic tuning has become very important, which is of great significance for improving the intelligence of compilers and improving the efficiency of program operation. Especially in the field of high-performance computing (HPC), performance optimization is particularly important because these systems usually run computationally intensive tasks such as climate simulation, genome sequencing, and high-energy physics calculations. These tasks often take a long time to run and occupy a lot of computing resources, so even a small performance improvement can bring significant cost savings and computing efficiency improvements. Compiler tuning in the context of HPC can make full use of hardware characteristics and architectural advantages, reduce computing time and energy consumption, and improve the efficiency of scientific research and engineering calculations. Therefore, in the HPC scenario, compiler automatic tuning is not only the key to improving the performance of a single application, but also a necessary means to improve the overall system performance and resource utilization.

[0004] In the prior art, many methods have been proposed to use various heuristic algorithms to automatically adjust compiler optimization flags to achieve the required runtime performance of a given program. Typically, these methods use specific search strategies to iteratively test different combinations of optimization flags, and ultimately output the best-performing sequence when the termination condition is reached. Although these methods have demonstrated a certain degree of effectiveness, they still have some limitations. More specifically, since heuristic algorithms usually set optimization flags to generate sequences to evaluate their fitness, these methods often ignore the characteristics of the intermediate representation of the program, are prone to premature convergence problems, and have insufficient global search capabilities when the dimension is high. Summary of the invention

[0005] In order to overcome at least one deficiency in the prior art, the present application provides a compiler automatic tuning method based on discrete particle swarm algorithm.

[0006] In a first aspect, a compiler automatic tuning method based on a discrete particle swarm algorithm is provided, comprising:

[0007] Step S1, generating an initial search space; the samples in the initial search space are a set of feature vectors corresponding to the optimized marker sequence;

[0008] Step S2, randomly selecting K samples from the initial search space, and determining the execution speedup ratio corresponding to each sample; constructing a training set based on the K samples and the execution speedup ratio, and training a multilayer perceptron based on the training set;

[0009] Step S3, in the current iteration, for each particle, in the initial search space, select a sample closest to the particle as the sample to be evaluated; input the sample to be evaluated into the multilayer perceptron to obtain the predicted speedup ratio of the sample to be evaluated;

[0010] Step S4, sorting all predicted speedup ratios from large to small, and selecting the samples to be evaluated corresponding to the first three predicted speedup ratios in the sorting results as samples with better prediction effects;

[0011] Step S5, among the samples with better prediction effects, select the sample with the largest execution speedup ratio as the best sample for the current iteration; determine whether the current iteration reaches the maximum number of iterations, if so, output the best sample, if not, execute step S6;

[0012] Step S6, adding the best sample and its corresponding execution speedup ratio to the training set; determining the optimization marker subsequence that has a significant impact on the optimization effect based on the sample with better prediction effect, and adding the optimization marker subsequence to the initial search space;

[0013] Step S7, calculating the crowding distance of each particle, and updating the position and moving speed of each particle based on the crowding distance of each particle and the fitness of each particle;

[0014] Step S8, determine whether the current iteration is a round of search space update, if so, return to step S2, if not, return to step S3.

[0015] In one embodiment, in step S1, generating an initial search space includes:

[0016] Randomly generate multiple optimization flag sequences according to the compiler's optimization flag set;

[0017] Each sequence of optimization flags compiles the source program into a set of intermediate representations;

[0018] Extract feature vectors from the intermediate representation set to generate a feature vector set;

[0019] The set of feature vectors corresponding to all optimized logo sequences constitutes the initial search space.

[0020] In one embodiment, in step S2, determining the execution speedup ratio corresponding to each sample includes:

[0021] The source program is compiled into a binary executable file based on the optimization flag sequence corresponding to each sample; the executable file is executed to obtain the execution time; and the execution speedup ratio is determined based on the execution time and the execution time after optimization with the highest optimization level provided by the compiler.

[0022] In one embodiment, in step S6, determining an optimization marker subsequence that has a significant impact on the optimization effect based on samples with better prediction effects includes:

[0023] According to the optimization mark sequence corresponding to the samples with better prediction effect, a directed weighted graph is constructed; the nodes of the directed weighted graph are the optimization marks, the directed edges are the order relationship of the optimization marks in the sequence, and the weight of the directed edges is the number of times the order relationship occurs;

[0024] In a directed weighted graph, the directed edges whose weights do not meet the set values ​​are removed multiple times to obtain multiple subgraphs;

[0025] The optimization mark sequence corresponding to each subgraph is the optimization mark subsequence that has an important influence on the optimization effect.

[0026] In one embodiment, in step S7, calculating the crowding distance of each particle includes:

[0027] The DBSCAN clustering algorithm is used to cluster the samples in the initial search space and divide all samples into multiple categories;

[0028] For each particle, calculate the distance between the particle and each category, and select the category corresponding to the minimum distance as the particle's neighbor set;

[0029] Calculate the distance between the particle and each sample in the neighbor set, sort the distances from small to large, and calculate the average of the first k distances in the sorting result as the crowding distance of the particle.

[0030] In one embodiment, in step S7, the position and moving speed of each particle are updated based on the crowding distance of each particle and the fitness of each particle, including:

[0031] Determine the direction of particle movement:

[0032] vec(d)=v(d-1)+r1·(pbest(d-1)-x(d-1))+r2·(gbest(d-1)-x(d-1))

[0033] Wherein, vec(d) is the moving direction of particle d in the current iteration, v(d-1) is the moving speed of particle d-1 in the previous iteration, pbest(d-1) is the individual optimal position in the previous iteration d-1, x(d-1) is the position of particle d-1 in the previous iteration, gbest(d-1) is the global optimal position in the previous iteration d-1; r1 and r2 are the individual optimal coefficient and the global optimal coefficient respectively;

[0034] Determine how fast a particle moves:

[0035]

[0036] Where v(d) is the moving speed of the particle updated in the current iteration d, cd(d) is the crowding distance of the particle in the current iteration d, and cur_score is the fitness of the particle in the previous iteration d-1;

[0037] Update the particle's position:

[0038] x(d)=x(d-1)+v(d)

[0039] Among them, x(d) is the position of the particle updated in the current iteration d, and x(d-1) is the position of the particle in the previous iteration d-1.

[0040] In a second aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned compiler automatic tuning method based on discrete particle swarm algorithm is implemented.

[0041] In a third aspect, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the above-mentioned automatic compiler tuning method based on discrete particle swarm algorithm.

[0042] Compared with the prior art, this application has the following beneficial effects:

[0043] 1. This application improves the particle swarm algorithm so that it can adapt to the discrete solution space of compiler optimization problems, and proposes a speed update strategy that combines crowding distance and fitness. By evaluating the crowding distance and fitness value of the solution sample in the area where the current particle is located, the particle movement speed is dynamically balanced to select the best optimization sequence, thereby improving the performance of the compiler during automatic tuning; it solves the premature convergence problem of many current discrete particle swarm algorithms and the problem of insufficient global search capability when the problem dimension is high, and significantly improves the tuning effect of compiler automatic tuning.

[0044] 2. This application has good scalability and can find optimization strategies suitable for most programs in a complex and changeable optimization environment. It can also improve the running performance of source programs more than that of the traditional optimized methods, and has broader application prospects and market potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present application may be better understood by referring to the following description given in conjunction with the accompanying drawings, which together with the following detailed description are included in this specification and form a part of this specification. In the drawings:

[0046] Figure 1 A flowchart of a compiler automatic tuning method based on a discrete particle swarm algorithm according to an embodiment of the present application is shown;

[0047] Figure 2 The figure shows the acceleration result of the automatic tuning result of the method of the present application compared with O3;

[0048] Figure 3 The figure shows the speedup result of the program as the number of iterations increases. DETAILED DESCRIPTION

[0049] The exemplary embodiments of the present application will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many implementation-specific decisions can be made in the process of developing any such actual embodiments in order to achieve the specific goals of the developer, and these decisions may vary from embodiment to embodiment.

[0050] It is also necessary to explain here that, in order to avoid obscuring the present application due to unnecessary details, only the device structure closely related to the scheme according to the present application is shown in the drawings, while other details that are not closely related to the present application are omitted.

[0051] It should be understood that the present application is not limited to the described implementation forms due to the following description with reference to the accompanying drawings. In this article, where feasible, the embodiments can be combined with each other, features between different embodiments can be replaced or borrowed, and one or more features can be omitted in one embodiment.

[0052] The present application embodiment provides a compiler automatic tuning method based on discrete particle swarm algorithm. Figure 1 A flowchart of a method for automatic compiler tuning based on a discrete particle swarm algorithm according to an embodiment of the present application is shown. Referring to the figure, the method includes:

[0053] Step S1, generating an initial search space; the samples in the initial search space are a set of feature vectors corresponding to the optimized marker sequence.

[0054] The compiler's optimization flag set O is expressed as:

[0055] O={O1,O2,…O m}

[0056] Each element in the optimization flag set O represents an optional optimization flag.

[0057] Multiple optimization flag sequences are randomly generated according to the compiler's optimization flag set; each optimization flag sequence S can be expressed as S = {S1, S2, ... S i …S n}, where S i ∈O, is the i-th optimization flag in the optimization flag sequence S.

[0058] Each optimization flag sequence compiles the source program into an intermediate representation set; each optimization flag sequence corresponds to an intermediate representation, thus completing the conversion of the optimization sequence to the intermediate representation. The intermediate representation set IRSet is represented as:

[0059] IRSet={I1,I2,…,I n}

[0060] Extract feature vectors from the intermediate representation set and generate a feature vector set Vec = {V1, V2, ..., V n};

[0061] The set of feature vectors corresponding to all optimized sign sequences constitutes the initial search space. This completes the mapping of optimized sign sequences to feature vectors, and converts the problem of finding an optimal optimized sign sequence into a problem of finding the feature vector of the optimal intermediate representation.

[0062] Step S2, randomly select K samples from the initial search space, and determine the execution speedup ratio corresponding to each sample; construct a training set based on the K samples and the execution speedup ratio, and train a multilayer perceptron (MLP), i.e., a cost model, based on the training set.

[0063] Specifically, the execution speedup ratio corresponding to each sample is determined, including:

[0064] Compile the source program into a binary executable file based on the optimization flag sequence corresponding to each sample; execute the executable file to obtain the execution time; determine the execution speedup ratio based on the execution time and the execution time after optimization with the highest optimization level O3 provided by the compiler. Here, the execution speedup ratio can be positioned as the ratio of the execution time to the execution time after optimization with the highest optimization level O3 provided by the compiler, for example.

[0065] Step S3, in the current iteration, for each particle, in the initial search space, select a sample closest to the particle as the sample to be evaluated; input the sample to be evaluated into the multilayer perceptron to obtain the predicted speedup ratio of the sample to be evaluated.

[0066] Here, if the current iteration is the first iteration, you need to set the number of particles and initialize the position and moving speed of each particle. By calculating the distance between the position of each sample in the initial search space and the position of the particle, select the sample closest to the particle as the sample to be evaluated.

[0067] Step S4, sorting all predicted speedup ratios from large to small, and selecting the samples to be evaluated corresponding to the first three predicted speedup ratios in the sorting results as samples with better prediction effects.

[0068] Step S5, among the samples with better prediction effect, select the sample with the largest execution acceleration ratio as the best sample for the current iteration; determine whether the current iteration reaches the maximum number of iterations, if so, output the best sample, if not, execute step S6.

[0069] Here, the optimization flag sequence corresponding to the best sample output is the best optimization flag sequence. The executable file generated after compiling the source program with the best optimization flag sequence has the best execution performance.

[0070] Step S6, adding the best sample and its corresponding execution acceleration ratio to the training set; determining the optimization marker subsequence that has a significant impact on the optimization effect based on the sample with better prediction effect, and adding the optimization marker subsequence to the initial search space.

[0071] Here, the training set is updated to make the prediction effect of the trained multi-layer perceptron more accurate. The optimized marker subsequence is added to the initial search space to provide more reliable options for the subsequent generation of new samples.

[0072] Step S7, calculating the crowding distance of each particle, and updating the position and moving speed of each particle based on the crowding distance of each particle and the fitness of each particle;

[0073] Step S8, determining whether the current iteration is a search space update round, if so, returning to step S2, if not, returning to step S3. Here, the search space update round can be, for example, a multiple of 10, that is, returning to step S2 after every 10 iterations to reselect samples from the updated initial search space.

[0074] This embodiment improves the particle swarm algorithm so that it can adapt to the discrete solution space of the compiler optimization problem, and proposes a speed update strategy that combines crowding distance and fitness. By evaluating the crowding distance and fitness value of the solution sample in the area where the current particle is located, the movement speed of the particle is dynamically balanced to select the best optimization sequence, thereby improving the performance of the compiler automatic tuning; it solves the premature convergence problem of many current discrete particle swarm algorithms and the problem of insufficient global search capability when the problem dimension is high, and significantly improves the tuning effect of the compiler automatic tuning.

[0075] In one embodiment, in step S6, determining an optimization marker subsequence that has a significant impact on the optimization effect based on samples with better prediction effects includes:

[0076] According to the optimization mark sequence corresponding to the samples with better prediction effect, a directed weighted graph is constructed; the nodes of the directed weighted graph are the optimization marks, the directed edges are the order relationship of the optimization marks in the sequence, and the weight of the directed edges is the number of times the order relationship occurs;

[0077] In a directed weighted graph, directed edges whose weights do not meet the set value are removed multiple times to obtain multiple subgraphs; here, directed edges with lower weights can be removed, for example, directed edges with weight 1 are removed to obtain a subgraph; and directed edges with weights 1 and 2 are removed to obtain a second subgraph. The set value can be set according to actual conditions and is not specifically limited.

[0078] The optimization mark sequence corresponding to each subgraph is the optimization mark subsequence that has an important influence on the optimization effect.

[0079] In one embodiment, in step S7, calculating the crowding distance of each particle includes:

[0080] The DBSCAN clustering algorithm is used to cluster the samples in the initial search space and divide all samples into multiple categories;

[0081] For each particle, calculate the distance between the particle and each category, and select the category corresponding to the minimum distance as the particle's neighbor set;

[0082] Calculate the distance between the particle and each sample in the neighbor set, sort the distances from small to large, and calculate the average of the first k distances in the sorting result as the particle's crowding distance. The specific formula is as follows:

[0083]

[0084] Among them, cd(i) is the crowding distance of particle i, and j is the sample number in the neighbor set of particle i.

[0085] In one embodiment, in step S7, the position and moving speed of each particle are updated based on the crowding distance of each particle and the fitness of each particle, including:

[0086] Determine the direction of particle movement:

[0087] vec(d)=v(d-1)+r1·(pbest(d-1)-x(d-1))+r2·(gbest(d-1)-x(d-1))

[0088] Wherein, vec(d) is the moving direction of particle d in the current iteration, v(d-1) is the moving speed of particle d-1 in the previous iteration, pbest(d-1) is the individual optimal position in the previous iteration d-1, x(d-1) is the position of particle d-1 in the previous iteration, gbest(d-1) is the global optimal position in the previous iteration d-1; r1 and r2 are the individual optimal coefficient and the global optimal coefficient respectively; here, gbest(d-1) refers to the position of the best sample output in the previous iteration d-1.

[0089] Determine how fast a particle moves:

[0090]

[0091] Where v(d) is the moving speed of the particle updated in the current iteration d, cd(d) is the crowding distance of the particle in the current iteration d, and cur_score is the fitness of the particle in the previous iteration d-1;

[0092] Update the particle's position:

[0093] x(d)=x(d-1)+v(d)

[0094] Among them, x(d) is the position of the particle updated in the current iteration d, and x(d-1) is the position of the particle in the previous iteration d-1.

[0095] In order to further verify the effectiveness of the method of this application, the following experimental analysis was carried out.

[0096] The method of the present application is compared with the optimization of the highest optimization option O3 level that comes with the LLVM compiler. The optimal optimization option sequence found by the method of the present application and the O3 option are used to compile the source program. Table 1 shows the list of the source program, and then the running time of the executable file generated is compared. Through the -mllvm-debug-pass=Arguments instruction, it can be known that the O3 optimization flag sequence uses a total of 256 optimization options (options are repeated), so the length of the generated sequence is adjusted to be the same as the O3 sequence. And each time the runtime performance is evaluated, each executable file is executed three times and the average is taken to eliminate the influence of accidental events.

[0097] Figure 2 The figure shows the acceleration result of the automatic tuning result of the method of the present application compared with O3, where the horizontal axis is different source programs and the vertical axis is the acceleration of the running time during execution compared with O3. It can be seen from the figure that the tuning method of the present application can find an optimization sequence with very good effect for most programs, where the highest acceleration reaches 1.5 times, but the lowest is less than 1.1. It can be seen that although the tuning method of the present application can significantly improve performance in some cases.

[0098] Table 1

[0099]

[0100] Figure 3 The figure shows the speedup ratio of the program as the number of iterations increases. As the number of iterations increases, the speedup ratio of most programs increases. For example, the speedup ratio of program P1 is 1.08 at 30 iterations, and it increases to 1.22 at 50 iterations. Similarly, the speedup ratio of program P6 increases from 1.11 to 1.38.

[0101] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned compiler automatic tuning method based on discrete particle swarm algorithm is implemented.

[0102] An embodiment of the present application provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the above-mentioned compiler automatic tuning method based on discrete particle swarm algorithm is implemented.

[0103] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A compiler automatic tuning method based on discrete particle swarm algorithm, characterized in that: include: Step S1, generating an initial search space; The samples in the initial search space are a set of feature vectors corresponding to the optimized marker sequence; Step S2, randomly selecting K samples from the initial search space, and determining the execution speedup ratio corresponding to each of the samples; constructing a training set based on the K samples and the execution speedup ratio, and training a multilayer perceptron based on the training set; Step S3, in the current iteration, for each particle, in the initial search space, select a sample closest to the particle as a sample to be evaluated; input the sample to be evaluated into the multilayer perceptron to obtain a predicted speedup ratio of the sample to be evaluated; Step S4, sorting all predicted speedup ratios from large to small, and selecting the samples to be evaluated corresponding to the first three predicted speedup ratios in the sorting results as samples with better prediction effects; Step S5, among the samples with better prediction effects, select the sample with the largest execution speedup ratio as the best sample for the current iteration; determine whether the current iteration reaches the maximum number of iterations, if so, output the best sample, if not, execute step S6; Step S6, adding the best sample and its corresponding execution acceleration ratio to the training set; determining an optimization marker subsequence that has a significant impact on the optimization effect based on the sample with better prediction effect, and adding the optimization marker subsequence to the initial search space; Step S7, calculating the crowding distance of each particle, and updating the position and moving speed of each particle based on the crowding distance of each particle and the fitness of each particle; Step S8, determine whether the current iteration is a round of search space update, if so, return to step S2, if not, return to step S3.

2. The method according to claim 1, characterized in that in, In step S1, an initial search space is generated, including: Randomly generate multiple optimization flag sequences according to the compiler's optimization flag set; Each of the optimization flag sequences compiles a source program into an intermediate representation set; Extracting feature vectors from the intermediate representation set to generate a feature vector set; The feature vector set corresponding to all optimized marker sequences constitutes the initial search space.

3. The method according to claim 1, characterized in that in, In step S2, determining the execution speedup ratio corresponding to each sample includes: Compile the source program into a binary executable file based on the optimization flag sequence corresponding to each sample; execute the executable file to obtain the execution time; determine the execution acceleration ratio based on the execution time and the execution time after optimization at the highest optimization level provided by the compiler.

4. The method according to claim 1, characterized in that in, In step S6, determining an optimization marker subsequence that has a significant impact on the optimization effect based on the samples with better prediction effect includes: According to the optimization mark sequence corresponding to the sample with better prediction effect, a directed weighted graph is constructed; the nodes of the directed weighted graph are the optimization marks, the directed edges are the order relationship of the optimization marks in the sequence, and the weights of the directed edges are the number of times the order relationship appears; In the directed weighted graph, directed edges whose weights do not meet set values ​​are removed multiple times to obtain multiple subgraphs; The optimization mark sequence corresponding to each of the subgraphs is an optimization mark subsequence that has an important influence on the optimization effect.

5. The method according to claim 1, characterized in that in, In step S7, the crowding distance of each particle is calculated, including: Using the DBSCAN clustering algorithm, clustering the samples in the initial search space, and dividing all samples into multiple categories; For each particle, calculate the distance between the particle and each category, and select the category corresponding to the minimum distance as the neighbor set of the particle; The distance between the particle and each sample in the neighbor set is calculated, and the distances are sorted from small to large, and the average of the first k distances in the sorting result is calculated as the crowding distance of the particle.

6. The method according to claim 1, characterized in that in, In step S7, the position and moving speed of each particle are updated based on the crowding distance of each particle and the fitness of each particle, including: Determine the direction of particle movement: vec(d)=v(d-1)+r1·(pbest(d-1)-x(d-1))+r2·(gbest(d-1)-x(d-1))wherein, vec(d) is the moving direction of particle d in the current iteration, v(d-1) is the moving speed of particle d-1 in the previous iteration, pbest(d-1) is the individual optimal position in the previous iteration d-1, x(d-1) is the position of particle d-1 in the previous iteration, and gbest(d-1) is the global optimal position in the previous iteration d-1; r1 and r2 are the individual optimal coefficient and the global optimal coefficient respectively; Determine how fast a particle moves: Where v(d) is the moving speed of the particle updated in the current iteration d, cd(d) is the crowding distance of the particle in the current iteration d, and cur_score is the fitness of the particle in the previous iteration d-1; Update the particle's position: x(d)=x(d-1)+v(d) Among them, x(d) is the position of the particle updated in the current iteration d, and x(d-1) is the position of the particle in the previous iteration d-1.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the compiler automatic tuning method based on discrete particle swarm algorithm according to any one of claims 1 to 6 is implemented.

8. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, implements the compiler automatic tuning method based on discrete particle swarm algorithm as described in any one of claims 1 to 6.

Images